A method for full-process tracing and performance analysis of trace_id in a distributed service chain.

Abstract

This invention relates to the field of service link tracing technology, specifically disclosing a method for full-process tracing and performance analysis of distributed service links using trace_id. The method includes: generating a globally unique TraceID at the request entry point and establishing a context object; embedding the TraceID into the call request header via HTTP or RPC protocol to ensure cross-service transmission continuity; for asynchronous parallel tasks, generating child TraceIDs using the "parent TraceID.sequence number" rule and registering them to the asynchronous task registry to establish hierarchical association; standardizing log format, retrieving and sorting log records from the distributed storage system based on the TraceID to reconstruct the complete call link; collecting performance data such as time consumption and status at key nodes; calculating the latency transmission coefficient and net root cause contribution to identify the true root cause of performance bottlenecks. This invention achieves cross-service and cross-node request association and precise bottleneck location, improving the observability and diagnostic efficiency of distributed AI systems.

Description

Technical Field

[0001] This invention relates to the field of service link tracing technology, and in particular to a method for full-process tracing and performance analysis of trace_id in distributed service links. Background Technology

[0002] With the deepening application of artificial intelligence technology in various vertical industries, distributed AI service architecture has become the mainstream technology choice to support intelligent business scenarios. Taking the education technology field as an example, the AI ​​Agent workflow system, as the core hub connecting user intent and AI service capabilities, needs to integrate multiple sub-modules to collaboratively complete complex tasks. Typical business processes involve multiple stages such as script generation, intent recognition and interaction strategy execution, and associated calls to material library resources. These modules are usually deployed as independent service nodes, completing the entire processing chain of user requests through inter-service calls.

[0003] The widely adopted distributed tracing solutions in the industry are based on TraceID technology. The core idea is to assign a globally unique identifier to each user request and transmit this identifier between the service nodes through which the request flows, thereby correlating and integrating scattered log data. For example, the "A Jaeger-based Full-Link Log Tracing Method" (Publication No.: CN121644344A) applied for by China Pacific Property Insurance Co., Ltd. achieves cross-service log correlation by generating a globally unique Trace ID at the starting point of the transaction link and embedding it into each log entry. The "A Distributed Link Tracing, Analysis Method and Server, Global Scheduler" (Publication No.: CN108228432B) applied for by Alibaba Group Holding Co., Ltd. reduces the data storage and analysis costs of traditional methods through local event storage and on-demand analysis in a distributed cluster. However, the above solutions are mainly designed for inter-service communication scenarios in synchronous call links, and their premise is that requests are transmitted synchronously between service nodes. In the AI ​​Agent workflow, the execution of a large amount of business logic exhibits significant asynchronous characteristics. When a request is broken down into multiple parallel subtasks, the traditional single TraceID scheme results in a large amount of log data of each parallel subtask being intertwined and written in parallel. Simple timestamp sorting cannot accurately restore the call sequence and dependency relationship between each subtask.

[0004] In asynchronous task scheduling scenarios, current tracing solutions generally employ a single-threaded TraceID transmission mechanism. When a request executes an asynchronous task via a thread pool, the parent-child Span relationship is easily lost, leading to a break in the tracing chain. At a deeper level, existing technologies still have gaps in the following areas: First, they cannot verify whether business rules such as material priority being higher than guiding scripts are correctly executed at runtime; second, they cannot distinguish whether performance bottlenecks are due to excessive time consumption of a single subtask or due to unexpected serialization between parallel subtasks; third, they cannot accurately separate the proportion of actual processing time of downstream services and network / scheduling waiting time in the total asynchronous call time.

[0005] Existing distributed tracing technologies face several technical bottlenecks when dealing with complex scenarios unique to AI Agent systems, such as asynchronous task scheduling, parallel subtask execution, and cross-service data transfer. These bottlenecks include broken TraceID transmission chains, difficulties in log time-series correlation, inaccurate collection of cross-module performance metrics, inability to automatically verify the correctness of business rules, and inability to quantitatively assess the health of parallel scheduling. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide a method for full-process tracing and performance analysis of trace_id in distributed service links, in order to solve the shortcomings of existing distributed tracing technologies in distributed AI Agent systems (especially AI service links in the education field with complex characteristics such as asynchronous task scheduling, parallel subtask execution, and cross-service data transmission), such as broken TraceID transmission chains, difficulty in log time sequence correlation, difficulty in accurately collecting cross-module performance indicators, and inability to verify business priority logic and parallel scheduling health.

[0007] This invention provides a method for full-process tracing and performance analysis of trace_id in a distributed service chain, including:

[0008] S1: Generate a globally unique TraceID at the user request entry point and establish a request context object associated with that TraceID;

[0009] S2: In each service call from the AI-Service main service to the downstream AI-Agent sub-module, the TraceID identifier is embedded in the HTTP request header or RPC call context;

[0010] S3: For asynchronous task scheduling scenarios in AI-Agent systems, assign a sub-TraceID to each asynchronous execution unit and establish a hierarchical relationship between the sub-TraceID and the parent TraceID;

[0011] S4: Connect log records scattered across different service nodes based on a unified TraceID identifier to reconstruct the complete call chain of the request;

[0012] S5: Collect time-consuming data, call status, and resource usage information at key nodes of service calls;

[0013] S6: Calculate the response latency and resource consumption ratio of each stage based on the collected performance data to pinpoint the specific source of performance bottlenecks.

[0014] Preferably, generating a globally unique TraceID specifically includes:

[0015] A 128-bit globally unique identifier is generated using the UUIDv4 algorithm;

[0016] The current system timestamp is encoded to the millisecond level and then appended to the UUID string to form a hybrid TraceID;

[0017] The service node identifier that generates the TraceID is encoded into a fixed-length string and appended to the end of the hybrid TraceID to form a complete TraceID string with self-describing characteristics.

[0018] Preferably, S3 includes:

[0019] S31: When the AI-Service main service receives a user request and generates a globally unique parent TraceID, it determines whether the processing flow of the request needs to be split into multiple parallel asynchronous execution units; if so, it enters the asynchronous task scheduling branch.

[0020] S32: For each asynchronous execution unit to be created, generate a unique sub-TraceID;

[0021] S33: After the sub-TraceID is generated, the asynchronous task scheduler binds the sub-TraceID to the corresponding asynchronous execution unit and writes the sub-TraceID into the execution context of the asynchronous task, so that all logs and performance data output by the asynchronous task during execution carry its sub-TraceID.

[0022] S34: The generated child TraceID, its corresponding parent TraceID, task type, creation timestamp, and task status are recorded as a single record and written to the asynchronous task registry. The asynchronous task registry uses the parent TraceID as the primary key to build an inverted index, which supports quick querying of the list of all child TraceIDs under the parent TraceID and the detailed status of each child task.

[0023] S35: When any asynchronous execution unit completes its execution, the execution result and final task status of the unit are sent back to the asynchronous task registration center, and the task status of the corresponding sub-TraceID is updated to "completed" or "abnormal".

[0024] S36: In the subsequent full-link performance analysis and link restoration process, the link reconstruction engine queries the asynchronous task registry based on the parent TraceID to obtain a list of all associated child TraceIDs. Then, combined with the records of each child TraceID in the log system and performance data storage table, the complete parallel call sequence of the asynchronous task is restored according to the timestamp and parallel execution interval.

[0025] Preferably, the step of assigning a child TraceID to each asynchronous execution unit and establishing a hierarchical association between the child TraceID and the parent TraceID includes:

[0026] Get the complete parent TraceID string as the prefix character, append an English period "." as the separator, and then append an incrementing sequence number starting from 1 to generate the child TraceID string;

[0027] The asynchronous task registry records all created child TraceIDs and their corresponding task execution statuses.

[0028] The asynchronous task registry uses the parent TraceID as the primary key to create an index, supporting queries for a list of all child task TraceIDs under the parent TraceID.

[0029] Preferably, the collection of time-consuming data, call status, and resource usage information at key nodes of the service call specifically includes:

[0030] At the request receiving node, collect the request arrival time, service interface name, and request parameter characteristics; at the service processing start node, collect the processing start time and current service load status.

[0031] Collect information such as the initiation time of downstream service calls, target service address, and call protocol type from external calling nodes;

[0032] Collect downstream service response time, response status code, and response data size from the external call return node;

[0033] Collect processing completion time, processing result status, and exception information at the service processing completion node;

[0034] When the AI-Agent submodule initiates a call to the media library service, it also collects the initiation time of the media query request, the API interface path of the media library, the query condition parameters, the response time of the media library service, the number of media records returned, and the completion time of the media data being transmitted to the speech assembly module.

[0035] Preferably, S6 includes:

[0036] Calculate the total time of the entire link The It equals the processing completion time of the request entry point minus the request arrival time;

[0037] Calculate the time taken for the i-th service node or external call. ;

[0038] Calculate each occupy proportion ,in ;

[0039] Set bottleneck determination threshold ,when When the i-th stage is identified, it is marked as the performance bottleneck.

[0040] The The default value is 0.2. When the time spent in a certain link accounts for more than 20% of the total time spent in the entire link, that link is determined to be a performance bottleneck.

[0041] Preferably, when the AI-Service main service receives a user request and generates a globally unique parent TraceID, it determines whether the processing flow of the request needs to be split into multiple parallel asynchronous execution units; if so, the asynchronous task scheduling branch includes:

[0042] After the AI-Service main service receives a user request and generates a globally unique parent TraceID, it extracts the business type identifier of the request. The business type identifier includes a script generation request from the supervisor module, an intent recognition request from the demand mining module, or a material library association call request.

[0043] Based on the business type identifier, query the preset task splitting rule table;

[0044] The task splitting rule table predefines a subtask partitioning strategy for each business type. The subtask partitioning strategy includes: the number of subtasks, the execution logic type of each subtask, and the dependencies between subtasks.

[0045] When the number of subtasks corresponding to the business type identifier is greater than 1, it is determined that it needs to be split into multiple parallel asynchronous execution units; otherwise, it is determined that no splitting is required and synchronous execution mode is directly adopted.

[0046] For requests that are determined to need to be split, the business parameters in the request are further analyzed, and the number and parallelism of subtasks are dynamically adjusted according to the business parameters.

[0047] After the splitting judgment is completed, if splitting is required, the list of split subtasks and their respective execution logic are passed to the asynchronous task scheduler, triggering the sub-TraceID generation and registration process in steps S32 to S36.

[0048] If no splitting is required, the parent TraceID will continue to be used for synchronization.

[0049] Preferably, S4 includes:

[0050] S41: Deploy a log formatting unit on each service node. This unit constructs each log record as a JSON object according to a unified log format specification.

[0051] S42: Deploy a log storage agent unit on each service node. The agent unit uses an asynchronous batch writing method to temporarily store the JSON format log records in a memory buffer. When the number of log records in the buffer reaches a preset threshold or the time interval between the last batch writing reaches a preset threshold, all log records in the buffer are sent to the distributed log storage system in batches.

[0052] S43: When it is necessary to restore the complete call chain of a certain request, the link reconstruction engine receives the target TraceID, which is input by the user or specified by the system, and records it as target_trace_id;

[0053] S44: Initiate an exact match query to the distributed log storage system through the link reconstruction engine, with the query condition "TraceID=target_trace_id", and obtain all log records containing TraceID to form the original log set;

[0054] S45: The original log set is sorted in ascending order by the timestamp field using the link reconstruction engine to generate a log sequence arranged in chronological order;

[0055] S46: By traversing the log sequence through the link reconstruction engine, based on the service node name in each log record and the call identifier in the log content, the entry, exit and internal processing nodes of the request on each service node are identified, and a call sequence diagram with time as the horizontal axis and service nodes as the vertical axis is constructed.

[0056] S47: For asynchronous task scenarios that include child TraceIDs, the link reconstruction engine queries the asynchronous task registry based on the parent TraceID to obtain a list of all associated child TraceIDs;

[0057] S48: Execute steps S44 to S46 for the parent TraceID and each child TraceID respectively to obtain multiple sets of log sequences;

[0058] S49: Based on the timestamps of each log sequence, align the execution time periods of the parent task and each child task in parallel to restore the complete parallel call sequence and output it in the form of a Gantt chart.

[0059] Preferably, the specific sources of the positioning performance bottleneck include:

[0060] For any adjacent upstream or downstream link in a single request call chain, collect the total processing time of the upstream link A. And the pure processing time of downstream stage B Historical sample data;

[0061] in This includes the time taken for A to call B and wait for its return. This does not include the time spent by B calling its downstream processes;

[0062] Based on the historical sample data, calculate the delay propagation coefficient between upstream link A and downstream link B. ;

[0063] The calculated delay propagation coefficient As a measure of the intensity of time-consuming fluctuations transmitted from downstream link B to upstream link A:

[0064] when When the time consumption is close to 1, it is determined that the abnormal time consumption of upstream A is mainly caused by the time consumption fluctuation of downstream B; when When the value is close to 0, the causal relationship between the fluctuations in the time consumption of the two is considered to be weak.

[0065] Among them, the The calculation formula is:

[0066]

[0067] In the formula, the for and covariance, for standard deviation for The standard deviation.

[0068] Preferably, the specific sources of the positioning performance bottleneck also include:

[0069] Get the actual time spent in upstream step A of the current request. and the actual time spent in downstream link B And query the historical median time of stage A and stage B from the historical database. and ;

[0070] Through the delay propagation coefficient Calculate the net root cause contribution of downstream stage B relative to upstream stage A. ;

[0071] when When >0, B is identified as a candidate root cause node; when When =0, it is determined that the abnormal time consumption of B is entirely caused by upstream transmission or normal fluctuations, and should not be marked as the root cause.

[0072] in, The calculation formula is:

[0073]

[0074] In the formula, This indicates the degree to which the time taken in downstream segment B deviates from its historical normal level. This indicates the degree of deviation in the time consumption of upstream link A. Used to suppress root cause misjudgments caused by strong transmission relationships;

[0075] when At that time, the pass Replacement, the It is a very small positive number.

[0076] The method for full-process tracing and performance analysis of trace_id in a distributed service chain provided by this invention has the following beneficial effects:

[0077] This invention further utilizes a causal analysis model based on the delay propagation coefficient and net root cause contribution to accurately distinguish between two types of performance bottlenecks: "delay caused by downstream propagation" and "abnormal self-processing." This avoids misjudgments by traditional fixed threshold methods or simple time-consuming ranking methods, and significantly improves the accuracy and interpretability of bottleneck root cause localization.

[0078] Meanwhile, the unified log format specification, material library call special collection, and dynamic / multi-factor bottleneck scoring method based on time consumption ratio defined in this invention enable the system to quantify the contribution ratio of AI-specific modules such as material library call, script generation, and intent recognition to the overall response latency in a fine-grained manner, thereby providing quantitative data support for priority logic verification and parallel scheduling health assessment at the business level.

[0079] This invention outputs end-to-end tracing results in the form of visualized call topology, time-series Gantt charts, and causal arrow diagrams, which significantly improves the observability, fault location efficiency, and scientific rigor of performance optimization decisions in distributed AI systems. Attached Figure Description

[0080] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, and these are all within the protection scope of the present invention.

[0081] Figure 1 This is a flowchart of a method for full-process tracing and performance analysis of a distributed service chain using trace_id. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In the description of the present invention, it should be understood that the terms "center," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements, but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, the element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Where there is no conflict, embodiments of the present invention and the various features thereof can be combined with each other, all of which are within the scope of protection of the present invention.

[0083] Example 1

[0084] Please see Figure 1 This invention provides a method for full-process tracing and performance analysis of trace_id in a distributed service chain, including:

[0085] S1: Generate a globally unique TraceID at the user request entry point and establish a request context object associated with that TraceID;

[0086] Specifically, a TraceID generator component is deployed at the request entry point of the AI-Service main service. When a user request, for example, from the tutoring module of the Education AIAgent system, arrives at this entry point, the TraceID generator is triggered. This generator contains three sub-modules: a UUID generation unit, a timestamp encoding unit, and a service identifier encoding unit. These three modules work together to generate a complete TraceID string with self-describing characteristics.

[0087] Specifically, the UUID generation unit calls the secure random number generator provided by the operating system to generate 16 bytes (128 bits) of random data, and sets the version number and variant flag according to the UUIDv4 specification to form a standard UUID string, such as "a1b2c3d4-e5f6-7890-abcd-ef1234567890". The timestamp encoding unit obtains the current system timestamp, accurate to the millisecond level, such as "1699500000123", converts the timestamp into a decimal string, and concatenates it before the UUID string to form the mixed TraceID "1699500000123-a1b2c3d4-e5f6-7890-abcd-ef1234567890".

[0088] The service identifier encoding unit obtains the service node identifier information that generated the TraceID. This identifier information includes the service name, service IP address, and service port number. It converts this information into a fixed-length string using predefined encoding rules, such as taking the first three letters of the service name, the last segment of the IP address, and the numbers from the port number; for example, "edu1008080". This string is then appended to the end of the mixed TraceID to form the complete TraceID: "1699500000123-a1b2c3d4-e5f6-7890-abcd-ef1234567890-edu1008080". The total length of this TraceID string is controlled to be no more than 64 characters, facilitating efficient transmission in HTTP headers and RPC metadata.

[0089] While generating the TraceID, the system creates a request context object. This context object is an in-memory data structure that contains at least the following fields:

[0090] The `traceId` stores the complete TraceID string generated above; `spanId` is initialized to the first 16 bits of the TraceID, used to identify the current request span; `parentSpanId` is set to empty for the entry request; `requestStartTime` records the timestamp of the request arrival; and `serviceName` is the name of the current service node. This context object is bound to the processing thread of the current request, and through thread-local variables or asynchronous context passing mechanisms, all subsequent processing logic of the same request can access this context object. The context object also provides methods such as `getTraceId()`, `setTraceId()`, and `createChildContext()`, which facilitate the derivation of new contexts when calling downstream services or creating subtasks.

[0091] In this embodiment, a globally unique TraceID is generated through S1, and a request context object that can run through the entire service chain is also established, providing a unified identification basis and operating environment for subsequent TraceID transmission, log association and performance collection.

[0092] S2: In each service call from the AI-Service main service to the downstream AI-Agent sub-module, the TraceID identifier is embedded in the HTTP request header or RPC call context;

[0093] In a specific embodiment of the present invention, step S2, "embedding the TraceID identifier in the HTTP request header or RPC call context in each service call from the AI-Service main service to the downstream AI-Agent sub-module," is achieved by deploying a TraceID transmission middleware. This middleware includes two implementation forms: an HTTP interceptor and an RPC interceptor, which are suitable for service call scenarios with different communication protocols.

[0094] TraceID passing in HTTP call scenarios:

[0095] When the main AI-Service needs to call the downstream AI-Agent submodule via HTTP, such as calling the HTTP interface of the media library service, a request interceptor is deployed on the HTTP client side of the main service. This interceptor automatically obtains the TraceID string from the current request context object before the request is sent by rewriting the HTTP client's sending method, and writes it into the "X-Trace-ID" field of the HTTP request header.

[0096] On the downstream AI-Agent submodule side, a corresponding HTTP request receiving interceptor is also deployed. When the submodule receives an HTTP request, this interceptor executes at the very beginning of the request processing chain, automatically extracting the value of the "X-Trace-ID" field from the HTTP request header. If the field exists and is not empty, it is set in the context environment object of the current request; if it does not exist (e.g., for the first request or in a scenario where the TraceID has not been transmitted), the submodule can choose to generate a new TraceID or reject the request based on its configuration. Through this bidirectional interception mechanism, it is ensured that the TraceID can be transmitted completely and continuously along the HTTP call chain.

[0097] TraceID passing in RPC call scenarios:

[0098] For scenarios involving inter-service calls using an RPC framework, this invention provides RPC interceptor implementations adapted to different frameworks.

[0099] Taking the gRPC framework as an example: When the AI-Service main service initiates a gRPC call as a client, a client interceptor is implemented through the "ClientInterceptor" interface provided by gRPC. In the "interceptCall" method, this interceptor retrieves the TraceID from the current request context and injects it into the gRPC call's "Metadata". Specifically, it creates a key named "trace-id" with the value of the TraceID string, and adds it to the call metadata using the "put" method of "Metadata". The downstream AI-Agent submodule, acting as the gRPC server, deploys a server-side interceptor. When a request arrives, it extracts the value corresponding to the "trace-id" key from the received "Metadata" and sets it into the current request's "Context" for use by subsequent business logic.

[0100] Taking the Dubbo framework as an example: The AI-Service main service uses the "setAttachment" method provided by Dubbo's "RpcContext" to attach the TraceID as attachment information to the RPC call, with the key "trace-id" and the value being the current TraceID string. When the downstream service receives the call, it extracts the TraceID using the "RpcContext.getContext().getAttachment("trace-id")" method and injects it into the local request context.

[0101] When the AI-Service main service, acting as a downstream service, receives a TraceID from an upstream service, the TraceID propagation middleware not only sets this TraceID as the "traceId" field of the current request context, but also generates a sub-span identifier based on this TraceID to identify the processing segment of the current service node. Specifically, the context object records two fields: "parentTraceId" stores the original TraceID passed from the upstream, and "currentTraceId" stores the TraceID used by the current service node when continuing to propagate downstream. By default, it is the same as the parent TraceID to ensure that the same root TraceID is used throughout the entire chain. When initiating new calls downstream, the middleware will uniformly inject "currentTraceId" into the request header or RPC metadata, thereby maintaining the continuity and consistency of TraceID propagation.

[0102] If the TraceID in the current request context is found to be empty or invalid during the call, the TraceID passing middleware will trigger a fallback strategy:

[0103] The system automatically invokes the TraceID generator to generate a new TraceID as a replacement, and logs a warning message indicating "TraceID missing, replacement identifier automatically generated." This mechanism ensures that even if the TraceID transmission chain is unexpectedly broken, the system can still maintain basic tracing capabilities and avoid the loss of the entire call chain due to the missing identifier.

[0104] S3: For asynchronous task scheduling scenarios in AI-Agent systems, assign a sub-TraceID to each asynchronous execution unit and establish a hierarchical relationship between the sub-TraceID and the parent TraceID;

[0105] Furthermore, S3 includes:

[0106] S31: When the AI-Service main service receives a user request and generates a globally unique parent TraceID, it determines whether the processing flow of the request needs to be split into multiple parallel asynchronous execution units; if so, it enters the asynchronous task scheduling branch.

[0107] Furthermore, after the AI-Service main service receives a user request and generates a globally unique parent TraceID, it determines whether the processing flow of the request needs to be split into multiple parallel asynchronous execution units; if so, it enters the asynchronous task scheduling branch, including:

[0108] After the AI-Service main service receives a user request and generates a globally unique parent TraceID, it extracts the business type identifier of the request. The business type identifier includes a script generation request from the supervisor module, an intent recognition request from the demand mining module, or a material library association call request.

[0109] Based on the business type identifier, query the preset task splitting rule table;

[0110] The task splitting rule table predefines a subtask partitioning strategy for each business type. The subtask partitioning strategy includes: the number of subtasks, the execution logic type of each subtask, and the dependencies between subtasks.

[0111] When the number of subtasks corresponding to the business type identifier is greater than 1, it is determined that it needs to be split into multiple parallel asynchronous execution units; otherwise, it is determined that no splitting is required and synchronous execution mode is directly adopted.

[0112] For requests that are determined to need to be split, the business parameters in the request are further analyzed, and the number and parallelism of subtasks are dynamically adjusted according to the business parameters.

[0113] After the splitting judgment is completed, if splitting is required, the list of split subtasks and their respective execution logic are passed to the asynchronous task scheduler, triggering the sub-TraceID generation and registration process in steps S32 to S36.

[0114] If no splitting is required, the parent TraceID will continue to be used for synchronization.

[0115] Specifically, after the AI-Service main service receives a user request and completes step S1, the system first enters the S31 judgment process. The core of this judgment process is to determine whether the processing flow of the current request needs to be split into multiple parallel asynchronous execution units, which includes the following sub-steps:

[0116] Extract business type identifier:

[0117] The system extracts the business type identifier from request parameters or request headers. In AIAgent systems for the education sector, preset business types include, but are not limited to:

[0118] The requests include: the script generation request from the supervisory module, the intent recognition request from the demand mining module, and the material library association call request.

[0119] For example, a script generation request from the supervisor module contains the field "bizType": "supervise_dialogue" in its request body. The system obtains this identifier through JSON parsing.

[0120] Query the task splitting rules table:

[0121] The system maintains a pre-configured task splitting rule table in memory. This table uses business type as the key and subtask splitting strategy as the value. The rule table can be stored in JSON format.

[0122] When the system starts up or before it runs, it loads a mapping table named "Task Splitting Rule Table" from the configuration center or local configuration file. This rule table uses the business type identifier as the index key, with one record corresponding to each business type, containing the task splitting strategy for that type of request.

[0123] Taking the AIAgent system in the education field as an example, the rule table predefines the following common business type splitting strategies:

[0124] For the script generation request in the supervisory module, the rule table records that there are 3 subtasks. The execution logic types of the three subtasks are "material library query", "script template matching" and "sensitive word filtering", respectively. The dependency relationship between the subtasks is "parallel execution", that is, the three subtasks can be initiated at the same time without waiting for each other. At the same time, the record also marks a dynamic parameter list, which includes a "material ID list", indicating that the number of material query subtasks can be dynamically adjusted according to the specific number of materials carried in the request during actual execution.

[0125] For the intent recognition request of the demand mining module, the rule table records that there are 2 subtasks. The execution logic types of the two subtasks are "intent recognition" and "interaction strategy selection" respectively. The dependency relationship between the subtasks is "serial execution", that is, the intent recognition subtask must be completed first, and then the interaction strategy selection subtask is executed according to the recognition result. The dynamic parameter list is empty.

[0126] For material library association call requests, the rule table records that the number of its subtasks is 1, the execution logic type of the subtask is "cached query", the dependency relationship is "none", and the dynamic parameter list is empty.

[0127] When the system needs to determine whether a request needs to be split into multiple parallel asynchronous execution units, it first extracts the business type identifier from the request, and then uses this identifier as the query key to perform an exact match search in the aforementioned rule table. The search process can use a hash mapping method to locate the corresponding splitting strategy record in constant time. If no corresponding record is found, the default strategy is adopted: the number of subtasks is 1, that is, no splitting is performed, and it is processed in synchronous mode.

[0128] In this embodiment, by querying the task splitting rule table, the system can quickly obtain the predefined splitting strategy for the current request, including the number of subtasks, the execution logic type of each subtask, the dependencies between subtasks, and dynamically adjustable parameter information, thereby providing a basis for subsequent splitting decisions and asynchronous scheduling.

[0129] Based on the business type identifier extracted in step A, the system queries the rule table to obtain the corresponding number of subtasks, the execution logic type of each subtask (such as material query, template matching, sensitive word filtering), the dependencies between subtasks (parallel, serial, or no dependency), and a list of dynamically adjustable parameters.

[0130] Determine if splitting is necessary:

[0131] The system reads the "subtask_count" field from the rule table. If the value is greater than 1, it determines that the task needs to be split into multiple parallel asynchronous execution units and enters the asynchronous task scheduling branch; otherwise, it determines that no splitting is needed and directly adopts the synchronous execution mode, continuing to use the parent TraceID for subsequent processing. For example, for "supervise_dialogue", the system determines that splitting is needed; for "material_query", no splitting is needed.

[0132] Dynamically adjust the number of subtasks and the degree of parallelism:

[0133] For requests deemed necessary to be split, the system further parses the business parameters within the request and dynamically adjusts the number and parallelism of subtasks based on the "dynamic_params" defined in the rule table. For example, in the scenario of generating dialogue in the supervisory module, the request parameters might contain an array of "material_ids," specifying multiple material library sources to be queried. The system will dynamically generate a corresponding number of asynchronous material query tasks based on the length of this array; that is, the actual number of subtasks is "len(material_ids)", rather than the fixed value of 3 in the rule table. Simultaneously, the system can set a maximum parallelism limit (e.g., a default maximum of 10 parallel tasks). When the number of dynamic subtasks exceeds this limit, the tasks will be executed in batches.

[0134] Pass the list of subtasks to the asynchronous task scheduler:

[0135] After completing the splitting judgment and dynamic adjustment, if splitting is required, the system will generate a list of subtasks. Each element in the list contains the subtask's type identifier, execution logic, dependencies, and necessary business parameters. This list of subtasks is passed to the asynchronous task scheduler, triggering subsequent processes S32 to S36. If splitting is not required, the asynchronous branch is skipped, and synchronous processing continues in the main thread using the parent TraceID. Simultaneously, the system will log "This request does not require splitting; synchronous mode is used."

[0136] Enabling asynchronous task scheduling branches:

[0137] After receiving the list of subtasks, the asynchronous task scheduler creates an independent execution unit for each subtask. Before execution, the scheduler calls step S32 to generate a unique child TraceID for each subtask, and completes binding, registration, and subsequent state management according to steps S33 to S36. Throughout this process, the parent TraceID always serves as the root identifier of the request, while the child TraceID is used to distinguish between different parallel branches, thus achieving a hierarchical association from the parent task to the child task.

[0138] Taking the script generation request in the supervisor module as an example, when a user clicks the "Generate Script" button on the front end, the request parameters include "bizType=supervise_dialogue" and "material_ids=["id1", "id2"]". The system executes S1 to generate the parent TraceID, assuming it is "1699500000123-a1b2c3d4...-edu1008080". After entering S31, the business type is extracted as "supervise_dialogue", and the rule table is queried to find that the number of basic subtasks is 3, and the dependency relationship is parallel. Then, the length of the "material_ids" array is parsed to be 2, so it is dynamically adjusted: actually creating 2 material query subtasks + 1 template matching subtask + 1 sensitive word filtering subtask, for a total of 4 parallel subtasks. The system passes these 4 subtasks to the asynchronous task scheduler, triggering the generation of sub-TraceIDs and binding them to each subtask. Subsequently, all subtasks are executed concurrently, and the execution results are summarized through the asynchronous task registry center, finally completing the assembly and return of the script.

[0139] S32: For each asynchronous execution unit to be created, generate a unique sub-TraceID;

[0140] Specifically, the complete string of the parent TraceID is used as a prefix. Since the parent TraceID may be quite long, to maintain the readability and manageable length of the child TraceID, the system uses the first 32 characters of the parent TraceID as the prefix. For example, the parent TraceID "1699500000123-a1b2c3d4-e5f6-7890-abcd-ef1234567890-edu1008080" has the first 32 characters "1699500000123-a1b2c3d4-e5f6-78". Then, a period "." is appended as a separator, followed by an integer sequence number starting from 1. This sequence number is unique among all subtasks under the same parent TraceID and increments sequentially according to the creation order of the subtasks. The resulting sub-TraceIDs will take the form of "1699500000123-a1b2c3d4-e5f6-78.1" and "1699500000123-a1b2c3d4-e5f6-78.2", etc. For the four sub-tasks mentioned above, sub-TraceIDs will be generated as ".1", ".2", ".3", and ".4", respectively. While generating the sub-TraceIDs, the system will record the mapping relationship between the sub-task's sequence number and its type, facilitating rapid identification of the sub-task's business meaning during subsequent log analysis.

[0141] S33: After the sub-TraceID is generated, the asynchronous task scheduler binds the sub-TraceID to the corresponding asynchronous execution unit and writes the sub-TraceID into the execution context of the asynchronous task, so that all logs and performance data output by the asynchronous task during execution carry its sub-TraceID.

[0142] Specifically, the system creates an independent execution context object for each subtask. This context object inherits from the parent context, but sets the traceId field to the child TraceID of the subtask, while retaining the parent TraceID as the parentTraceId field. The execution context object also contains information such as the subtask's sequence number, task type, and creation timestamp. This context object is associated with the execution logic of the subtask through thread-local variables or explicit passing.

[0143] In the actual execution code of the subtask, all log output statements and performance data collection points obtain the value of the `traceId` field from the current execution context. Since this value has been set to the child `TraceID`, all log records and performance data generated during the execution of this subtask will automatically carry its child `TraceID`, instead of using the parent `TraceID`. In this way, the logs of the parent task and each subtask can be distinguished by different `TraceID`s, while also being linked together through the parent-child association.

[0144] S34: The generated child TraceID, its corresponding parent TraceID, task type, creation timestamp, and task status are recorded as a single record and written to the asynchronous task registry. The asynchronous task registry uses the parent TraceID as the primary key to build an inverted index, which supports quick querying of the list of all child TraceIDs under the parent TraceID and the detailed status of each child task.

[0145] Specifically, after a child TraceID is generated and bound to a subtask, the scheduler immediately writes a record to the asynchronous task registry. This registry can be implemented using Redis or MySQL, and its core data structure consists of an index table with the parent TraceID as the primary key and a table storing detailed records of the subtasks. The written record contains the following fields:

[0146] Parent TraceID: The root TraceID of the original request.

[0147] Sub-TraceID: A unique identifier generated for this subtask (e.g., ".1").

[0148] Task type: Material search, template matching, or sensitive word filtering.

[0149] Creation timestamp: The moment the subtask was created (accurate to milliseconds).

[0150] Task status: The initial status is "pending".

[0151] Meanwhile, the registry center builds an inverted index with the parent TraceID as the primary key. Through this index, the system can quickly query a list of all child TraceIDs under the parent TraceID and the detailed status of each subtask without scanning the entire table.

[0152] S35: When any asynchronous execution unit completes its execution, the execution result and final task status of the unit are sent back to the asynchronous task registration center, and the task status of the corresponding sub-TraceID is updated to "completed" or "abnormal".

[0153] Specifically, once any asynchronous execution unit completes, the subtask execution thread sends the processing result and final task status back to the asynchronous task registry. The registry locates the corresponding record based on the sub-TraceID, updates its task status field, and records the completion timestamp. If an exception occurs during task execution, a summary of the exception information is also appended to the record.

[0154] For example, after two material query subtasks successfully return material data, the status is updated to "completed"; if the sensitive word filtering subtask detects illegal content, the status may be updated to "abnormal" and the illegal word information will be recorded.

[0155] S36: In the subsequent full-link performance analysis and link restoration process, the link reconstruction engine queries the asynchronous task registry based on the parent TraceID to obtain a list of all associated child TraceIDs. Then, combined with the records of each child TraceID in the log system and performance data storage table, the complete parallel call sequence of the asynchronous task is restored according to the timestamp and parallel execution interval.

[0156] Specifically, when full-link performance analysis or fault location is required later, the link reconstruction engine will receive the parent TraceID specified by the user. The engine first sends a query request to the asynchronous task registry center, using the parent TraceID as the key, to obtain a list of all child TraceIDs under its name and the detailed status of each subtask.

[0157] After obtaining the list of child TraceIDs, the trace reconstruction engine queries the distributed log storage system and performance data storage table using the parent TraceID and each child TraceID as query conditions to retrieve all related log records and performance data. Then, the engine reconstructs the complete parallel call sequence of the asynchronous task in the following manner:

[0158] The parent task's own processing is taken as a main timeline, with its start time being the request reception time and its end time being the response return time after all subtasks are completed.

[0159] For each subtask, a time segment parallel to the main timeline is plotted based on its creation and completion timestamps recorded in the asynchronous task registry, as well as the specific operation time points extracted from the log system. For example, two material query subtasks may start simultaneously, but one takes 300ms and the other takes 350ms, thus appearing as two overlapping bars on the Gantt chart.

[0160] Align the parent task's timeline with the timelines of all child tasks to generate a Gantt chart with time on the horizontal axis and task instances on the vertical axis. The chart clearly shows when the parent task created which child tasks, when the child tasks started executing, when they ended, and how the parent task continued with subsequent processing after the child tasks were completed.

[0161] Furthermore, the step of assigning a child TraceID to each asynchronous execution unit and establishing a hierarchical association between the child TraceID and the parent TraceID includes:

[0162] Get the complete parent TraceID string as the prefix character, append an English period "." as the separator, and then append an incrementing sequence number starting from 1 to generate the child TraceID string;

[0163] Specifically, the system uses the complete parent TraceID string as the prefix of the child TraceID. Unlike the above implementation of "taking the first 32 bits of the parent TraceID", in this preferred embodiment, in order to maintain the most direct visual association and string traceability between the child TraceID and the parent TraceID, the system directly uses the complete parent TraceID string, then appends an English period "." as a separator, and then appends an integer sequence number starting from 1, finally generating a child TraceID string in the form of "complete parent TraceID.sequence number".

[0164] Taking the parent TraceID generated in step S1 as an example, assume the complete parent TraceID string is “1699500000123-a1b2c3d4-e5f6-7890-abcd-ef1234567890-edu1008080” (61 characters long). When the first asynchronous execution unit needs to be created, the system generates a child TraceID of “1699500000123-a1b2c3d4-e5f6-7890-abcd-ef1234567890-edu1008080.1”; the child TraceID of the second asynchronous execution unit is “1699500000123-a1b2c3d4-e5f6-7890-abcd-ef1234567890-edu1008080.2”, and so on. This design ensures that the header of any child TraceID is exactly the same as the parent TraceID, with only the tail having additional separators and sequence numbers. Operations personnel can quickly identify which parent request a child TraceID belongs to with the naked eye, without the need for additional queries.

[0165] Since this generation rule does not truncate, the length of the child TraceID is "parent TraceID length + 1 (dots) + number of digits for the sequence number". In most scenarios, the parent TraceID length is controlled within 64 characters, and the sequence number does not exceed 999. Therefore, the total length of the child TraceID is usually within 68 characters, which still meets the length limits of HTTP headers and RPC metadata.

[0166] The asynchronous task registry records all created child TraceIDs and their corresponding task execution statuses.

[0167] The asynchronous task registry uses the parent TraceID as the primary key to create an index, supporting queries for a list of all child task TraceIDs under the parent TraceID.

[0168] The string structure of the child TraceID itself contains a hierarchical relationship:

[0169] By checking if a TraceID contains a "." character and whether the character following the "." is a numeric sequence, it's easy to determine if it's a child TraceID. Removing the "." and all subsequent characters reveals the parent TraceID. For example, for the child TraceID "....2", removing ".2" gives the parent TraceID. This design eliminates the need for additional storage of parent-child mappings; the strings themselves are self-descriptive.

[0170] In the actual code implementation, the system provides two utility functions: `isChildTraceId(traceId)` is used to determine whether a given TraceID is a child TraceID; `extractParentTraceId(childTraceId)` is used to extract the parent TraceID from the child TraceID. The extraction method is as follows: find the position of the last period and extract all characters before that position.

[0171] Although the child TraceID string itself contains the parent TraceID information, the system still maintains an asynchronous task registry to support efficient batch queries. This registry has already been mentioned in step S34, and its index structure will be further refined here.

[0172] The registry center uses Redis as its underlying storage, and its data structure is designed as follows:

[0173] The main index uses Redis's hash structure, with the key named `async_task_registry`, the field being the parent TraceID, and the value being a serialized list of child TraceIDs (e.g., a JSON array [".1", ".2", ".3", ".4"]). This allows the system to retrieve all its child TraceIDs in O(1) time complexity, given any parent TraceID.

[0174] Subtask details: Use another hash structure with the key async_task_details, the field subTraceID, and the value a JSON object containing the following fields: parentTraceId, taskType, createTime, status, completeTime, and resultSummary.

[0175] When it's necessary to query the status of all subtasks based on the parent TraceID, the system first retrieves the list of subtask IDs from the `async_task_registry`, and then obtains detailed information for each subtask ID from `async_task_details`. For large-scale concurrency scenarios, Redis's Pipeline can be used for batch retrieval, reducing the number of network round trips.

[0176] Taking the example from the previous learning supervision module, the parent TraceID is "1699500000123-a1b2c3d4-e5f6-7890-abcd-ef1234567890-edu1008080". The system creates four subtasks, generating sub-TraceIDs: ...1, ...2, ...3, ...4. In the registry center, the field of the main index record is the parent TraceID, with values ​​of [".1", ".2", ".3", ".4"]. The subtask details store information such as the type and status of each subtask. When the link reconstruction engine needs to restore the complete call chain of the request, it first obtains these four sub-TraceIDs from the registry center, and then queries the logs and performance data respectively. This allows it to merge the logs of the parent task with the logs of each subtask according to the timeline, forming a complete parallel call sequence diagram. This invention utilizes the hierarchical structure of the string itself to achieve self-descriptive parent-child TraceID, and provides efficient parent-child relationship query capabilities through an asynchronous task registry. The two complement each other and jointly support the complete tracking of asynchronous parallel tasks in the AIAgent system.

[0177] S4: Connect log records scattered across different service nodes based on a unified TraceID identifier to reconstruct the complete call chain of the request;

[0178] Furthermore, S4 includes:

[0179] S41: Deploy a log formatting unit on each service node. This unit constructs each log record as a JSON object according to a unified log format specification.

[0180] When each service node starts up, the system automatically loads the log formatting unit. This unit rewrites or wraps the original log output framework, constructing each log record into a JSON object according to a unified format specification. This JSON object must contain at least the following five required fields:

[0181] timestamp: The timestamp generated by the log, using the ISO8601 standard, accurate to milliseconds, in a format such as "2023-11-09T10:30:25.123Z".

[0182] trace_id: The TraceID in the current request context.

[0183] node_name: The name of the service node that generates the logs, such as "ai-service-main", "material-agent", or "template-agent".

[0184] level: Log level, such as INFO, WARN, ERROR.

[0185] message: Log content, a human-readable string, which should contain key call identifiers, such as "received request", "start processing", "call downstream service", "return response", "asynchronous task created", "asynchronous task completed", etc.

[0186] In addition, optional fields such as span_id and parent_span_id can be included to more precisely identify the call span. For example, a typical log record is: {"timestamp": "2023-11-09T10:30:25.123Z", "trace_id": "1699500000123-a1b2c3d4...", "node_name": "ai-service-main", "level": "INFO", "message": "Received a request to generate supervisory scripts, material quantity = 2"}.

[0187] S42: Deploy a log storage agent unit on each service node. The agent unit uses an asynchronous batch writing method to temporarily store the JSON format log records in a memory buffer. When the number of log records in the buffer reaches a preset threshold or the time interval between the last batch writing reaches a preset threshold, all log records in the buffer are sent to the distributed log storage system in batches.

[0188] On each service node, the system simultaneously deploys a log storage proxy unit. This proxy unit does not directly write synchronously to the distributed log storage system (such as Elasticsearch), but instead uses asynchronous batch writing to reduce the impact on business processing performance. The specific implementation is as follows:

[0189] The proxy unit maintains a memory buffer to temporarily store JSON log records generated by the log formatting unit. When a business thread generates a log entry, the proxy unit quickly puts the log object into the buffer, and the business thread immediately returns to continue execution without waiting for the write operation to complete. Simultaneously, the proxy unit runs a separate consumer thread in the background, which continuously checks the following two triggering conditions:

[0190] Quantity threshold: The number of accumulated log records in the buffer reaches a preset threshold, such as 1000.

[0191] Time threshold: The time interval since the last batch write reaches a preset threshold, such as 5 seconds.

[0192] Once any condition is met, the consumer thread retrieves all current log records from the buffer, constructs a batch request, and sends it to the distributed log storage system via HTTP or TCP. After sending, the buffer is cleared. Using this mechanism, logs may remain in the buffer for a few seconds before being written under low load, but in high-concurrency scenarios, it significantly reduces I / O operations, preventing log writing from becoming a system bottleneck.

[0193] S43: When it is necessary to restore the complete call chain of a certain request, the link reconstruction engine receives the target TraceID, which is input by the user or specified by the system, and records it as target_trace_id;

[0194] S44: Initiate an exact match query to the distributed log storage system through the link reconstruction engine, with the query condition "TraceID=target_trace_id", and obtain all log records containing TraceID to form the original log set;

[0195] S45: The original log set is sorted in ascending order by the timestamp field using the link reconstruction engine to generate a log sequence arranged in chronological order;

[0196] S46: By traversing the log sequence through the link reconstruction engine, based on the service node name in each log record and the call identifier in the log content, the entry, exit and internal processing nodes of the request on each service node are identified, and a call sequence diagram with time as the horizontal axis and service nodes as the vertical axis is constructed.

[0197] When it is necessary to restore the complete call chain of a request, the system call chain reconstruction engine performs the following operations:

[0198] The engine receives the TraceID string input by the user, denoted as target_trace_id.

[0199] The engine constructs an Elasticsearch query request with the query condition that the `trace_id` field exactly matches `target_trace_id`. This query uses a term query instead of full-text search to ensure efficiency and accuracy. The query results return all log records containing that `trace_id`, forming the original log set. If the log data volume is large, pagination or cursor mechanisms can be used to retrieve it in batches.

[0200] The engine sorts the raw log collection in ascending order by the timestamp field, generating a log sequence arranged chronologically. This step makes the timeline of the request processing process clear from start to finish.

[0201] The engine iterates through the sorted log sequence and identifies key event points of the request on each service node based on specific keywords in the `node_name` and `message` fields of each log entry. For example:

[0202] When a message contains "received request", it is marked as an entry point on that node.

[0203] When a message contains "Start processing", it is marked as the start of processing.

[0204] When a message contains "call downstream service" and includes the target service address, it is marked as the start of a cross-node call.

[0205] When a message contains "return response", it is marked as an exit point on that node.

[0206] When a message contains "processing complete", it is marked as the end of the processing.

[0207] The engine organizes these event points according to their chronological order and the nodes they belong to, constructing a call sequence diagram with time as the horizontal axis and service nodes as the vertical axis. The sequence diagram can be represented in JSON intermediate format, for example: {"startTime": ...,"endTime": ...,"node":"main","children":[{"node":"material-agent","start": ...,"end": ...}]}. This sequence diagram can ultimately be output as an image or displayed as a Gantt chart on the front end.

[0208] S47: For asynchronous task scenarios that include child TraceIDs, the link reconstruction engine queries the asynchronous task registry based on the parent TraceID to obtain a list of all associated child TraceIDs;

[0209] S48: Execute steps S44 to S46 for the parent TraceID and each child TraceID respectively to obtain multiple sets of log sequences;

[0210] S49: Based on the timestamps of each log sequence, align the execution time periods of the parent task and each child task in parallel to restore the complete parallel call sequence and output it in the form of a Gantt chart.

[0211] For asynchronous task scenarios involving child TraceIDs, querying only the parent TraceID yields a log sequence that only shows the parent task's own timeline, not the internal details of each asynchronous child task. Therefore, the engine executes S47 to S49:

[0212] The engine first queries the asynchronous task registry based on the parent TraceID, which is the target_trace_id entered by the user. The registry returns a list of all associated child TraceIDs under that parent TraceID, such as ["...1", "...2", "...3", "...4"].

[0213] The engine repeats steps S44 to S46 for the parent TraceID and each child TraceID, resulting in multiple log sequences. Each log sequence corresponds to a complete timeline of an execution unit.

[0214] The engine aligns the timeline of the parent task with the timelines of all child tasks in parallel based on the timestamps of each log sequence. Specifically, it uses the start time of the parent task as the global starting point and the end time of the parent task as the global ending point. For each child task, its execution interval is determined by the earliest and latest timestamps in its log sequence. Since child tasks are executed in parallel, these intervals may partially overlap on the timeline. The engine arranges the execution intervals of all tasks by task instance on the vertical axis and time on the horizontal axis, generating a Gantt chart. In the Gantt chart, the parent task typically occupies the entire time period from the start of the request to the completion and summary return of all child tasks, while each child task is displayed as a bar corresponding to its execution interval. If a child task encounters an exception, its bar can be marked in red. Finally, the engine presents the Gantt chart to the user in a visual format, thus fully reconstructing the asynchronous parallel call sequence.

[0215] S5: Collect time-consuming data, call status, and resource usage information at key nodes of service calls;

[0216] Furthermore, the collection of time-consuming data, call status, and resource usage information at key nodes of the service call specifically includes:

[0217] At the request receiving node, collect the request arrival time, service interface name, and request parameter characteristics; at the service processing start node, collect the processing start time and current service load status.

[0218] Collect information such as the initiation time of downstream service calls, target service address, and call protocol type from external calling nodes;

[0219] Collect downstream service response time, response status code, and response data size from the external call return node;

[0220] Collect processing completion time, processing result status, and exception information at the service processing completion node;

[0221] When the AI-Agent submodule initiates a call to the media library service, it also collects the initiation time of the media query request, the API interface path of the media library, the query condition parameters, the response time of the media library service, the number of media records returned, and the completion time of the media data being transmitted to the speech assembly module.

[0222] In one specific embodiment of the present invention, step S5, "collecting time-consuming data, call status, and resource usage information at key nodes of service calls," is achieved by deploying a performance data collector on each service node. This collector is embedded into key locations in the business processing flow using instrumentation, automatically collecting relevant data according to predefined event types, and writing the collected results into a performance data storage table after associating them with the TraceID of the current request. The collection method is described below for different key nodes.

[0223] Request the receiving node to collect data:

[0224] When the AI-Service main service receives a user request at its request entry point, the performance data collector triggers a "request reception" event before the request is dispatched to a specific business processing function. In this event, the collector gathers the following data:

[0225] Arrival time: The current timestamp is obtained from the system clock, accurate to milliseconds, and denoted as "arrival_time".

[0226] Service interface name: Extracted from request metadata, such as the URL path of an HTTP request or the method name of an RPC call.

[0227] Request parameter characteristics include, but are not limited to, key business parameters such as the request's business type identifier, request body size, and whether it includes a list of material IDs. To protect sensitive information, parameter values ​​can be anonymized or only the structural characteristics of the parameters can be recorded.

[0228] After collection is complete, the collector generates a performance record containing the aforementioned fields and the TraceID obtained from the request context, and marks the record's "event_type" as "REQUEST_RECEIVED". This record is temporarily stored in a local memory queue, to be written to the performance data storage table in batches later.

[0229] Data collection for service startup nodes:

[0230] Before the business processing function begins execution, the performance data collector triggers the "service processing start" event.

[0231] The collected data includes:

[0232] Processing start time: Current system timestamp, denoted as "process_start_time".

[0233] Current service load status: This includes the CPU utilization of the current service node, JVM memory usage, number of active threads, and the average response time over the past minute. This load information reflects the system pressure on the service when processing the current request.

[0234] The event's "event_type" is marked as "PROCESS_START". By associating this with the record of the request receiving node, the time spent waiting for scheduling before service processing can be calculated.

[0235] Collection of data from external calling nodes:

[0236] When the main AI-Service needs to call the downstream AI-Agent submodule, the performance data collector triggers an "external call" event at the last moment before the call is initiated.

[0237] The collected data includes:

[0238] Downstream service call initiation time: denoted as "call_start_time".

[0239] Target service address: The IP address and port number of the downstream service, or the service name in the service registry.

[0240] Call protocol type: HTTP or RPC, and the specific protocol version.

[0241] The event's "event_type" is marked as "EXTERNAL_CALL_START". The record also stores the TraceID of the current request to associate this call with the parent request.

[0242] Collection of nodes returned by external calls:

[0243] When the downstream service returns a response, the performance data collector triggers an "external call returned" event at the first processing point after receiving the response data.

[0244] The collected data includes:

[0245] Downstream service response return time: denoted as "call_end_time".

[0246] Response status code: HTTP status code or error code of the RPC framework.

[0247] Response data size: The length of the response body in bytes.

[0248] The event's "event_type" is marked as "EXTERNAL_CALL_END". By associating the "EXTERNAL_CALL_START" record on the same call chain, the network and service processing time of the external call can be calculated: "call_duration = call_end_time - call_start_time".

[0249] Collection of data at the service processing completion node:

[0250] When a business processing function completes normally or throws an exception, the performance data collector triggers a "service processing complete" event. The collected data includes:

[0251] Processing completion time: Current system timestamp, denoted as "process_end_time".

[0252] Processing result status: Success, Business failure, or System error.

[0253] Exception Information: If an exception occurs during processing, record the exception type, exception message, and key stack frames. This field is empty for requests that complete normally.

[0254] The event's "event_type" is marked as "PROCESS_END". Combined with the time it took to start the process node, the processing time of the service node itself can be calculated: "self_process_time = process_end_time - process_start_time".

[0255] Specialized data collection for media library service calls:

[0256] For common media library call scenarios in the AI-Agent system, the performance data collector provides additional specialized data collection capabilities. When the AI-Agent submodule initiates a call to the media library service, the collector not only collects the aforementioned general external call data, but also the following specialized fields:

[0257] Material query request initiation time: Same as "call_start_time" of external calling nodes, but recorded separately to highlight the material library.

[0258] The API interface path of the material library: for example, " / api / material / query" or " / api / material / batch_get".

[0259] Query criteria parameters: including material type, tags, keywords, or a specific list of material IDs. Note that long lists should be truncated when recording (e.g., only record the first 5 IDs and the total number).

[0260] The response time of the material library service is the total time from the time the request is initiated to the time the response is returned, which is consistent with the time taken for general external calls.

[0261] The number of returned material records: obtained from the response parsing, for example, "result_count=json.getInt("total")".

[0262] The completion time of transferring material data to the script assembly module: This is a business-level time point, referring to the moment when the data returned by the material library is parsed, transformed, and successfully transferred to the script assembly module. This time may be later than the return time of the external call. The collector inserts a dedicated tracking point after the data transfer is completed on the material library client, denoted as "transfer_complete_time".

[0263] These specific data points are written to a separate media library call log table, while general performance data remains in the "performance_data" table. The two tables are linked by the "trace_id" field. This targeted data collection allows business analysts to precisely quantify the contribution of media library calls to overall response latency and identify fine-grained bottlenecks in the media library call chain.

[0264] Data writing and storage:

[0265] All performance records generated by the aforementioned collection nodes are not immediately written to the database. Instead, they are processed by an asynchronous batch write thread within the performance data collector. This thread maintains a memory buffer. When the number of records in the buffer reaches a preset threshold or the interval since the last write exceeds a set limit, a batch write operation is triggered, inserting multiple records into the performance data storage table and the media library call record table at once. The write operation uses prepared statements and batch commits to reduce database I / O overhead and impact on business threads. In case of write failure, the system logs an error and attempts to retry (up to 3 times). Records that exceed the retry limit are stored in a local file as a backup to prevent data loss.

[0266] This invention enables fine-grained performance data collection for key nodes in a distributed AI service chain, particularly for data collection specifically for material library calls, providing accurate data support for subsequent bottleneck analysis and business optimization.

[0267] S6: Calculate the response latency and resource consumption ratio of each stage based on the collected performance data to pinpoint the specific source of performance bottlenecks.

[0268] Furthermore, S6 includes:

[0269] Calculate the total time of the entire link The It equals the processing completion time of the request entry point minus the request arrival time;

[0270] Calculate the time taken for the i-th service node or external call. ;

[0271] Calculate each occupy proportion ,in ;

[0272] Set bottleneck determination threshold ,when When the i-th stage is identified, it is marked as the performance bottleneck.

[0273] The The default value is 0.2. When the time spent in a certain link accounts for more than 20% of the total time spent in the entire link, that link is determined to be a performance bottleneck.

[0274] In a specific embodiment of the present invention, step S6, "calculating the response latency and resource consumption ratio of each link based on the collected performance data, and locating the specific source of the performance bottleneck," is implemented in the following way. The core of this step is to automatically identify the link most likely to become a performance bottleneck by quantifying the proportion of the time consumed by each service link in the total time consumed in the entire link and setting dynamic or fixed thresholds, thus providing accurate data guidance for operation and maintenance optimization.

[0275] Calculation of total end-to-end time:

[0276] The performance analysis and bottleneck diagnosis module first extracts all event records associated with the current request's TraceID from the performance data storage table. By retrieving records where "event_type" is "REQUEST_RECEIVED", the request arrival time is obtained. The processing completion time can be obtained by retrieving records where "event_type" is "PROCESS_END". The total time for the entire link is then... Defined as:

[0277]

[0278] It reflects the complete end-to-end latency from when a user initiates a request to when they receive the final response, and is the most intuitive indicator for measuring system performance.

[0279] Extraction and calculation of time consumption for each step:

[0280] For the i-th service node or external call link in the call chain, the system calculates the time consumption of that link based on its corresponding event. :

[0281] For internal processing within the service node: it is obtained through the timestamp difference between the "PROCESS_START" and "PROCESS_END" events.

[0282] For external service calls: the timestamp difference between the "EXTERNAL_CALL_START" and "EXTERNAL_CALL_END" events is used.

[0283] Then, calculate the proportion of this step's time in the total time of the entire link. ; This reveals the contribution of each step to the overall response latency. For example, the time taken to access a certain media library... =380 milliseconds, while the total time of the entire link is 380 milliseconds. =1000 milliseconds, then =38%, indicating that the resource library call contributed more than one-third of the latency.

[0284] To automatically identify performance bottlenecks, the system sets a bottleneck determination threshold. The default value is 0.2, which is 20%. This represents the percentage of time spent on a particular stage. When this threshold is exceeded, the system automatically marks that process as a performance bottleneck. The determination criteria are:

[0285] like If so, then link i will be identified as the performance bottleneck.

[0286] The 20% threshold is chosen based on a rule of thumb: in a normal distributed call chain, the total time is usually shared among multiple stages, and the percentage of any single stage rarely exceeds 20%. Exceeding this value indicates a significant performance anomaly in that stage, warranting close attention. This threshold can be configured according to different business scenarios.

[0287] In this embodiment, The core significance lies in transforming the abstract problem of "slow requests" into a quantifiable percentage indicator, achieving a shift from "qualitative perception" to "quantitative attribution." Its specific functions include:

[0288] The absolute time consumption of different links may vary greatly, but by calculating the proportion, we can fairly compare their contribution to the overall latency and avoid ignoring the high proportion of certain links in short links due to small absolute values.

[0289] By setting fixed thresholds, the system can automatically filter out processes that exceed limits, eliminating the need for manual analysis of time-consuming data and significantly improving operational efficiency. In large-scale AIAgent systems, there may be hundreds of thousands of call chains every day, making manual investigation impractical. This formula makes automated bottleneck location possible.

[0290] For multiple exceeding the standard, the system can follow... Sort by size from largest to smallest, prompting users to prioritize optimizing the processes with the highest resource allocation to maximize the return on investment.

[0291] Example 2

[0292] Please see Figure 1 This invention provides a method for full-process tracing and performance analysis of trace_id in a distributed service chain.

[0293] In this invention, the method described in Example 1 is based on the proportion of the time spent in each service link to the total time spent in the entire link. It adopts a fixed 20% threshold and has the advantages of simple calculation, high real-time performance, and no need to rely on historical data support. It is suitable for rapid bottleneck screening and real-time alarm triggering in online monitoring scenarios.

[0294] In this embodiment, the method described introduces a causal analysis model based on the delay propagation coefficient ρ and the net root cause contribution, by statistically analyzing the total processing time of upstream link A. Compared with the pure processing time of downstream stage B The ratio of covariance to standard deviation between upstream and downstream processes quantifies the causal relationship strength of time fluctuations, thereby eliminating pseudo-bottleneck interference caused by downstream delays being transmitted upstream and accurately identifying the true root cause.

[0295] Specifically, this method requires maintaining historical sample data and has high computational complexity, but it effectively avoids the misjudgment problem commonly found in simple threshold methods in asynchronous parallel call chains. It is suitable for offline deep analysis or precise diagnosis of fault root causes. The two methods are not mutually exclusive, but rather serve two different levels of needs: rapid anomaly detection and precise root cause localization, and complement each other.

[0296] The specific sources of the positioning performance bottleneck in S6 include:

[0297] For any adjacent upstream or downstream link in a single request call chain, collect the total processing time of the upstream link A. And the pure processing time of downstream stage B Historical sample data;

[0298] in This includes the time taken for A to call B and wait for its return. This does not include the time spent by B calling its downstream processes;

[0299] Based on the historical sample data, calculate the delay propagation coefficient between upstream link A and downstream link B. ;

[0300] The calculated delay propagation coefficient As a measure of the intensity of time-consuming fluctuations transmitted from downstream link B to upstream link A:

[0301] When | When | approaches 1, it is determined that the abnormal time consumption of upstream A is mainly caused by the time consumption fluctuation of downstream B; when | When | approaches 0, the causal relationship between the fluctuations in the time consumption of the two is considered to be weak;

[0302] Among them, the The calculation formula is:

[0303]

[0304] In the formula, the for and covariance, for standard deviation for The standard deviation.

[0305] In a preferred embodiment of the present invention, the "locating the specific source of the performance bottleneck" in step S6 not only includes the simple threshold determination based on the time consumption ratio in Embodiment 1, but also further introduces a bottleneck root cause localization method based on causal analysis. This method distinguishes between two different types of performance problems—"time consumption anomalies caused by downstream service delays" and "service-related processing anomalies"—by calculating the delay transmission coefficient between upstream and downstream links, thereby avoiding misjudging upstream services as bottlenecks.

[0306] Specifically, for any pair of adjacent upstream and downstream service links in a single request call chain, denoted as upstream link A and downstream link B, the performance data collector continuously collects two types of time data during multiple request processing processes:

[0307] Total processing time of upstream link A The total time elapsed from when A begins processing the request to when A completes processing it. This time includes the execution time of A's own business logic, the network transmission time for A to call B and wait for B's response, and B's processing time. In other words, It covers the delay in A's dependency on B.

[0308] Pure processing time of downstream stage B This refers to the time elapsed from when B receives the upstream request to when B completes its own business logic. This time reflects only B's own processing capacity and excludes the waiting time for B to call downstream services. If B has no downstream calls, then... Equal to the complete processing time of B.

[0309] These data are stored in a historical database as key-value pairs, where the key is a unique identifier for the request, and the value contains the values ​​of A and B in that request. Sample. The system maintains a sliding time window to ensure that the data reflects the current operating status of the system.

[0310] Based on the collected historical sample dataset:

[0311]

[0312] Calculate the delay propagation coefficient between upstream link A and downstream link B. ;

[0313]

[0314] In the formula, the for and The covariance is calculated using the following formula: ,in and They are respectively The sample mean. The sign of the covariance reflects whether the trends of change of the two are in the same or opposite direction, and the magnitude of the absolute value reflects the strength of the association.

[0315] for standard deviation for The standard deviation; the introduction of the standard deviation makes Dimensionless, with values ​​ranging from [−1, 1].

[0316] absolute value | The closer it is to 1, the more it indicates... and There is a strong linear correlation between them, meaning that the time fluctuations of upstream A can be largely explained by the time fluctuations of downstream B. This indicates that the time anomalies in A are likely caused by delayed transmission from B. Conversely, if | If the value is close to 0, it indicates that there is almost no linear relationship between the two. The time fluctuation of A is unrelated to B. Therefore, if the time of A is too high, the root cause of the fault should be A itself or its upstream, rather than B.

[0317] Specifically, when | When the threshold is greater than 0.8 (close to 1), it is determined that the abnormal time consumption of upstream A is mainly caused by the time consumption fluctuation of downstream B. At this time, even if the time consumption of A exceeds the 20% threshold, A should not be marked as the root cause bottleneck, but the performance of downstream B should be further analyzed.

[0318] When | When |<0.3 (close to 0), the causal relationship between the time fluctuations of the two is weak, and the time change of upstream A is independent of downstream B. At this time, if the time of A is high, the root cause is likely to be A itself, and A should be marked as a candidate root cause node.

[0319] When 0.3 <| When |<0.8, it is considered moderately correlated. The system will make a comprehensive judgment based on other indicators, or rely on the net root cause contribution in claim 10 for further quantification.

[0320] Taking the script generation task in the supervisory module as an example, upstream link A is the "script generation main service," and downstream link B is the "material library service." The system collects sample data from the past 1000 requests and calculates... =0.92. The total processing time for A in a certain request. The abnormally high processing time of B increased to 800 milliseconds. It also abnormally increased to 350 milliseconds. Due to... With a latency as high as 0.92, the system determined that the abnormal time consumption of A was mainly caused by the delayed transmission of B. Therefore, A was not marked as a bottleneck, and the focus of analysis was shifted to B. A subsequent in-depth diagnosis of B ultimately revealed slow queries in the material database upon which B depended, thus accurately pinpointing the root cause. Conversely, if =0.15, and if A's time consumption is abnormal, the system will directly determine A as the root cause of the performance problem and prompt the system to check A's own business logic.

[0321] Furthermore, the specific sources of the positioning performance bottleneck also include:

[0322] Get the actual time spent in upstream step A of the current request. and the actual time spent in downstream link B And query the historical median time of stage A and stage B from the historical database. and ;

[0323] Through the delay propagation coefficient Calculate the net root cause contribution of downstream stage B relative to upstream stage A. ;

[0324] when When >0, B is identified as a candidate root cause node; when When =0, it is determined that the abnormal time consumption of B is entirely caused by upstream transmission or normal fluctuations, and should not be marked as the root cause.

[0325] Based on delay propagation coefficient Based on the initial assessment of the strength of the causal relationship between upstream and downstream processes, this invention further proposes a quantitative indicator called Net Root Cause Contribution to accurately determine whether downstream process B is the true root cause of the performance bottleneck. This indicator comprehensively considers the actual time deviation of each process in the current request and the historical propagation correlation, effectively eliminating "pseudo-bottleneck" interference caused by strong propagation relationships and achieving accurate root cause localization.

[0326] When it is necessary to perform root cause analysis on a specific request call, the performance analysis and bottleneck diagnosis module first extracts the following two values ​​from the performance data of that request:

[0327] : The actual total processing time of upstream link A in the current request.

[0328] : The actual pure processing time of downstream link B in the current request.

[0329] Simultaneously, the module queries the historical median time for each of stages A and B from the historical database, and records them as follows: and The median is less sensitive to outliers than the mean and is a better representative of typical performance under normal load. The selection of the historical window can be configured according to system operation.

[0330] Net Root Cause Contribution The calculation formula is as follows:

[0331]

[0332] In the formula, This represents the absolute deviation of the processing time of downstream process B from its historical normal level. If this value is positive and large, it indicates that B itself has experienced a significant delay in processing; if it is negative or close to zero, B is performing normally.

[0333] This represents the absolute deviation of the time taken in step A from its historical normal level. This value reflects the overall degree of anomaly in A and is used to normalize the deviation in B. When A itself is not anomaly, i.e. When the mother approaches zero, the contribution of B needs special handling for calculation. pass Replacement, the It is a very small positive number.

[0334] In actual calculations, you may encounter... Or, in cases where the absolute value is extremely small, meaning the time taken by upstream step A is exactly equal to or very close to its historical median, directly using the above formula will result in a denominator of zero or unstable values. Therefore, this invention employs the following robust handling strategy:

[0335] Set a very small positive number .when When, replace the denominator with ;

[0336] This treatment ensures that, assuming A performs normally, B's net contribution is proportional to... .because The deviation is very small. Therefore, even a slight deviation in B will be amplified. This is logical: when the upstream is completely normal but the downstream experiences a time-consuming deviation, that deviation is very likely the root cause. At the same time, The introduction of this avoids division by zero errors.

[0337] Used to suppress root cause misjudgments caused by strong transitivity; when the transitivity coefficient is delayed When the factor approaches 1 (strong transmission relationship), it approaches 0, thus significantly reducing the net root cause contribution of B and avoiding misjudging B as the root cause (because at this point, the abnormal time consumption of A is mainly caused by the delayed transmission of B, but B itself may not be the root cause and needs to be traced further downstream). When When the value is close to 0 (weak transitivity), the factor is close to 1, and the contribution of B is mainly determined by its own time consumption bias.

[0338] Max(0,⋅) is the cutoff function, ensuring that the net root factor contribution is non-negative. If the calculated result is negative, for example... If the value is below the historical median, it indicates that B's performance is better than normal, so a value of 0 is taken, meaning that B is not considered a candidate root cause.

[0339] Calculated Then, the system performs the following judgment:

[0340] like If the value is positive, then downstream link B is identified as a candidate root cause node. The larger the value, the higher the probability that B itself is the actual performance bottleneck. For multiple adjacent pairs in a call chain, the system calculates the root cause for each downstream link and takes the link with the largest value as the most likely root cause.

[0341] like If the abnormal time consumption of B is determined to be entirely caused by upstream transmission or is a normal fluctuation, it should not be marked as the root cause. In this case, the focus of analysis should shift to more downstream processes, or examine whether there are other problems upstream A.

[0342] Using the same example of the supervisory module mentioned above: upstream stage A is the "script generation main service," and downstream stage B is the "material library service." Historical data shows... millisecond, Milliseconds. Measured during a fault request. millisecond, Milliseconds, and already calculated. Substitute into the formula:

[0343]

[0344] This value is positive (0.05), indicating that B is marked as a candidate root cause node. However, if this value is not the largest compared to other downstream links, there may be other root causes. In another scenario: If other values ​​remain unchanged, then:

[0345]

[0346] The significantly larger value indicates that, under weak transmission relationships, the time-consuming bias of B itself contributes significantly to the root cause.

[0347] Now consider the scenario where A is normal and B is abnormal: millisecond, millisecond\ At this point, the denominator is 0, triggering... Processing, setting =0.001, then:

[0348]

[0349] The large numbers clearly indicate that B is the root cause, as expected.

[0350] In this embodiment, by introducing the net root cause contribution, it is possible to distinguish between "delay caused by downstream transmission" and "delay caused by downstream abnormalities", thus avoiding misjudging the abnormal links due to transmission as the root cause.

[0351] Furthermore, using the median as the normal baseline eliminates the impact of time-varying fluctuations (such as peak periods) on the judgment results.

[0352] As a further implementation, the system uses the method described in Example 1 as a first-level filter to calculate the real-time time consumption percentage of all service steps. When the time consumption percentage of a certain step... When the threshold is exceeded by 20%, it is marked as a candidate bottleneck and an alarm is triggered or an event to be analyzed is recorded. Subsequently, the system automatically calls the method described in this embodiment as a second-level diagnostic tool (deep root cause analysis stage) for the candidate bottleneck and its upstream and downstream adjacent links. Through this two-stage collaborative mechanism of coarse screening followed by fine diagnosis, the system ensures both the timeliness and low overhead of real-time monitoring, and significantly improves the accuracy of root cause localization, avoiding false alarms caused by transitivity delays.

[0353] The methods described in Example 1 and this example can also be selectively deployed or dynamically switched according to the characteristics of different business scenarios in the AIAgent system. For lightweight services with extremely high request volume, stringent real-time requirements, and relatively simple call chains, the time consumption percentage threshold method in Example 1 alone can meet the operation and maintenance monitoring needs. Its computational overhead is almost negligible, enabling rapid screening of performance bottlenecks at extremely low resource cost. For core business scenarios with complex call chains, multiple layers of asynchronous parallel calls, and downstream services that may become sources of delay propagation, the causal analysis method in this example is prioritized to accurately distinguish between transitive delays caused by "slow downstream" and root cause delays caused by "abnormal processing of its own," avoiding misjudging upstream services as bottlenecks and leading to incorrect optimization decisions.

[0354] In addition, the system can dynamically adjust the activation strategy of the two methods based on runtime monitoring data:

[0355] Under normal load, only the threshold monitoring in Example 1 is run;

[0356] When the total end-to-end latency exceeds a certain percentage of the historical baseline or the bottleneck alarm frequency rises abnormally, the deep causal analysis in this embodiment is automatically triggered to achieve adaptive diagnosis. This flexible selection strategy allows the present invention to balance performance overhead and analysis accuracy in different scenarios, making it widely applicable.

[0357] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for full-process tracing and performance analysis of trace_id in a distributed service chain, characterized in that, include: S1: Generate a globally unique TraceID at the user request entry point and establish a request context object associated with that TraceID; S2: In each service call from the AI-Service main service to the downstream AI-Agent sub-module, the TraceID identifier is embedded in the HTTP request header or RPC call context; S3: For asynchronous task scheduling scenarios in AI-Agent systems, assign a sub-TraceID to each asynchronous execution unit and establish a hierarchical relationship between the sub-TraceID and the parent TraceID; S4: Connect log records scattered across different service nodes based on a unified TraceID identifier to reconstruct the complete call chain of the request; S5: Collect time-consuming data, call status, and resource usage information at key nodes of service calls; S6: Calculate the response latency and resource consumption ratio of each stage based on the collected performance data to pinpoint the specific source of performance bottlenecks.

2. The method for full-process tracing and performance analysis of trace_id in a distributed service chain according to claim 1, characterized in that, The generation of a globally unique TraceID specifically includes: A 128-bit globally unique identifier is generated using the UUIDv4 algorithm; The current system timestamp is encoded to the millisecond level and then appended to the UUID string to form a hybrid TraceID; The service node identifier that generates the TraceID is encoded into a fixed-length string and appended to the end of the hybrid TraceID to form a complete TraceID string with self-describing characteristics.

3. The method for full-process tracing and performance analysis of trace_id in a distributed service link according to claim 1, characterized in that, S3 include: S31: When the AI-Service main service receives a user request and generates a globally unique parent TraceID, it determines whether the processing flow of the request needs to be split into multiple parallel asynchronous execution units; if so, it enters the asynchronous task scheduling branch. S32: For each asynchronous execution unit to be created, generate a unique sub-TraceID; S33: After the sub-TraceID is generated, the asynchronous task scheduler binds the sub-TraceID to the corresponding asynchronous execution unit and writes the sub-TraceID into the execution context of the asynchronous task, so that all logs and performance data output by the asynchronous task during execution carry its sub-TraceID. S34: The generated child TraceID, its corresponding parent TraceID, task type, creation timestamp, and task status are recorded as a single record and written to the asynchronous task registry. The asynchronous task registry uses the parent TraceID as the primary key to build an inverted index, which supports quick querying of the list of all child TraceIDs under the parent TraceID and the detailed status of each child task. S35: When any asynchronous execution unit completes its execution, the execution result and final task status of the unit are sent back to the asynchronous task registration center, and the task status of the corresponding sub-TraceID is updated to "completed" or "abnormal". S36: In the subsequent full-link performance analysis and link restoration process, the link reconstruction engine queries the asynchronous task registry based on the parent TraceID to obtain a list of all associated child TraceIDs. Then, combined with the records of each child TraceID in the log system and performance data storage table, the complete parallel call sequence of the asynchronous task is restored according to the timestamp and parallel execution interval.

4. The method for full-process tracing and performance analysis of trace_id in a distributed service link according to claim 3, characterized in that, The process of assigning a child TraceID to each asynchronous execution unit and establishing a hierarchical association between the child TraceID and the parent TraceID includes: Get the complete parent TraceID string as the prefix character, append an English period "." as the separator, and then append an incrementing sequence number starting from 1 to generate the child TraceID string; The asynchronous task registry records all created child TraceIDs and their corresponding task execution statuses. The asynchronous task registry uses the parent TraceID as the primary key to create an index, supporting queries for a list of all child task TraceIDs under the parent TraceID.

5. The method for full-process tracing and performance analysis of trace_id in a distributed service chain according to claim 1, characterized in that, The collection of time-consuming data, call status, and resource usage information at key nodes of service calls specifically includes: At the request receiving node, collect the request arrival time, service interface name, and request parameter characteristics; at the service processing start node, collect the processing start time and current service load status. Collect information such as the initiation time of downstream service calls, target service address, and call protocol type from external calling nodes; Collect downstream service response time, response status code, and response data size from the external call return node; Collect processing completion time, processing result status, and exception information at the service processing completion node; When the AI-Agent submodule initiates a call to the media library service, it also collects the initiation time of the media query request, the API interface path of the media library, the query condition parameters, the response time of the media library service, the number of media records returned, and the completion time of the media data being transmitted to the speech assembly module.

6. The method for full-process tracing and performance analysis of trace_id in a distributed service chain according to claim 1, characterized in that, S6 includes: Calculate the total time of the entire link The It equals the processing completion time of the request entry point minus the request arrival time; Calculate the time taken for the i-th service node or external call. ; Calculate each occupy proportion ,in Set bottleneck determination threshold ,when When the i-th stage is marked as a performance bottleneck; the The default value is 0.

2. When the time spent in a certain link accounts for more than 20% of the total time spent in the entire link, that link is determined to be a performance bottleneck.

7. The method for full-process tracing and performance analysis of trace_id in a distributed service link according to claim 3, characterized in that, When the AI-Service main service receives a user request and generates a globally unique parent TraceID, it determines whether the processing flow of the request needs to be split into multiple parallel asynchronous execution units. If necessary, proceed to the asynchronous task scheduling branch, including: After the AI-Service main service receives a user request and generates a globally unique parent TraceID, it extracts the business type identifier of the request. The business type identifier includes a script generation request from the supervisor module, an intent recognition request from the demand mining module, or a material library association call request. Based on the business type identifier, query the preset task splitting rule table; The task splitting rule table predefines a subtask partitioning strategy for each business type. The subtask partitioning strategy includes: the number of subtasks, the execution logic type of each subtask, and the dependencies between subtasks. When the number of subtasks corresponding to the business type identifier is greater than 1, it is determined that it needs to be split into multiple parallel asynchronous execution units; otherwise, it is determined that no splitting is required and synchronous execution mode is directly adopted. For requests that are determined to need to be split, the business parameters in the request are further analyzed, and the number and parallelism of subtasks are dynamically adjusted according to the business parameters. After the splitting judgment is completed, if splitting is required, the list of split subtasks and their respective execution logic are passed to the asynchronous task scheduler, triggering the sub-TraceID generation and registration process in steps S32 to S36. If no splitting is required, the parent TraceID will continue to be used for synchronization.

8. The method for full-process tracing and performance analysis of trace_id in a distributed service link according to claim 1, characterized in that, S4 includes: S41: Deploy a log formatting unit on each service node. This unit constructs each log record as a JSON object according to a unified log format specification. S42: Deploy a log storage agent unit on each service node. The agent unit uses an asynchronous batch writing method to temporarily store the JSON format log records in a memory buffer. When the number of log records in the buffer reaches a preset threshold or the time interval between the last batch writing reaches a preset threshold, all log records in the buffer are sent to the distributed log storage system in batches. S43: When it is necessary to restore the complete call chain of a certain request, the link reconstruction engine receives the target TraceID, which is input by the user or specified by the system, and records it as target_trace_id; S44: Initiate an exact match query to the distributed log storage system through the link reconstruction engine, with the query condition "TraceID=target_trace_id", and obtain all log records containing TraceID to form the original log set; S45: The original log set is sorted in ascending order by the timestamp field using the link reconstruction engine to generate a log sequence arranged in chronological order; S46: By traversing the log sequence through the link reconstruction engine, based on the service node name in each log record and the call identifier in the log content, the entry, exit and internal processing nodes of the request on each service node are identified, and a call sequence diagram with time as the horizontal axis and service nodes as the vertical axis is constructed. S47: For asynchronous task scenarios that include child TraceIDs, the link reconstruction engine queries the asynchronous task registry based on the parent TraceID to obtain a list of all associated child TraceIDs; S48: Execute steps S44 to S46 for the parent TraceID and each child TraceID respectively to obtain multiple sets of log sequences; S49: Based on the timestamps of each log sequence, align the execution time periods of the parent task and each child task in parallel to restore the complete parallel call sequence and output it in the form of a Gantt chart.

9. The method for full-process tracing and performance analysis of trace_id in a distributed service link according to claim 1, characterized in that, The specific sources of the positioning performance bottleneck include: For any adjacent upstream or downstream link in a single request call chain, collect the total processing time of the upstream link A. And the pure processing time of downstream stage B Historical sample data; in This includes the time taken for A to call B and wait for its return. This does not include the time spent by B calling its downstream processes; Based on the historical sample data, calculate the delay propagation coefficient between upstream link A and downstream link B. ; The calculated delay propagation coefficient As a measure of the intensity of time-consuming fluctuations transmitted from downstream link B to upstream link A: When | When | approaches 1, it is determined that the abnormal time consumption of upstream A is mainly caused by the time consumption fluctuation of downstream B; when | When | approaches 0, the causal relationship between the fluctuations in the time consumption of the two is relatively weak; Among them, the The calculation formula is: In the formula, the for and covariance, for standard deviation for The standard deviation.

10. The method for full-process tracing and performance analysis of trace_id in a distributed service link according to claim 9, characterized in that, The specific sources of the positioning performance bottleneck also include: Get the actual time spent in upstream step A of the current request. and the actual time spent in downstream link B And query the historical median time of stage A and stage B from the historical database. and ; Through the delay propagation coefficient Calculate the net root cause contribution of downstream stage B relative to upstream stage A. ; when When >0, B is identified as a candidate root cause node; when When = 0, the abnormal time consumption of B is determined to be entirely caused by upstream transmission or normal fluctuations, and should not be marked as the root cause. The calculation formula is: In the formula, This indicates the degree to which the time consumed in downstream segment B deviates from its historical normal level. This indicates the degree of deviation in the time consumption of upstream link A. Used to suppress root cause misjudgments caused by strong transmission relationships; when At that time, the pass Replacement, the It is a very small positive number.

Images