A method and system for building a call chain based on benthos event stream

Abstract

The application relates to a Benthos event stream-based non-invasive call chain construction method and system, and belongs to the field of distributed system observability. The method comprises the following steps: a Benthos plug-in event stream access layer is used to batch-pull original events from various data sources; a multi-level CTE query is performed through a DuckDB memory engine to complete columnar preprocessing; a double-layer association algorithm combining a same-host time life cycle coverage greedy association and a cross-host stream queue time stamp greedy association is adopted to construct a complete call chain in combination with a delay correction and a self-recovery mechanism; a heartbeat-driven full-state aging mechanism is used to manage memory, a lock separation mechanism is adopted to persistently store stable call chain data into DuckDB, and the data is stored into a Parquet file in a time stamp partition mode. The application solves the problems of rigid architecture, low preprocessing efficiency, poor association robustness and chaotic storage management in the prior art, and has extremely high industrial application value.

Description

Technical Field

[0001] This invention belongs to the field of distributed system observability, and particularly relates to a method and system for constructing a call chain based on Benthos event streams. Background Technology

[0002] With the widespread adoption of microservice architectures, the call relationships in distributed systems are becoming increasingly complex, making call chain tracing a core tool for fault location, performance bottleneck analysis, and system optimization. Existing call chain construction technologies suffer from the following insurmountable drawbacks:

[0003] 1. Rigid architecture and poor scalability: Data access, preprocessing, association, and persistence are tightly coupled, and business logic is implemented using hard-coded methods. Adding new data sources or adjusting calculation rules requires modifying the code and restarting the service, making it unable to adapt to dynamically changing production environments.

[0004] 2. Low preprocessing efficiency: The cleaning, transformation and filtering of raw events are all processed line by line by general programming language, which cannot take advantage of the vectorized computing and query optimization capabilities of modern columnar databases. When processing massive amounts of events, the CPU utilization is high and the throughput is difficult to improve.

[0005] 3. Insufficient robustness of the association algorithm: It adopts full scan matching with a fixed time window, resulting in high average latency and lack of effective fault tolerance and self-healing mechanisms. It cannot handle the out-of-order and loss of events caused by network latency and system jitter, resulting in high false association rate and missed association rate.

[0006] 4. Disorganized storage and memory management: The lack of a unified aging mechanism for memory status can easily lead to memory leaks; persistent data is stored in a single table, resulting in extremely low efficiency in historical data querying and cleaning, which cannot support long-term storage of large-scale data.

[0007] Therefore, there is an urgent need for a non-intrusive distributed call chain construction method and system that features high throughput, low latency, high robustness, and scalability. Summary of the Invention

[0008] The purpose of this invention is to provide a non-intrusive call chain construction method and system based on Benthos event streams, so as to solve the problems of rigid architecture, low preprocessing efficiency, poor robustness of association, and chaotic storage and memory management in existing call chain construction technologies.

[0009] To achieve one of the aforementioned objectives, one embodiment of the present invention provides a call chain construction based on Benthos event streams, including:

[0010] The Benthos streaming engine batches raw events and writes them to temporary JSON files with timestamps in the filenames.

[0011] The Duck DB memory engine executes multi-level common table expression (CTE) queries, preprocessing raw events in temporary JSON files;

[0012] For the preprocessed results, the request events and their corresponding response events are merged into a complete Span;

[0013] Based on a two-layer greedy association mechanism, relationships are established between Spans to form a call chain.

[0014] As a further improvement to one embodiment of the present invention, the preprocessing result, which merges the request events and corresponding response events into a complete Span, includes:

[0015] For the preprocessing results, perform the following processing according to the event type: If it is a request event, check if there is a corresponding response event with the SpanID in the current cache. If so, merge the request event and the corresponding response event into a complete Span. Otherwise, write the request event to the cache. If it is a response event, check if there is a corresponding request event with the SpanID in the current cache. If so, merge the response event and the corresponding request event into a complete Span. Otherwise, write the response event to the cache.

[0016] As a further improvement to one embodiment of the present invention, the Duck DB memory engine executes multi-level common table expression (CTE) queries, and preprocesses the original events in the temporary JSON file, including...

[0017] After loading and reading the temporary JSON file, the DuckDB memory engine initiates a multi-level CTE query to complete the preprocessing of the original event. The multi-level CTE query includes JSON merging, field extraction, type conversion, automatic 5-tuple reversal, and abnormal data filtering.

[0018] Serialize the CTE query results into structured data.

[0019] As a further improvement to one embodiment of the present invention, the step of establishing the association relationship between Spans and forming a call chain based on the two-layer greedy association mechanism includes,

[0020] Traverse the Spans in the current batch in the original order of event slices, and perform the following processing on each Span:

[0021] If the target Span is an outbound Span, then based on the same host process-level greedy association mechanism, its parent node is searched within the same process on the same host. If found, the target Span and its parent node are associated, and the target Span's parent_spanID is set to the parent node's spanID, inheriting its parent node's TraceID, and its call level is set to the parent node's call level plus 1. The traceID and call level are then recursively propagated to all descendant nodes. If not found, the target Span is marked as pending correction and stored in the cache queue. Simultaneously, based on the cross-host network flow-level greedy association mechanism, the corresponding remote Span is searched between different hosts to establish a cross-host association. If found, a cross-host association is established, and the parent_spanID of the child Span is set to the parent node's spanID, inheriting the parent node's TraceID, and its call level is set to the parent node's call level plus 1. The traceID and call level are then recursively propagated to all descendant nodes. If not found, the target Span is marked as pending correction and stored in the cache queue.

[0022] If the target span is an inbound span, then based on the cross-host network flow-level greedy association mechanism, the corresponding remote span is searched for across different hosts to establish a cross-host association. If found, the cross-host association is established, and the parent_spanID of the child span is set to the spanID of the parent node, inheriting the TraceID of the parent node, and the call level is set to the call level of the parent node plus 1. The traceID and call level are then recursively propagated to all descendant nodes. If not found, the target span is stored in the cache queue. At the same time, based on the self-healing mechanism, the child nodes of the target span are searched for among the outbound spans marked as pending correction within the same process on the same host. If found, the target span and its child nodes are associated, the parent_spanID of the child node is set to the spanID of the parent node, inheriting the TraceID of the parent node, and the call level is set to the call level of the parent node plus 1. The traceID and call level are then recursively propagated to all descendant nodes. At the same time, the pending correction status of the found outbound span is removed.

[0023] Based on the established relationships, a call chain is formed;

[0024] Once the next batch of Spans arrives, a new round of traversal will begin, updating the call chain.

[0025] As a further improvement to one embodiment of the present invention, if the target Span is an outbound Span, then based on a greedy association mechanism at the same host process level, its parent node is searched within the same process on the same host, including:

[0026] 1.1 For the target Span, determine the search range of its parent nodes according to the following priority:

[0027] Highest priority: If the target Span carries a valid goroutine_id, then search for the parent node in the inbound Span of the same host, process, and goroutine.

[0028] Second highest priority: If the target Span does not carry a valid goroutine_id, but carries a valid tid, then search for the parent node in the inbound Span of the same host, process, and thread.

[0029] Lowest priority: If the target Span does not carry a valid goroutine_id and does not carry a valid tid, then search for the parent node in the inbound Span of the same host and process.

[0030] 1.2 Within the search range determined in step 1.1, obtain all inbound Spans without associated child nodes, and select the inbound Spans that satisfy the condition that "the start time of the inbound Span is less than or equal to the start time of the target Span, and the end time of the inbound Span is greater than or equal to the start time of the target Span" as candidate parent inbound Spans of the target Span; if there is a candidate parent inbound Span, proceed to step 1.4, otherwise mark the target Span as pending correction and store it in the cache queue;

[0031] 1.3 From all candidate parent inbound Spans, select the inbound Span with the latest start time and closest to the target Span as the parent node of the target Span;

[0032] 1.4. Associate the target Span with its parent node, set the target Span's parent_spanID to the parent node's spanID, inherit the parent node's TraceID, set the call level to the parent node's call level plus 1, and recursively propagate the traceID and call level to all descendant nodes.

[0033] As a further improvement to one embodiment of the present invention, the step of establishing a cross-host association by finding the corresponding remote Span among different hosts based on the cross-host network flow-level greedy association mechanism includes,

[0034] 2.1 Store all Spans in the current cache queue into the inbound flow queue and the outbound flow queue respectively, according to their corresponding four-tuples and event arrival times; where the four-tuples include source IP, source port, destination IP, and destination port;

[0035] 2.2 For the target Span, find the Span whose timestamps satisfy the causal relationship in the corresponding reverse flow queue. If found, select the Span with the smallest timestamp difference with the target Span and associate it with it. At the same time, remove it from the cache queue and execute step 2.4; otherwise, execute step 2.3.

[0036] 2.3. Determine if the target Span carries a valid URL. If it does, trigger the secondary backup association logic; otherwise, store the target Span in the cache queue. The secondary backup association logic specifically includes:

[0037] (1) Convert the source port of the target Span into a string, concatenate it with the corresponding URL string, and generate the corresponding joint hash value using a hash algorithm;

[0038] (2) Store all spans carrying valid URLs in the current cache queue into the second-level inbound hash queue and the second-level outbound hash queue respectively, and generate the corresponding joint hash value;

[0039] (3) For the target Span, search for Spans with the same joint hash value and whose timestamps satisfy the causal relationship in the corresponding secondary reverse hash queue. If found, select the Span with the smallest timestamp difference with the target Span and associate it, and execute step 2.4; otherwise, store the target Span in the cache queue.

[0040] 2.4 For an associated Span pair, set the parent_spanID of the Span that is a child node to the spanID of the parent node, inherit the TraceID of the parent node, set the call level to the call level of the parent node plus 1, and recursively propagate the traceID and call level to all descendant nodes.

[0041] 2.5 Remove the associated Span from the cache queue, the corresponding inbound or outbound flow queue, and the secondary inbound or outbound hash queue simultaneously.

[0042] As a further improvement to one embodiment of the present invention, the step of finding the child nodes of the target span among outbound spans marked as pending correction within the same process on the same host, based on the self-healing mechanism, includes:

[0043] (1) For the target Span, determine the search range of its child nodes according to the following priority:

[0044] Highest priority: If the target span carries a valid pid and goroutine_id, then child nodes are searched first in the outbound span of the same host, process, and goroutine.

[0045] Second highest priority: If the target Span carries a valid pid and tid, then search for child nodes in the outbound Span of the same host, process, and thread;

[0046] Lowest priority: If the target span carries a valid PID, then search for child nodes in the outbound spans of the same host and process.

[0047] (2) Within the defined search range, obtain all outbound Spans marked as pending correction status as candidate sub-outbound Spans;

[0048] (3) For each candidate outbound Span, determine whether the time lifecycle of the target Span covers the start time of the candidate outbound Span. If it does, the candidate outbound Span is a child node of the target Span. Associate the target Span with its child node, and the child node inherits the TraceID of its parent node. Set the call level to the call level of its parent node plus 1, and recursively propagate the traceID and call level to all descendant nodes. At the same time, cancel the pending correction status mark of the candidate outbound Span. Otherwise, store the target Span in the cache queue.

[0049] To achieve one of the above-mentioned objectives, an embodiment of the present invention also provides a call chain construction system based on Benthos event streams, including a Benthos pluggable event stream access layer, a CTE preprocessing layer, and a two-layer greedy association layer;

[0050] Benthos pluggable event stream access layer, used to batch pull raw events through the Benthos stream processing engine and write them to temporary JSON files with timestamps in the filenames;

[0051] The CTE preprocessing layer is used to execute multi-level common table expression CTE queries through the Duck DB memory engine, preprocess the raw events in the temporary JSON file, and merge the request events and corresponding response events in the preprocessing result into a complete Span.

[0052] The two-layer greedy association layer is used to establish associations between Spans based on the two-layer greedy association mechanism, forming a call chain.

[0053] To achieve one of the above-mentioned objectives, an embodiment of the present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and when the program is executed on the processor, it implements the steps in the call chain construction method based on Benthos event flow as described above.

[0054] To achieve one of the above-mentioned objectives, an embodiment of the present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps in the call chain construction method based on Benthos event stream as described above.

[0055] Compared with the prior art, the beneficial effects of this invention include:

[0056] 1. Extreme Scalability: Benthos' plug-in architecture supports rapid integration of new data sources and dynamic adjustment of processing flow through YAML configuration without requiring service restarts;

[0057] 2. Ultra-high preprocessing performance: DuckDB in-memory CTE preprocessing utilizes columnar vectorization computation, significantly reducing CPU utilization and improving system throughput;

[0058] 3. Industrial-grade robustness: Non-intrusive data collection covers all service types, and fault tolerance and self-healing mechanisms can handle out-of-order events and data loss, significantly reducing false association rates and missed association rates;

[0059] 4. Low-latency processing: The on-demand triggering matching mechanism effectively reduces processing latency, far superior to traditional fixed-window algorithms;

[0060] 5. Efficient storage management: The heartbeat aging mechanism ensures stable memory usage, and Parquet partitioned storage significantly improves the speed of historical data query and the efficiency of expired data cleanup. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of the overall architecture of the system described in this invention.

[0062] Figure 2 This is a flowchart of the DuckDB memory CTE multi-level preprocessing process.

[0063] Figure 3 This is the Span generation flowchart.

[0064] Figure 4 This is a flowchart of the two-layer greedy association and self-healing algorithm for outbound Span.

[0065] Figure 5 This is a flowchart of the two-layer greedy association and self-healing algorithm for inbound Span.

[0066] Figure 6 This is a flowchart of heartbeat-driven aging and partition persistence. Detailed Implementation

[0067] The present invention will now be described in detail with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of the present invention.

[0068] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0069] In one specific embodiment of the present invention, the overall system architecture of the present invention is as follows: Figure 1 As shown, a five-layer hierarchical design is adopted, with each layer communicating through a standard interface, achieving a high degree of decoupling and scalability.

[0070] 1. Benthos Plugin-Based Event Streaming Access Layer: Built on the Benthos stream processing engine, it provides unified event access capabilities. Through custom Bloblang plugins, it supports batch retrieval of raw events from various data sources. All plugins use a unified data exchange protocol, and the processing flow is defined through YAML configuration files, supporting hot reloading and allowing adjustments to processing logic without restarting the service.

[0071] 2. DuckDB In-Memory CTE Preprocessing Layer: Implemented based on the DuckDB columnar in-memory engine, this is the core performance optimization point of this invention. It transforms the processing logic of the original events from hard-coded code into multi-level CTE queries, leveraging DuckDB's vectorized computation and query optimizer to significantly improve preprocessing throughput. The preprocessing process is entirely completed in memory, requiring no disk I / O and resulting in extremely low latency.

[0072] 3. Two-layer Greedy Association and Self-Healing Layer: Implements a two-layer greedy association algorithm (same-host and cross-host) and a delay correction and self-healing mechanism. Same-host association is based on process context and time lifecycle, while cross-host association is based on network flow and timestamps. The combination of these two achieves high-precision call chain construction. The self-healing mechanism automatically corrects association errors caused by out-of-order events, ensuring the eventual consistency of the system.

[0073] 4. Heartbeat-driven full-state aging layer: Unifies the management of all memory states and automatically cleans up expired states that exceed a preset threshold through timed heartbeat tasks. Employing a lock-separated persistence mechanism, it separates memory operations from I / O operations, significantly shortening the global lock holding time and improving system throughput.

[0074] 5. DuckDB Partitioned Storage and Query Layer: Implements separate storage for hot and cold data. Real-time data is stored in DuckDB in-memory tables, supporting fast queries; historical data exceeding a preset time is automatically dumped into Parquet partitioned files, partitioned by timestamps, supporting fast queries and one-click expiration cleanup. An HTTP SQL query interface is provided, allowing users to query call chain data using standard SQL statements.

[0075] Specifically, the core of the Benthos event stream access layer is a custom set of Bloblang plugins, with all functionality implemented as plugins. Core plugins include:

[0076] ① Data source plugin: Used to retrieve raw events from various data sources;

[0077] ②Preprocessing plugin: Used to call DuckDB to execute preprocessed queries;

[0078] ③ Association Plugin: Used to invoke the call chain association algorithm;

[0079] ④ Tools and plugins: Used for temporary file management and partition cleanup.

[0080] Furthermore, the processing flow of the Benthos event stream access layer is defined through a YAML configuration file, as shown in the following example:

[0081] yaml input: generate: interval: "* / 10 * * * * * *" mapping: "root = {}" pipeline: processors: - bloblang: | let events = redis_list_all( address: "10.10.6.190:6379", db: 15, key: "sysdig_events", codec: "json", batch_count: 10000 ) let tmp_path = create_tmp( name: "events_%v.json".format(timestamp_unix()), content: $events.data ) let raw_spans = duckdb_query( sql: "SELECT * FROM read_json('%v')".format($tmp_path) ) let del = del_tmp(path: $tmp_path) x = traced_spans(spans: parse_json($raw_spans)) output: drop: {}

[0082] It should be noted that in the architecture of this invention, the core functions of each layer appear in the form of Benthos custom plugins. For example, the association plugin encapsulates all the algorithm logic for greedy association at the same host process level, greedy association at the cross-host network flow level, and self-healing correction; the tool plugin encapsulates utility classes for temporary file management and partition cleanup across multiple layers. Temporary file management serves the input / output process of the DuckDB memory CTE preprocessing layer (cleaning up temporary files to release disk space after preprocessing), and partition cleanup serves the DuckDB partition storage and query layer (cleaning up expired Parquet partition files).

[0083] Specifically, the DuckDB in-memory CTE preprocessing layer decomposes complex event handling logic into multiple simple steps through multi-level common table expressions, facilitating optimization and maintenance. The specific implementation steps are as follows:

[0084] 2.1 Batch Fetching and Temporary File Writing: Raw events are fetched in batches from the message queue using the Benthos data source plugin and written to a temporary JSON file. The temporary file path is specified by the system configuration, and the filename includes the current timestamp to avoid conflicts.

[0085] 2.2 Memory Table Loading: Temporary files are loaded into memory tables using DuckDB's file reading functions. The entire process requires no disk I / O and is extremely fast.

[0086] 2.3 Multi-level CTE query execution:

[0087] CTE1: JSON Merging: Merges multiple nested JSON fields from the original event into a single unified JSON object, facilitating subsequent field extraction;

[0088] CTE2: Field Extraction and Type Conversion: Extract core fields from the merged JSON object and convert them to standard data types;

[0089] CTE3: Automatic 5-tuple flip: Based on the direction field of the event, when the event is inbound, the source address and destination address, source port and destination port are swapped to unify the representation of the request direction;

[0090] CTE4: Abnormal Data Filtering: Filters out abnormal data with missing fields or incorrect formats to ensure the correctness of subsequent association algorithms.

[0091] 2.4 Result Output and Temporary File Cleanup: The CTE query results are serialized into structured Spans (call units) and passed to the association layer. After preprocessing, temporary files are automatically deleted to avoid disk space leaks.

[0092] Specifically, the core algorithm of this invention is a two-layer greedy association and self-healing mechanism consisting of greedy association at the same host process level and greedy association at the cross-host network flow level, which achieves high-precision and high-robustness call chain construction. It should be noted that the system maintains a request-response pairing cache before association processing—after the original event is preprocessed and output by CTE, requests and responses from the same call are first paired into a complete Span according to SpanID before entering association processing. If the request arrives before the response, the request is temporarily stored in the request cache awaiting the response; if the response arrives before the request, the response is temporarily stored in the response cache awaiting the request. The request is immediately removed from the cache after pairing is complete.

[0093] 3.1 Greedy association at the same host process level

[0094] Host association is used to establish parent-child relationships between Span calls within the same process on the same host. The core feature of this method is that the association is not unidirectional, but rather a bidirectional complementary process of "outbound search for parent (child to parent)" and "inbound search for child (parent to child)". Combined with multi-level identifier priority and the monotonically increasing uptime characteristic on one side, it achieves high-precision, high-robust association and self-healing.

[0095] 3.1.1 Outbound Lookup: Finds the parent inbound span for the outbound span.

[0096] For an outbound Span, it needs to find out which upstream inbound Span triggered it. The specific steps are as follows:

[0097] (1) Define the search scope. The outbound Span carries the host identifier, process identifier (pid), and optionally, the goroutine identifier (goroutine_id) and thread identifier (tid). The system determines the identifier combination key used to find the parent node according to the following priority:

[0098] Priority 1 (highest): host + pid + goroutine_id. If the outbound span carries a valid goroutine_id, it will prioritize searching in the inbound span of the same host, process, and goroutine.

[0099] Priority 2: host + pid + tid. If the outbound span does not carry a valid goroutine_id but carries a valid tid, then search in the inbound span of the same host, process, and thread.

[0100] Priority 3 (fallback): host + pid. If both identifiers of the outbound Span are invalid, then search the global inbound Span list for the same host and process.

[0101] It's important to note that if no valid match is found for any of the three priority levels mentioned above, and the current outbound Span carries a valid parent process identifier (ppid), then the host + ppid for cross-process degradation will be considered. The parent node will then be searched for inbound Spans on the same host and with the same parent process. This mechanism addresses the cross-process matching problem in containerized deployments where the sidecar proxy process and the business process belong to the same Pod but have different process IDs. It still falls under the category of a 'greedy association at the same host process level' mechanism.

[0102] (2) Filter candidate parent nodes. Within the defined search range, obtain all inbound Spans without associated child nodes, and select the inbound Spans that meet the necessary conditions to become parent nodes as candidate parent inbound Spans for the target outbound Span. The necessary conditions for an inbound Span to become a parent node are: the start time of the inbound Span ≤ the start time of the outbound Span, and the end time of the inbound Span ≥ the start time of the outbound Span. If there is a candidate parent inbound Span, proceed to step (3); otherwise, mark the target Span as pending correction for subsequent self-healing mechanism correction. It should be noted that the pending correction mark (pending_correction) indicates that the outbound Span has not yet found a parent node within the local host. It is not the same as the "unassociated Span" mentioned later. Only outbound Spans are marked as pending correction after the first step of searching with the host fails. Inbound Spans do not carry this mark; and the mark is removed when the self-healing mechanism successfully corrects the error.

[0103] (3) Select the optimal parent node. From all candidate parent inbound spans that meet the conditions, sort them according to the uptime of one side and monotonically increasing within the search range, and take the inbound span with the largest uptime (i.e. the latest start and the closest to the target outbound span) as the parent node.

[0104] (4) Establish association. The parent_spanID of the target outbound Span is set to the spanID of the parent node, inherits the TraceID of its parent node, and sets the call level to the depth of the parent node plus 1. The traceID and call level are then recursively propagated to all descendant nodes.

[0105] It should be noted that the initial value of TraceID is automatically generated by the system when the first Span arrives (based on the UUID of the timestamp and host identifier), and the initial value of the call level is 0, which identifies the root node of the call chain.

[0106] 3.1.2 Inbound Response: Based on the inbound span, reverse the search and correct the outbound sub-spans marked as pending correction.

[0107] When an inbound Span is completed, the system proactively initiates a reverse lookup to discover previously generated outbound child events for which a parent node has not yet been found. This is a crucial step in achieving self-healing.

[0108] (1) Define the search scope. Similar to the outbound search in 3.1.1, the search scope is determined according to the priority order of host + pid + goroutine_id → host + pid + tid → host + pid.

[0109] (2) Filter candidate child nodes. Within the defined search range, obtain all outbound spans that have not yet found a parent node, and use them as candidate child outbound spans.

[0110] (3) Matching and Correction. For each candidate outbound sub-event, check whether the time lifecycle of the current inbound Span covers the start time of the candidate outbound sub-event (the judgment condition is the same as for outbound lookup). If it covers, correct the parent node of the candidate outbound sub-event to the current inbound Span, associate the two, and set the parent_spanID of the candidate outbound sub-event to the spanID of the parent node, inherit the TraceID of its parent node, and set the call level to the call level of its parent node plus 1. This correction will be recursively propagated to all descendant nodes of the outbound event.

[0111] 3.2 Greedy Algorithm for Cross-Host Network Flows

[0112] Cross-host network flow-level greedy association is used to establish call relationships between different hosts. It adopts a two-layer matching mechanism of a four-tuple primary index + a source port-URL joint hash secondary index to solve the matching failure problem in scenarios such as Network Address Translation (NAT), container network IP reuse, and dynamic port allocation. The specific logic is as follows:

[0113] 3.2.1 Each network flow is uniquely identified by a four-tuple (source IP, source port, destination IP, destination port);

[0114] 3.2.2 All unpaired spans are stored in the inbound flow queue and the outbound flow queue respectively, and the queues are sorted by event arrival time;

[0115] 3.2.3 When a new Span is generated, the reverse flow queue of the corresponding quadruple is scanned first (outbound Span scans inbound flow queue, inbound Span scans outbound flow queue).

[0116] 3.2.4 Filter out unpaired Spans whose timestamps satisfy a causal sequence:

[0117] ① For a new departing Span: the start time of the arriving Span must be greater than the start time of the departing Span.

[0118] ② For newly arriving spans: the start time of the departing span must be less than the start time of the arriving span.

[0119] 3.2.5 From the candidate Spans that meet the conditions, select the Span with the smallest timestamp difference for pairing;

[0120] 3.2.6 If no unpaired Span meeting the conditions is found in the reverse flow queue of the corresponding quadruple, the secondary backup matching logic is triggered:

[0121] (1) Calculate the source port + URL combined hash index: Convert the source port of the current Span into a string and concatenate it with the complete URL string. Generate a unique hash value using the SHA-256 hash algorithm as the secondary index key;

[0122] (2) All unpaired Spans that carry a valid URL field are simultaneously stored in the corresponding secondary inbound hash queue and secondary outbound hash queue according to the combined hash value;

[0123] (3) Scan the reverse secondary hash queue that has the same combined hash value as the current Span, and repeat the timestamp causal filtering and optimal matching logic above;

[0124] (4) Second-level matching only applies to HTTP / HTTPS protocol events that carry a non-empty URL field. This logic is not triggered for TCP / UDP events without a URL.

[0125] 3.2.7 After a successful match, update the association relationship, set the parent_spanID of the child Span to the spanID of the parent Span, inherit the TraceID of the parent node, set the call level to the call level of the parent node plus 1, and recursively propagate the traceID and depth information to all descendant nodes.

[0126] 3.2.8 Remove paired spans from the corresponding first-level quadruple queue and second-level joint hash queue simultaneously to avoid duplicate matching.

[0127] It should be noted that the two-level federated hash index effectively solves the four-tuple conflict problem caused by the reuse of container IPs on different host machines in a container environment, as well as the four-tuple mismatch problem caused by changes in the source IP after NAT gateway translation. In microservice scenarios where the HTTP protocol is the primary protocol, it can improve the cross-host matching success rate by more than 30%, while maintaining the low-latency characteristics of on-demand matching.

[0128] 3.3 Delayed Correction and Self-Healing Mechanism

[0129] The core logic of the delayed correction and self-healing mechanism has been integrated into the "Inbound Response" process in section 3.1.2. When a new inbound event completes, the system automatically scans all unrelated outbound events under the same host and process and attempts to correct the relationships to achieve eventual consistency. For outbound events that have not found a parent node after exceeding the preset aging time, they are marked as root spans, and the waiting period ends.

[0130] Specifically, the heartbeat-driven aging and partition persistence mechanisms manage memory and disk resources in a unified manner to ensure stable system operation.

[0131] 4.1 Heartbeat-driven full-state aging

[0132] The system periodically triggers a heartbeat task to perform full-state aging cleanup:

[0133] 4.1.1 Calculate the expiration time threshold;

[0134] 4.1.2 Clean up all expired memory states, including request cache, response cache, stream queue elements, and inbound Span list on the same host;

[0135] 4.1.3. Filter out all expired and stable call chain data and copy them to a temporary buffer;

[0136] 4.1.4. Remove the filtered call chain data from memory to free up memory space.

[0137] 4.2 Lock Separation Persistence

[0138] The persistence process is divided into two phases, separating memory operations and I / O operations:

[0139] 4.2.1 Lock-in Phase: Holding the global state lock, performing the heartbeat aging and cleanup operations described above, filtering out the call chain data that needs to be persisted, and copying it to a temporary buffer. This phase only involves memory pointer operations and takes very little time.

[0140] 4.2.2, Out-of-Lock Phase: Release the global state lock, serialize the data in the temporary buffer into JSON format, write it to a temporary JSON file, and then write the data to the real-time in-memory table using DuckDB's batch import function. This phase involves I / O operations and is time-consuming, but it does not hold a global lock and does not affect the system's real-time processing capabilities.

[0141] 4.3 Parquet Partitioned Storage

[0142] DuckDB automatically dumps real-time data older than a preset time into Parquet partitioned files, partitioned by timestamp. Parquet is a high-efficiency columnar storage format with high compression ratio and fast query speed. Partitioned storage allows historical data queries to only scan the relevant partitions, eliminating the need for a full table scan and significantly improving query speed. Furthermore, expired data cleanup only requires deleting the corresponding partition directory, making it extremely efficient.

[0143] In the third embodiment of the present invention, combined with Figure 2 The four-level CTE chain preprocessing flow driven by the Benthos plugin and relying on the DuckDB memory engine is further elaborated. This flow completes the structured transformation of nested JSON events in memory throughout the entire process, avoiding unnecessary disk I / O and achieving a high-performance raw event cleaning and standardized output process.

[0144] I. Preliminary Data Preparation Stage

[0145] 1. The Benthos plugin retrieves raw event data in batches from the data source;

[0146] 2. Write the original event to a temporary JSON file with a timestamp in the filename to complete the temporary storage of data to disk;

[0147] 3. The Benthos plugin sends a request to the DuckDB memory engine to load the temporary file;

[0148] 4. DuckDB reads the entire contents of the target temporary JSON file, loads it into memory, and starts the preprocessing process.

[0149] Second and fourth level CTE chain processing stages

[0150] DuckDB executes four levels of Common Table Expressions (CTEs) sequentially in memory:

[0151] 1. CTE1: Merging Nested JSON Structures

[0152] Parse the multi-layered nested JSON fields in the original event, flatten and merge the scattered nested structure, eliminate data hierarchy redundancy, and prepare for subsequent field extraction.

[0153] 2. CTE2: Extract core fields and convert their types.

[0154] The core fields required for link analysis are selected from the merged structured data, and the field types are standardized and converted to a unified data format to adapt to subsequent calculation requirements.

[0155] 3. CTE3: Flip the quintuple based on the event direction.

[0156] Based on the inbound / outbound direction attributes of the event, the source and destination IP and source and destination port fields of the network 5-tuple are automatically flipped to ensure the direction consistency of the 5-tuple when associated across host links.

[0157] 4. CTE4: Filtering null values ​​and outlier data

[0158] Remove abnormal, dirty data such as missing fields, invalid formats, and out-of-bounds values, and output a clean and compliant structured event dataset.

[0159] III. Process Closure Stage

[0160] 1. After all preprocessing is complete, DuckDB returns the final structured event array to the Benthos plugin;

[0161] 2. The Benthos plugin deletes the temporary JSON files generated during the process, freeing up disk space and completing the entire process loop.

[0162] In the fourth embodiment of the present invention, combined with Figures 3 to 5 The association between Spans and the process of building the call chain are further explained. All operations in this process are scheduled by EventProcessor as the core scheduling node, which connects multiple dedicated components to complete the pairing of Spans, link association, and attribute propagation.

[0163] I. Event Reception and Span Generation Phase

[0164] After the EventProcessor receives the preprocessed event, such as Figure 3 As shown, the paired logic will be executed first according to the event type branch:

[0165] 1. Request event branch

[0166] The EventProcessor queries the ReqBuffer to check if a response event with the corresponding spanID exists.

[0167] ① If it exists: RespBuffer returns the cached response event, EventProcessor retrieves the cache and removes it from RespBuffer, directly merging the request and response to generate a complete Span;

[0168] ② If it does not exist: EventProcessor writes the current request into the ReqBuffer cache, marks the heartbeat time of the cache, and enters the state of waiting for subsequent response events.

[0169] 2. Event response branches

[0170] The EventProcessor queries the RespBuffer to check if a request event with the corresponding spanID exists.

[0171] ① If it exists: ReqBuffer returns the cached request event, EventProcessor retrieves the cache and removes it from ReqBuffer, directly merging the request and response to generate a complete Span;

[0172] ② If it does not exist: EventProcessor writes the current response into the RespBuffer cache, marks the heartbeat time of the cached entry, and enters a state of waiting for subsequent request events.

[0173] 3. Regardless of which branch is taken, after the complete Span is generated, the EventProcessor will set the initial heartbeat time for the Span and mark the start point of its life cycle.

[0174] II. Outbound Span (dir=0) Association Processing Stage

[0175] For the generated outbound span, such as Figure 4 As shown, the process will execute two levels of logic sequentially: same-host matching and cross-host association.

[0176] 1. Greedy matching with the same host

[0177] The EventProcessor calls SameHostMatcher to find the most recently inbound Span whose lifecycle completely includes the current outbound Span:

[0178] ① If a matching parent inbound Span is found: SameHostMatcher returns complete information about the parent Span, and EventProcessor sets three core link attributes for the current outbound Span: parent_spanID, traceID, and depth.

[0179] ② If no match is found: mark the outbound Span as pending correction, and wait for the self-healing logic to process it.

[0180] 2. Cross-host associated execution

[0181] The EventProcessor adds the outbound Span to the outflowQueue queue of the corresponding 5-tuple in the CrossHostMatcher; if the URL field of the Span is not empty, it also adds it to the second-level outflowHashQueue queue.

[0182] Trigger cross-host association logic, searching for a matching inbound Span in inFlowQueue:

[0183] ① Match found: Complete the first-level match, return the child Span to the EventProcessor, and simultaneously remove the two paired Spans from the first-level and second-level queues. Notify the TracesStore to recursively propagate the traceID and depth attributes.

[0184] ② No matching inbound site found: Check if the current outbound span carries a valid URL field:

[0185] A. If the URL is not empty: Calculate the combined hash index of the source port and the URL, and search for a matching inbound Span in the secondary inFlowHashQueue:

[0186] a. Find a matching entry point: Complete the secondary matching, return the child Span to the EventProcessor, and delete the two paired Spans from the primary and secondary queues. Notify TracesStore to recursively propagate the traceID and depth attributes.

[0187] b. No matching inbound event found: Leave the Span in the queue and enter the state of waiting for subsequent inbound events;

[0188] B. If the URL is empty: Leave the Span in the queue and enter a state of waiting for subsequent inbound events.

[0189] III. Inbound Span (dir=1) Association Processing Stage

[0190] For the generated inbound span, such as Figure 5 As shown, the process will sequentially execute queue enqueueing, cross-host association, and self-healing correction logic:

[0191] 1. Queue Enqueue Operation

[0192] The EventProcessor adds the inbound Span to the inFlowQueue queue of the corresponding 5-tuple in the CrossHostMatcher; if the URL field of the Span is not empty, it also adds it to the second-level inFlowHashQueue queue.

[0193] The EventProcessor adds the inbound Span to the SameHostMatcher's list of Spans in the same host and process.

[0194] 2. Cross-host associated execution

[0195] Trigger cross-host association logic to search for a matching outbound Span in the outFlowQueue:

[0196] ① Find a matching outbound: Complete the first-level matching, return the parent inbound Span to the EventProcessor, and at the same time remove the two paired Spans from the first-level and second-level queues, and notify TracesStore to recursively propagate the traceID and depth attributes;

[0197] ② No matching outbound: Check if the current inbound span carries a valid URL field:

[0198] A. If the URL is not empty: Calculate the combined hash index of the source port and the URL, and search for a matching outbound Span in the secondary outFlowHashQueue.

[0199] a. Find a matching outbound: Complete the secondary matching, return the parent inbound Span to the EventProcessor, and delete the paired Span from the primary and secondary queues. Notify TracesStore to recursively propagate the traceID and depth attributes.

[0200] b. No matching outbound event found: Leave the Span in the queue and enter the state of waiting for subsequent outbound events;

[0201] B. If the URL is empty: Leave the Span in the queue and enter a state of waiting for subsequent outbound events.

[0202] 3. Self-healing and correction logic

[0203] The EventProcessor triggers a self-healing correction process, scanning all outbound spans to be corrected under the same host and process in SameHostMatcher:

[0204] ① Find an associatable outbound Span: The EventProcessor corrects the association of the Span, sets the parent_spanID and other link fields for it, and notifies the TracesStore to recursively propagate the traceID and depth attributes;

[0205] ② No matching outbound span found: Keep these outbound spans in the pending correction state and wait for new spans to enter before trying to match again.

[0206] 4. After all the aforementioned related logic has been executed, the EventProcessor notifies the TracesStore to update the heartbeat time of all related Spans, covering the current Span and all recursively related descendant Span nodes, thus completing the entire process of this event.

[0207] In the fifth embodiment of the present invention, combined with Figure 6 The heartbeat-driven aging and partition persistence process is further elaborated. This is a closed-loop process based on timed heartbeat triggering for automatic memory state aging and data persistence to disk. It is completed collaboratively by four core components, ensuring both efficient reclamation of memory resources and reliable persistent storage of linked data. The Scheduler sends a timed task trigger signal to the StateManager to officially start this aging and persistence process.

[0208] I. Memory Status Aging and Cleanup Phase

[0209] 1. The StateManager first holds a global state lock to avoid data inconsistency issues caused by multiple threads concurrently modifying the state;

[0210] 2. Calculate the expiration time threshold: Subtract the preset aging time from the current time to define the boundary of all expired data that needs to be cleaned up;

[0211] 3. Perform multi-dimensional cleanup operations sequentially:

[0212] Clean up expired request and response caches;

[0213] Clean up expired inbound and outbound flow queues;

[0214] Clean up expired inbound lists for the same host;

[0215] 4. Filter out all expired and stable call chain data, delete this data from memory, and complete the memory resource reclamation;

[0216] 5. After all cleanup operations are completed, the StateManager releases the global state lock, thus releasing the exclusive lock on the global state.

[0217] II. Data Serialization and File Writing Stage

[0218] 1. The StateManager serializes the data to be persisted in the temporary buffer into standard JSON format;

[0219] 2. Send a write request to FileSystem to write the serialized data to a temporary JSON file;

[0220] 3. After completing the write operation, FileSystem returns a confirmation signal to StateManager confirming the successful write.

[0221] III. Database Batch Import and Partition Dump Stage

[0222] 1. The StateManager sends a batch import request to DuckDB to import the newly generated temporary JSON file into the real-time in-memory table;

[0223] 2. During the import process, DuckDB automatically executes background dump logic to dump all expired data into Parquet format partitioned files, achieving efficient archiving of cold data;

[0224] 3. Once the import and dump are complete, DuckDB returns a confirmation signal to the StateManager indicating that the import is complete;

[0225] 4. The StateManager sends a deletion request to the FileSystem to clean up the temporary JSON files generated during the process;

[0226] 5. After the FileSystem completes the deletion operation, it returns a confirmation signal to the StateManager indicating successful deletion. This completes the heartbeat-driven aging and partition persistence process.

[0227] In summary, this invention employs a five-layer end-to-end architecture: "Benthos pluggable event stream access - DuckDB memory CTE multi-level preprocessing - two-layer greedy association and self-healing - heartbeat-driven full-state aging - lock-separated partition persistence." This architecture solves the problems of rigid architecture, low preprocessing efficiency, poor association robustness, and chaotic storage management in existing technologies, and has extremely high industrial application value. Its core technical solution is as follows:

[0228] 1. Benthos event-driven pluggable access architecture

[0229] A unified event access layer is built based on the Benthos stream processing engine. All functional modules are implemented as custom Bloblang plugins. The processing flow is dynamically defined through YAML configuration files, allowing access to various data sources without modifying the code, and supports hot reloading configuration.

[0230] It should be noted that the innovation of this invention does not lie in Benthos / Bloblang itself, but in: (1) a plugin architecture system and overall processing pipeline built on the Benthos framework for call chain-oriented construction; (2) a technical solution that seamlessly integrates DuckDB CTE columnar preprocessing and two-layer greedy association algorithm into the Benthos event stream processing pipeline through a custom Bloblang plugin.

[0231] 2. The DuckDB memory engine's real-time CTE multi-level preprocessing writes batches of raw events to temporary files and executes multi-level common table expression (CTE) queries through the DuckDB memory engine. This sequentially completes JSON merging, field extraction, type conversion, automatic 5-tuple reversal, and outlier filtering, while leveraging columnar vectorized computation and the query optimizer to improve preprocessing throughput.

[0232] 3. Highly robust bi-layered greedy association and self-healing mechanism

[0233] Non-intrusive data collection: Based on kernel-mode network event collection technology, it requires no modification to any application code and supports all programming languages ​​and binary services;

[0234] Greedy association at the host process level: A time-lifecycle-covering greedy algorithm is used to select the nearest inbound Span whose lifecycle includes its start time as the parent node for the outbound Span;

[0235] Cross-host network flow-level greedy association: adopts a flow queue on-demand matching mechanism. When a new Span arrives, only the reverse flow queue of the corresponding quad is scanned, and the unpaired Span with the closest timestamp is selected for pairing.

[0236] Fault tolerance and self-healing: Early Spans are processed through bidirectional request / response caching. After a new inbound Span is completed, outbound Spans that are not associated with the same host and process are automatically scanned, and previous association errors are corrected to ensure eventual consistency.

[0237] 4. Heartbeat-driven full-state aging and partitioned persistence mechanism

[0238] Full-state heartbeat tagging: All memory states (request cache, response cache, stream queue, call chain) are marked with their last update timestamp, triggering a heartbeat task periodically;

[0239] Lock-separated persistence: The persistence process is divided into two stages: "in-lock filtering" and "out-of-lock I / O". The global lock is held only when filtering memory states, and all I / O operations are performed outside the lock, which greatly shortens the lock holding time.

[0240] Parquet partitioned storage: Aging stable call chain data is persisted to DuckDB, and existing data is automatically partitioned and stored as Parquet files according to timestamps, supporting fast querying and one-click expiration data cleanup.

[0241] In a sixth embodiment of the present invention, the present invention provides an electronic device including a memory and a processor. The memory stores a computer program that can run on the processor. When the program is executed on the processor, it implements the steps in the non-intrusive call chain construction method based on Benthos event flow as described above.

[0242] In Embodiment Seven of the present invention, the present invention provides a storage medium storing a computer program, which, when executed by a processor, implements the steps in the non-intrusive call chain construction method based on Benthos event streams as described above.

[0243] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the modules described above can be referred to the corresponding process in the aforementioned method implementation, and will not be repeated here.

[0244] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0245] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in a combination of hardware and software functional modules.

[0246] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer system (which may be a personal computer, server, or network system, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0247] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A call chain construction method based on Benthos event streams, characterized in that: include, The Benthos streaming engine batches raw events and writes them to temporary JSON files with timestamps in the filenames. The Duck DB memory engine executes multi-level common table expression (CTE) queries, preprocessing raw events in temporary JSON files; For the preprocessed results, the request events and their corresponding response events are merged into a complete Span; Based on a two-layer greedy association mechanism, relationships are established between Spans to form a call chain.

2. The method according to claim 1, characterized in that: The preprocessing results are then combined by merging the request events and their corresponding response events into a complete Span. include, For the preprocessing results, perform the following processing according to the event type: If it is a request event, check if there is a corresponding response event with the SpanID in the current cache. If so, merge the request event and the corresponding response event into a complete Span. Otherwise, write the request event to the cache. If it is a response event, check if there is a corresponding request event with the SpanID in the current cache. If so, merge the response event and the corresponding request event into a complete Span. Otherwise, write the response event to the cache.

3. The method according to claim 2, characterized in that: The Duck DB memory engine executes multi-level common table expression (CTE) queries and preprocesses the raw events in the temporary JSON file, including... After loading and reading the temporary JSON file, the DuckDB memory engine initiates a multi-level CTE query to complete the preprocessing of the original event. The multi-level CTE query includes JSON merging, field extraction, type conversion, automatic 5-tuple reversal, and abnormal data filtering. Serialize the CTE query results into structured data.

4. The method according to claim 1, characterized in that: The method for establishing relationships between Spans and forming a call chain based on a two-layer greedy association mechanism includes: Traverse the Spans in the current batch in the original order of event slices, and perform the following processing on each Span: If the target Span is an outbound Span, then based on the same host process-level greedy association mechanism, its parent node is searched within the same process on the same host. If found, the target Span and its parent node are associated, and the target Span's parent_spanID is set to the parent node's spanID, inheriting the parent node's TraceID, and the call level is set to the parent node's call level plus 1. The traceID and call level are then recursively propagated to all descendant nodes. If not found, the target Span is marked as pending correction and stored in the cache queue. Simultaneously, based on the cross-host network flow-level greedy association mechanism, the corresponding remote Span is searched between different hosts to establish a cross-host association. If found, a cross-host association is established, and the parent_spanID of the child Span is set to the parent node's spanID, inheriting the parent node's TraceID, and the call level is set to the parent node's call level plus 1. The traceID and call level are then recursively propagated to all descendant nodes. If not found, the target Span is marked as pending correction and stored in the cache queue. If the target span is an inbound span, then based on the cross-host network flow-level greedy association mechanism, the corresponding remote span is searched for across different hosts to establish a cross-host association. If found, the cross-host association is established, and the parent_spanID of the child span is set to the spanID of the parent node, inheriting the TraceID of the parent node, and the call level is set to the call level of the parent node plus 1. The traceID and call level are then recursively propagated to all descendant nodes. If not found, the target span is stored in the cache queue. At the same time, based on the self-healing mechanism, the child nodes of the target span are searched for among the outbound spans marked as pending correction within the same process on the same host. If found, the target span and its child nodes are associated, the parent_spanID of the child node is set to the spanID of the parent node, inheriting the TraceID of the parent node, and the call level is set to the call level of the parent node plus 1. The traceID and call level are then recursively propagated to all descendant nodes. At the same time, the pending correction status of the found outbound span is removed. Based on the established relationships, a call chain is formed; Once the next batch of Spans arrives, a new round of traversal will begin, updating the call chain.

5. The method according to claim 4, characterized in that: If the target span is an outbound span, then based on the greedy association mechanism at the same host process level, its parent node is searched within the same process on the same host, including: 1.1 For the target Span, determine the search range of its parent nodes according to the following priority: Highest priority: If the target Span carries a valid goroutine_id, then search for the parent node in the inbound Span of the same host, process, and goroutine. Second highest priority: If the target Span does not carry a valid goroutine_id, but carries a valid tid, then search for the parent node in the inbound Span of the same host, process, and thread. Lowest priority: If the target Span does not carry a valid goroutine_id and does not carry a valid tid, then search for the parent node in the inbound Span of the same host and process. 1.2 Within the search range determined in step 1.1, obtain all inbound Spans without associated child nodes, and select the inbound Spans that satisfy "the start time of the inbound Span ≤ the start time of the target Span, and the end time of the inbound Span ≥ the start time of the target Span" as candidate parent inbound Spans of the target Span; if there is a candidate parent inbound Span, proceed to step 1.4, otherwise mark the target Span as pending correction and store it in the cache queue; 1.3 From all candidate parent inbound Spans, select the inbound Span with the latest start time and closest to the target Span as the parent node of the target Span; 1.

4. Associate the target Span with its parent node, set the target Span's parent_spanID to the parent node's spanID, inherit the parent node's TraceID, set the call level to the parent node's call level plus 1, and recursively propagate the traceID and call level to all descendant nodes.

6. The method according to claim 5, characterized in that: The cross-host network flow-level greedy association mechanism, which involves finding the corresponding remote Span among different hosts to establish cross-host associations, includes... 2.1 Store all Spans in the current cache queue into the inbound flow queue and the outbound flow queue respectively, according to their corresponding four-tuples and event arrival times; where the four-tuples include source IP, source port, destination IP, and destination port; 2.2 For the target Span, find the Span whose timestamps satisfy the causal relationship in the corresponding reverse flow queue. If found, select the Span with the smallest timestamp difference with the target Span and associate it with it. At the same time, remove it from the cache queue and execute step 2.4; otherwise, execute step 2.

3. 2.

3. Determine if the target Span carries a valid URL. If it does, trigger the secondary backup association logic; otherwise, store the target Span in the cache queue. The secondary backup association logic specifically includes: (1) Convert the source port of the target Span into a string, concatenate it with the corresponding URL string, and generate the corresponding joint hash value using a hash algorithm; (2) Store all spans carrying valid URLs in the current cache queue into the second-level inbound hash queue and the second-level outbound hash queue respectively, and generate the corresponding joint hash value; (3) For the target Span, search for Spans with the same joint hash value and whose timestamps satisfy the causal relationship in the corresponding secondary reverse hash queue. If found, select the Span with the smallest timestamp difference with the target Span and associate it, and execute step 2.4; otherwise, store the target Span in the cache queue. 2.4 For an associated Span pair, set the parent_spanID of the Span that is a child node to the spanID of the parent node, inherit the TraceID of the parent node, set the call level to the call level of the parent node plus 1, and recursively propagate the traceID and call level to all descendant nodes. 2.5 Remove the associated Span from the cache queue, the corresponding inbound or outbound flow queue, and the secondary inbound or outbound hash queue simultaneously.

7. The method according to claim 6, characterized in that: The method of finding child nodes of the target span among outbound spans marked as pending correction within the same process on the same host, based on the self-healing mechanism, includes: (1) For the target Span, determine the search range of its child nodes according to the following priority: Highest priority: If the target span carries a valid pid and goroutine_id, then child nodes are searched first in the outbound span of the same host, process, and goroutine. Second highest priority: If the target Span carries a valid pid and tid, then search for child nodes in the outbound Span of the same host, process, and thread; Lowest priority: If the target span carries a valid PID, then search for child nodes in the outbound spans of the same host and process. (2) Within the defined search range, obtain all outbound Spans marked as pending correction status as candidate sub-outbound Spans; (3) For each candidate outbound Span, determine whether the time lifecycle of the target Span covers the start time of the candidate outbound Span. If it does, the candidate outbound Span is a child node of the target Span. Associate the target Span with its child node, and the child node inherits the TraceID of its parent node. Set the call level to the call level of its parent node plus 1, and recursively propagate the traceID and call level to all descendant nodes. At the same time, cancel the pending correction status mark of the candidate outbound Span. Otherwise, store the target Span in the cache queue.

8. A call chain construction system based on Benthos event streams, characterized in that: This includes a Benthos pluggable event stream access layer, a CTE preprocessing layer, and a two-layer greedy association layer; Benthos pluggable event stream access layer, used to batch pull raw events through the Benthos stream processing engine and write them to temporary files; The CTE preprocessing layer is used to execute multi-level common table expression CTE queries through a columnar memory engine to preprocess batches of raw events and obtain structured spans. The two-layer greedy association layer is used to establish associations between Spans and form a call chain based on the two-layer greedy association mechanism.

9. An electronic device, comprising a memory and a processor, characterized in that: The memory stores a computer program that can run on the processor, and when the program is executed on the processor, it implements the steps of the method as described in any one of claims 1-7.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the steps of the method as described in any one of claims 1-7.

Images