Service calling link determination method and device, computer equipment and storage medium

Abstract

The invention relates to a service calling link determination method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a log file, and analyzing the log file to obtain multiple pieces of service operation information corresponding to each request chain; wherein each piece of service operation information corresponds to one service operation; classifying the service operation according to the multiple pieces of service operation information, and determining an operation calling link corresponding to each request chain; and determining a service calling link corresponding to each request chain according to the operation calling link corresponding to each request chain and the service operation information corresponding to each service operation. By adopting the method, the service calling link corresponding to each request chain can be determined in a high-concurrency environment, and when the calling chain crossing a plurality of service applications is queried, the calling path between the service applications can be quickly positioned through the complete service calling link of each request chain, so that the query accuracy and efficiency are remarkably improved, and the query efficiency is improved. And thus, the fault positioning accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of software development technology, and in particular to a method, apparatus, computer equipment, storage medium and computer program product for determining a service call link. Background Art

[0002] In modern distributed and microservice architectures, various services and system components work together through call relationships to complete complex business functions. These services are usually composed of multiple service operations, each of which executes specific business logic and calls other services through the network. As the scale of business expands and the number of services increases, the call relationship between services becomes complex. How to effectively track and analyze service call links has become a key issue in distributed system monitoring and optimization.

[0003] Traditional techniques typically rely on logs, metrics, and simple aggregate analysis to determine service call information.

[0004] However, traditional methods cannot effectively track the fine-grained calling relationships between services in a high-concurrency environment. Especially when it comes to cross-service calls, the tracing of service links often cannot cover the entire chain, making it difficult to locate faults and analyze performance bottlenecks. Summary of the invention

[0005] Based on this, it is necessary to provide a service call link determination method, device, computer equipment, computer-readable storage medium and computer program product that can determine the service call link corresponding to each request chain in a high-concurrency environment, thereby improving the accuracy of fault location, in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for determining a service call link, comprising:

[0007] Obtaining a log file, parsing the log file to obtain a plurality of service operation information corresponding to each request chain; wherein each service operation information corresponds to a service operation;

[0008] Classify and process the service operations according to multiple service operation information, and determine the operation call link corresponding to each request chain;

[0009] According to the operation call link corresponding to each request chain and the service operation information corresponding to each service operation, the service call link corresponding to each request chain is determined.

[0010] In one embodiment, the service operations are classified and processed according to the multiple service operation information, and the operation call links corresponding to the request chains are determined, including:

[0011] Obtain the corresponding request chain identifier, service operation identifier and parent service operation identifier in each service operation information;

[0012] Classify each service operation according to the request chain identifier to obtain a service operation set corresponding to each request chain;

[0013] A filter is used to perform deduplication processing on the service operation set corresponding to each request chain to obtain a deduplication processing service operation set corresponding to each request chain;

[0014] Based on the deduplicated service operation set corresponding to each request chain, the operation call link corresponding to each request chain is determined according to the service operation identifier and the parent service operation identifier.

[0015] In one embodiment, determining the service call link corresponding to each request chain according to the operation call link corresponding to each request chain and the service operation information corresponding to each service operation includes:

[0016] Obtain the service name identifier in the service operation information corresponding to each service operation;

[0017] Each service operation in the operation call link corresponding to each request chain is regarded as a node, and each call relationship in the operation call link is regarded as an edge;

[0018] Using the service name identifier of each service operation as the node identifier of the node corresponding to each service operation;

[0019] Determine the service call link corresponding to each request chain based on the nodes, edges and node identifiers.

[0020] In one embodiment, the method further comprises:

[0021] Obtaining the target task information corresponding to each service operation information, splitting the target task information, and obtaining the split target task information;

[0022] Integrate the split target task information with the service operation information to obtain service operation reorganization information;

[0023] According to the service operation reorganization information, the service operations in each request chain are analyzed.

[0024] In one embodiment, the method further comprises:

[0025] storing the service operation reorganization information in a target database in a preset dimension, and determining a document identifier of each document in the target database in a target naming format;

[0026] According to the target database, multiple indicators of the service application corresponding to each service operation are determined.

[0027] In one embodiment, a log file is obtained and parsed to obtain multiple service operation information corresponding to each request chain, including:

[0028] Get the log file from the target queue;

[0029] The log file is parsed by a data parser to extract a plurality of service operation information corresponding to each request chain; wherein each service operation information includes a service operation identifier of its corresponding service operation.

[0030] In a second aspect, the present application also provides a service call link determination device, including:

[0031] A parsing module is used to obtain a log file, parse the log file to obtain a plurality of service operation information corresponding to each request chain; wherein each service operation information corresponds to a service operation;

[0032] A classification module is used to classify the service operations according to the multiple service operation information and determine the operation call link corresponding to each request chain;

[0033] The determination module is used to determine the service call link corresponding to each request chain according to the operation call link corresponding to each request chain and the service operation information corresponding to each service operation.

[0034] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0035] Obtaining a log file, parsing the log file to obtain a plurality of service operation information corresponding to each request chain; wherein each service operation information corresponds to a service operation;

[0036] Classify and process the service operations according to multiple service operation information, and determine the operation call link corresponding to each request chain;

[0037] According to the operation call link corresponding to each request chain and the service operation information corresponding to each service operation, the service call link corresponding to each request chain is determined.

[0038] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0039] Obtaining a log file, parsing the log file to obtain a plurality of service operation information corresponding to each request chain; wherein each service operation information corresponds to a service operation;

[0040] Classify and process the service operations according to multiple service operation information, and determine the operation call link corresponding to each request chain;

[0041] According to the operation call link corresponding to each request chain and the service operation information corresponding to each service operation, the service call link corresponding to each request chain is determined.

[0042] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0043] Obtaining a log file, parsing the log file to obtain a plurality of service operation information corresponding to each request chain; wherein each service operation information corresponds to a service operation;

[0044] Classify and process the service operations according to multiple service operation information, and determine the operation call link corresponding to each request chain;

[0045] According to the operation call link corresponding to each request chain and the service operation information corresponding to each service operation, the service call link corresponding to each request chain is determined.

[0046] The above-mentioned service call link determination method, device, computer equipment, storage medium and computer program product obtain log files, parse the log files to obtain multiple service operation information corresponding to each request chain; wherein each service operation information corresponds to a service operation; classify the service operations according to the multiple service operation information, and determine the operation call link corresponding to each request chain; determine the service call link corresponding to each request chain according to the operation call link corresponding to each request chain and the service operation information corresponding to each service operation. The present application can determine the service call link corresponding to each request chain in a high-concurrency environment, thereby improving the accuracy of fault location. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 An application environment diagram of a method for determining a service call link in an embodiment;

[0049] Figure 2 A schematic diagram of a flow chart of a method for determining a service call link in an embodiment;

[0050] Figure 3 A schematic diagram of a process for analyzing service operations in each request chain in an embodiment;

[0051] Figure 4 It is a structural block diagram of a device for determining a service call link in one embodiment;

[0052] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] The service call link determination method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 sends a service call link determination request to the server 104, and the server 104 receives the service call link determination request, obtains the log file, and parses the log file to obtain multiple service operation information corresponding to each request chain; wherein each service operation information corresponds to a service operation; the service operation is classified and processed according to the multiple service operation information, and the operation call link corresponding to each request chain is determined; according to the operation call link corresponding to each request chain and the service operation information corresponding to each service operation, the service call link corresponding to each request chain is determined. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0055] In modern highly distributed and microservice-based information technology (IT) architectures, the call relationships between services have become extremely complex, the number of services is huge and changes dynamically, and traditional monitoring and analysis methods are difficult to fully and effectively capture and analyze these call links. Traditional monitoring technologies can often only rely on logs, indicators, and simple aggregation analysis, but these methods cannot effectively track the fine-grained call relationships between services in a real-time, high-concurrency environment. Especially when it comes to cross-service calls, the tracking of service links often cannot cover the entire chain, making it difficult to locate faults and analyze performance bottlenecks. In addition, the number of service instances in a microservice architecture is huge and constantly changing, and the call patterns between services are complex, which further increases the complexity of the problem. Therefore, distributed tracing technology has emerged.

[0056] Distributed tracing technology, such as Pinpoint (an open source distributed tracing system), can track the distributed requests of a single transaction, record the link relationship in real time according to the order of calls, and store the link relationship in an HBase table through encoding. In this way, distributed tracing, performance indicator monitoring, anomaly detection and alarming can be achieved.

[0057] However, HBase (an open source, distributed column storage database) table uses the row primary key as the unique index. When querying the service call link in the HBase table, the relevant data must be found by scanning the entire table. This will lead to extremely low query efficiency when the data volume is large, especially when real-time statistical indicators (such as the total number of application calls, the number of abnormal requests, the total call time, etc.) are required. It is often necessary to merge and calculate a large amount of detailed data, resulting in a high computational burden. Secondly, for link analysis across multiple application services, the traditional HBase query method cannot provide sufficient support, resulting in query timeouts, high latency, inaccurate query results, and other problems. This application proposes a service call link determination method, which determines the service call link corresponding to each request chain by analyzing the Pinpoint log, that is, dynamically rebuilding the complete service call link of each request chain, and each request chain includes service operation information corresponding to each service operation. In the case of a large amount of data, the target data can be found through the service operation information, without scanning the entire table to find the relevant data, which can significantly improve the query efficiency. At the same time, the complete service call link of each request chain can capture the upstream and downstream call relationships of the entire request chain. When querying call chains across multiple service applications, the complete service call link of each request chain can quickly locate the call path between service applications, significantly improving the accuracy and efficiency of the query.

[0058] In an exemplary embodiment, Figure 2 As shown, a service call link determination method is provided, and the method is applied to Figure 1 The server in the example is used to illustrate, including the following steps 202 to 206. Among them:

[0059] Step 202, obtaining a log file, parsing the log file to obtain a plurality of service operation information corresponding to each request chain; wherein each service operation information corresponds to a service operation.

[0060] Among them, the log file refers to the Pinpoint log, and the Pinpoint log refers to the log information automatically generated and recorded by Pinpoint when using the Pinpoint distributed tracing system. Service operation information refers to Span information. Span is a separate "work unit" for a service to process a request. In a complex system, the request chain usually spans multiple services. Each service generates a Span when processing a transaction, that is, a service operation in the request chain is mapped to a Span. Span information records the specific situation of the service operation (Span), and one Span information corresponds to a unique Span. Optionally, Span information can include the request chain identifier corresponding to the request chain where the Span is located, the service operation (Span) identifier, the parent service operation identifier, etc.

[0061] Step 204: classify the service operations according to the multiple service operation information, and determine the operation call links corresponding to each request chain.

[0062] For each service operation information (Span information), the Span information includes the request chain identifier corresponding to the request chain where the service operation (Span) corresponding to the service operation information is located. All spans are classified using the request chain identifier, and multiple spans with the same request chain identifier are taken as a set to obtain all spans in each request chain. Then, according to the Span information corresponding to each span in the request chain, the operation call link corresponding to the request chain is determined.

[0063] Step 206: Determine the service call link corresponding to each request chain according to the operation call link corresponding to each request chain and the service operation information corresponding to each service operation.

[0064] The span information includes the service application corresponding to the service operation (span) corresponding to the service operation information (i.e., the service itself corresponding to the span). For each request chain, the call relationship between any two spans in the request chain can be determined based on the span information corresponding to each span in the request chain, and the service call link corresponding to each request chain can be determined based on the call relationship and the service application corresponding to the span.

[0065] In the above-mentioned service call link determination method, a log file is obtained, and the log file is parsed to obtain multiple service operation information corresponding to each request chain; wherein each service operation information corresponds to a service operation; the service operation is classified and processed according to the multiple service operation information, and the operation call link corresponding to each request chain is determined; according to the operation call link corresponding to each request chain and the service operation information corresponding to each service operation, the service call link corresponding to each request chain is determined. The present application can determine the service call link corresponding to each request chain, that is, dynamically rebuild the complete service call link of each request chain, and each request chain includes the service operation information corresponding to each service operation. Therefore, in the case of a large amount of data, the target data can be found through the service operation information, without scanning the entire table to find the relevant data, which can significantly improve the query efficiency. At the same time, the upstream and downstream call relationship of the entire request chain can be captured through the complete service call link of each request chain. When querying the call chain across multiple service applications, the call path between service applications can be quickly located through the complete service call link of each request chain, which significantly improves the accuracy and efficiency of the query.

[0066] In an exemplary embodiment, a log file is obtained and parsed to obtain multiple service operation information corresponding to each request chain, including: obtaining a log file from a target queue; parsing the log file using a data parser to extract multiple service operation information corresponding to each request chain; wherein each service operation information includes a service operation identifier of its corresponding service operation.

[0067] The target queue refers to the Kafka (open source distributed stream processing platform) queue. Kafka is a distributed streaming message queue system used to process and transmit large amounts of data streams.

[0068] Optionally, write the log data generated by Pinpoint to the message queue (Kafka queue), and extract the Span information from the Pinpoint log through the data parser on the consumer side. The data parser is a tool that uses code to identify "Received Span=TSpan" in the Pinpoint log, and then extracts the required information through pre-designed matching rules and encapsulates it into Span information. Span is a separate "work unit" for a service to process a request, that is, Span is a job that processes the corresponding service application call transaction. The Span information contains various data related to the transaction job, and Span can also be regarded as an application.

[0069] The Span information extracted from the Pinpoint log by the consumer-side data parser can include traceId (request chain identifier), spanId (service operation identifier), parentSpanId (parent service operation identifier), applicationName (service name identifier), applicationId (service identifier), parentApplicationId (parent service identifier), startTime (operation start time), elapsed (operation time) and SpanEvent (target task information). Among them, traceId represents the unique identifier of a complete request chain, which spans multiple services. A complete request chain refers to the entire request process initiated by the user, from the first service to the last service, which may be called by multiple services. This request chain flows between multiple services. spanId represents the unique identifier of the current span in the request chain. spanId can indicate the position of a service in the entire request chain. Each span has a unique spanId to distinguish the processing process of different services in the same link. parentSpanId indicates the identifier of the parent Span of the current Span, which helps to track the operation call chain, that is, parentSpanId refers to the spanId of the upstream service of the current Span. applicationName indicates the name of the service currently processing the request, such as "user service", "order service", "payment service", etc. applicationId indicates the identifier of the service application corresponding to the current Span. parentApplicationId indicates the identifier of the upstream service application that initiated the current Span request. startTime indicates the start time of the current Span processing the service request. elapsed indicates the time taken by the current Span to process the service request (usually in milliseconds). SpanEvent indicates the specific operation or method call that occurs in a Span. A Span may contain multiple SpanEvents, which are used to record the time and results of multiple specific operations in a service request, that is, SpanEvent is the specific method call or operation that occurs in the Span during the service request processing process. In particular, when the current Span is the initial initiator of the request chain, the parentSpanId in the current Span information is recorded as -1. The Span information of the called service can be found through the SpanId of the current Span, and the Span information of the calling service can be found through the parentSpanId.

[0070] In this embodiment, the log data generated by Pinpoint is written into the message queue so that the log data can flow efficiently, ensuring that the log data is transmitted to the backend in real time and efficiently for parsing and processing. The data parser is used to parse the log file, and multiple service operation information corresponding to each request chain is extracted, so that the life cycle of each service operation (Span) can be accurately understood and restored, and the entire distributed service link can be effectively tracked. The Span information can be used to achieve complete monitoring and performance analysis across services. For example, the entire process of a request can be tracked through traceId and spanId to find possible performance bottlenecks, service call dependencies, and potential failure points. Moreover, the Span information can be used to visualize the call link, and the dependencies and call paths between services can be clearly displayed.

[0071] In an exemplary embodiment, service operations are classified and processed based on multiple service operation information to determine the operation call link corresponding to each request chain, including: obtaining the request chain identifier, service operation identifier and parent service operation identifier corresponding to each service operation information; classifying each service operation according to the request chain identifier to obtain a service operation set corresponding to each request chain; using a filter to deduplicate the service operation set corresponding to each request chain to obtain a deduplicated service operation set corresponding to each request chain; based on the deduplicated service operation set corresponding to each request chain, determine the operation call link corresponding to each request chain according to the service operation identifier and the parent service operation identifier.

[0072] The filter refers to a Bloom filter. Deduplication processing is performed on the service operation set corresponding to each request chain. This means that when processing a large amount of data, in order to avoid the insertion and storage of duplicate data, the Bloom filter can be used to avoid repeated processing of existing spans and reduce data redundancy.

[0073] For example, based on the Span information extracted from the Pinpoint log by the consumer-side data parser, the Span is classified and processed according to the traceId (request chain identifier). The traceId is a unique identifier that identifies a complete request chain in distributed tracing. All spans with the same traceId are grouped together to obtain a set of spans (service operations) corresponding to a request chain. This span set is called a trace object. A trace object usually represents a complete request chain. A trace contains multiple spans, and these spans record the call information and operation details of each service request in the complete request chain.

[0074] When each span is classified and processed to obtain a span set (a trace), a Bloom filter can be used to deduplicate the spans in a trace before the spans are classified. Specifically, the Bloom filter determines whether a span has been processed by combining traceId and SpanId, that is, the Bloom filter reduces the insertion and update operations of duplicate data by determining whether the combination of traceId and SpanId already exists. Although the Bloom filter allows a certain degree of false recognition rate, it has little effect on the overall accuracy of the system. At the same time, it can significantly reduce the demand for data update frequency, avoid frequent write operations to the database in subsequent data storage, and improve system performance.

[0075] After the trace is deduplicated, a deduplicated service operation set (trace / request chain) is obtained. In the deduplicated trace, the spans are associated with each other through spanId and parentSpanId. SpanId represents the unique identifier of the current operation, while parentSpanId represents the spanId of the parent service that initiated the current operation. Through spanId and parentSpanId, a complete operation call chain of the tracee / request chain can be constructed. Optionally, each span in the request chain is regarded as a node in the request chain, and the call relationship between the two nodes is determined by the association between spanId and parentSpanId.

[0076] In this embodiment, Span (service operation) is classified and processed according to the request chain identifier (traceId), and all spans of the same request chain can be aggregated together to form a trace (service operation set) object, thereby effectively organizing the complex operation call links in the distributed system, making the logical relationship of the operation call links clearer, and providing a basis for subsequent analysis and query. Using Bloom filters to deduplicate the service operation sets corresponding to each request chain can effectively reduce the insertion and update operations of duplicate data, prevent frequent data updates during subsequent warehousing, and thus improve storage performance. Through the association of traceId, SpanId (service operation) and parentSpanId (parent service operation identifier), the operation call link corresponding to the request chain can be clearly constructed, which helps to understand the call logic between service applications and can provide accurate path information for fault location, performance bottleneck analysis and optimization.

[0077] In an exemplary embodiment, the service call link corresponding to each request chain is determined based on the operation call link corresponding to each request chain and the service operation information corresponding to each service operation, including: obtaining the service name identifier in the service operation information corresponding to each service operation; taking each service operation in the operation call link corresponding to each request chain as a node, and taking each call relationship of the operation call link as an edge; taking the service name identifier of each service operation as the node identifier of the node corresponding to each service operation; and determining the service call link corresponding to each request chain based on the node, edge and node identifier.

[0078] For example, detailed Span information is obtained by parsing Pinpoint logs. Span information contains key fields such as traceId, spanId, parentSpanId and applicationName, that is, Span information can identify the call relationship of service operations and the association between service applications. Further, spans are classified according to traceId, and spans with the same traceId are classified as the same trace object, and duplicate spans are deduplicated through Bloom filters to ensure the uniqueness and efficiency of data. In a trace object (i.e., a complete request chain), the operation call link between spans (i.e., the call relationship between spans) is constructed through the spanId and parentSpanId fields in each span. For each request chain, all spans in the request chain are defined as nodes in Neo4j (graph database), and the call relationship between spans is defined as the edge between these nodes. Specifically, the current span is the A node, and the called span is the B node. The call relationship between them is represented by a line (i.e., edge), which can be in the form of "A calls B".

[0079] Each node in Neo4j has a corresponding node identifier, which can be represented by applicationName (the English name of the service application).

[0080] Optionally, applicationName is used as a node identifier to identify different service applications. If a node of a service application (recorded as a service node) already exists in the Neo4j graph database, the node information will be updated if it already exists in the Neo4j graph database. The node information can also add required attributes such as the computer room where the application service is deployed, the product to which it belongs, and other information. In the node of the current application, the deployed machine can be dynamically added as the parent node of the node, and the host where the application is deployed can be clearly and intuitively seen in the topology relationship. Write all nodes (Span) and edges (call relationships) in the trace object into the Neo4j graph database to form a complete service call link. The following is an example of the key Cypher statement for inserting data into the Neo4j graph database:

[0081] Insert node statement example:

[0082] MERGE (s:AppTopology {nId: 'service_a'}) SET s.name='service_a',s.timestamp = 1728531301,s.env = 'production',s.product = 'a product',s.host='127.0.0.1';

[0083] MERGE (s:AppTopology {nId: 'service_b'}) SET s.name='service_b',s.timestamp = 1728531301,s.env = 'production',s.product = 'a product',s.host='127.0.0.1',

[0084] AppTopology corresponds to the table name of the graph database, nId and name correspond to the name of the Span (that is, the English name of the application), name is the name displayed on the node, timestamp is the insertion time, env is used to distinguish between the production environment and the test environment, and host is the host IP where the application is deployed;

[0085] Insert edge relationship statement example:

[0086] MATCH (startNode:AppTopology {nId: 'service_a'}) MATCH (endNode:AppTopology {nId:'service_b'})

[0087] MERGE (startNode)-[s:INVOKE]->(endNode) SET s.timestamp =1728531301",

[0088] Among them, startNode is the current node, endNode is the pointed node, and INVOKE is the relationship between the two nodes, which is a calling relationship in this case.

[0089] In this embodiment, the applicationName in the Span information is used as the node identifier, and the call relationship (edge) between nodes is defined by spanId and parentSpanId, which can clearly reflect the service applications and the call paths between them. This topological relationship is not only complete, but also can dynamically adapt to changes in service calls, solving the problem that traditional flat data is difficult to intuitively display service relationships. Through the Neo4j graph database, the service call link is stored in the form of nodes and edges. Compared with traditional relational databases, graph databases are more efficient in processing such complex relationship queries. The dynamic insertion and update of nodes and edges ensures the flexibility of the system while avoiding duplicate storage and data redundancy.

[0090] In an exemplary embodiment, Figure 3 As shown, the method includes steps 302 to 306. Among them:

[0091] Step 302, obtaining the target task information corresponding to each service operation information, splitting the target task information, and obtaining the split target task information.

[0092] Step 304: Integrate the split target task information with the service operation information to obtain service operation reorganization information.

[0093] Step 306: Analyze the service operations in each request chain according to the service operation reorganization information.

[0094] The target task information refers to SpanEvent.

[0095] Exemplarily, each Span has multiple method calls (SpanEvent), and each method call (SpanEvent) in each Span is split into separate records to obtain target task information after the split processing, and the target task information and Span information after the split processing are integrated to obtain service operation reorganization information (reassembly information). The service operation reorganization information may include applicationName, traceId, Span spanId, parentSpanId, transactionId, whether an exception occurs (hasException), error code (errCode), start and end time of method execution (startElapsed, endElapsed), API ID (apiId) and timestamp (timestamp) and other fields. The service operation reorganization information is conducive to subsequent statistics and analysis, and can analyze the service operations in each request chain. The service operation reorganization information can be dynamically added and deleted when the data parser modifies the matching rules, and there is no need to modify the table structure.

[0096] In the previous exemplary embodiment, the method also includes: storing the service operation reorganization information in the target database with preset dimensions, and determining the document identifier of each document in the target database in the target naming format; and determining multiple indicators of the service application corresponding to each service operation according to the target database.

[0097] The preset dimension refers to the dimension of each day, the target database refers to Mongodb (a type of database), and the target naming format can be TOPO_YYYMMDD format.

[0098] The service operation reorganization information is stored in Mongodb by day, and each Mongodb document is named in the TOPO_YYYMMDD format, so that the daily call details between application services can be efficiently queried. Each record stored in Mongodb can be quickly retrieved through information such as applicationName, traceId, spanId, etc., and supports analysis of service call links within a specific time range. Using Mongodb's built-in functions, various key indicators of each service application (i.e., multiple indicators of service applications) can also be efficiently counted, such as out-degree (number of requests initiated by the application), in-degree (number of requests received), total number of requests, number of successful requests, number of error requests, total response time, etc. These indicators can help developers or operation and maintenance personnel understand the performance status of the application, monitor anomalies, optimize performance, and provide basic data to support service analysis.

[0099] In this embodiment, the target task information (SpanEvent) is split and processed to obtain the target task information after the split processing. Detailed records and analysis can be performed for each method call (SpanEvent), which can accurately determine the execution status, execution time, and whether an exception occurs for each method. This can help developers and operation and maintenance personnel better understand the performance of the service, especially in complex systems, and can track specific bottlenecks and problems. The service operation reorganization information is stored in the target database with preset dimensions, that is, the service operation reorganization information is stored in Mongodb according to the dimension of days, which can greatly improve the query efficiency of service call data, especially for the query of historical data, avoiding the retrieval of the full amount of data, thereby improving query performance. At the same time, by storing the service operation reorganization information (i.e., refined indicators and information) in Mongodb, detailed data support can be provided for subsequent troubleshooting, performance optimization, and problem location.

[0100] In an exemplary embodiment, in the Neo4j graph database and Mongodb obtained above, the call link graph of the associated service application can be quickly queried according to the conditions such as the name of a single or multiple service applications, call depth, time (optionally hours, minutes, seconds), whether the link is normal, etc., and the call volume, successful request volume, error request volume, average response time, link call start time, data storage time, total response time, etc. corresponding to each service application can be counted from Mongodb in real time. According to the preset thresholds of error request volume and total response time, the nodes and edges in the topology are highlighted, and clicking on the topology node can display the detailed requests of each service application calling and being called in real time.

[0101] The following is an example of a key statement to query a single application link topology graph from the Neo4j graph database:

[0102] MATCH (n:AppTopology)-[r*1..5]->(m:AppTopology) WHERE n.name in ['service_a'] RETURN n,m,r

[0103] The following is an example of key statements for querying multiple application link topology graphs from the Neo4j graph database:

[0104] MATCH (n:AppTopology)-[r*1..5]->(m:AppTopology) WHERE n.name in ['service_a','service_b'] RETURN n,m,r

[0105] Among them, n represents the starting node, m represents the ending node, r represents the relationship between the two nodes, 1..5 represents the link depth of the query, starting from the 1st layer to the 5th layer, and the depth can be modified. WHERE is the query condition, which can be flexibly modified. The name in the query condition is the node name, which supports multiple nodes. The above service_a and service_b are service names. The returned results are all the topological relationships associated with the two nodes and all the relationships between the two nodes extending downward to a depth of 5. RETURN is the execution result, which returns n (starting node object), m (ending node object), and r (relationship between nodes).

[0106] In this embodiment, by combining the graph structure storage of Neo4j graph database and the data statistics function of Mongodb, an efficient and flexible service call link query and performance monitoring system is provided. Through multi-dimensional real-time statistics and query capabilities, users can quickly obtain the call relationship and performance indicators between service applications, and highlight the abnormal parts according to the preset threshold, helping operation and maintenance personnel to quickly locate and troubleshoot faults, while also improving the flexibility and scalability of the system, thereby effectively improving the efficiency of troubleshooting, performance monitoring and system optimization.

[0107] In another embodiment, a method for determining a service call link is provided, the method comprising:

[0108] Obtain a log file from a target queue; parse the log file using a data parser to extract a plurality of service operation information corresponding to each request chain; wherein each service operation information corresponds to a service operation, and each service operation information includes a service operation identifier of the corresponding service operation.

[0109] Obtain the request chain identifier, service operation identifier and parent service operation identifier corresponding to each service operation information; classify each service operation according to the request chain identifier to obtain a service operation set corresponding to each request chain; use a filter to deduplicate the service operation set corresponding to each request chain to obtain a deduplicated service operation set corresponding to each request chain; based on the deduplicated service operation set corresponding to each request chain, determine the operation call link corresponding to each request chain according to the service operation identifier and the parent service operation identifier.

[0110] Obtain the service name identifier in the service operation information corresponding to each service operation; regard each service operation in the operation call link corresponding to each request chain as a node, and regard each call relationship of the operation call link as an edge; regard the service name identifier of each service operation as the node identifier of the node corresponding to each service operation; determine the service call link corresponding to each request chain based on the node, edge and node identifier.

[0111] Obtain the target task information corresponding to each service operation information, split the target task information to obtain the split target task information; integrate the split target task information with the service operation information to obtain service operation reorganization information; analyze the service operations in each request chain according to the service operation reorganization information.

[0112] The service operation reorganization information is stored in a target database with preset dimensions, and the document identifier of each document in the target database is determined in a target naming format; and multiple indicators of the service application corresponding to each service operation are determined according to the target database.

[0113] In this embodiment, the log data generated by Pinpoint is written into the message queue so that the log data can flow efficiently, ensuring that the log data is transmitted to the backend in real time and efficiently for parsing and processing. The data parser is used to parse the log file, and multiple service operation information corresponding to each request chain is extracted, so that the life cycle of each service operation (Span) can be accurately understood and restored, and the entire distributed service link can be effectively tracked. Through Span information, complete monitoring and performance analysis across services can be achieved. For example, the entire process of a request can be tracked through traceId and spanId to find possible performance bottlenecks, service call dependencies, and potential failure points. Moreover, the visualization of the call link can be achieved through Span information, and the dependencies and call paths between services can be clearly displayed. According to the request chain identifier (traceId), the Span (service operation) is classified and processed, and all spans of the same request chain can be aggregated together to form a trace (service operation collection) object, thereby effectively organizing the complex operation call link in the distributed system, making the logical relationship of the operation call link clearer, and providing a basis for subsequent analysis and query. Using Bloom filters to deduplicate the service operation sets corresponding to each request chain can effectively reduce the insertion and update operations of duplicate data, prevent frequent data updates during subsequent storage, and thus improve storage performance. Through the association of traceId, SpanId (service operation) and parentSpanId (parent service operation identifier), the operation call link corresponding to the request chain can be clearly constructed, which helps to understand the call logic between service applications and provides accurate path information for fault location, performance bottleneck analysis and optimization. Using applicationName in the Span information as the node identifier and defining the call relationship (edge) between nodes through spanId and parentSpanId can clearly reflect the call paths between each service application and between them. This topological relationship is not only complete, but also can dynamically adapt to changes in service calls, solving the problem that traditional flat data is difficult to intuitively display service relationships. Through the Neo4j graph database, the service call link is stored in the form of nodes and edges. Compared with traditional relational databases, graph databases are more efficient in processing such complex relationship queries. The dynamic insertion and update of nodes and edges ensures the flexibility of the system while avoiding duplicate storage and data redundancy.

[0114] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0115] Based on the same inventive concept, the embodiment of the present application also provides a service call link determination device for implementing the service call link determination method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more service call link determination device embodiments provided below can refer to the limitations of the service call link determination method above, and will not be repeated here.

[0116] In an exemplary embodiment, Figure 4 As shown, a service call link determination device is provided, including: a parsing module 402, a classification module 404 and a determination module 406, wherein:

[0117] The parsing module 402 is used to obtain a log file, and parse the log file to obtain a plurality of service operation information corresponding to each request chain; wherein each service operation information corresponds to a service operation;

[0118] A classification module 404 is used to classify the service operations according to the multiple service operation information and determine the operation call link corresponding to each request chain;

[0119] The determination module 406 is used to determine the service call link corresponding to each request chain according to the operation call link corresponding to each request chain and the service operation information corresponding to each service operation.

[0120] In an exemplary embodiment, the classification module 404 is also used to obtain the corresponding request chain identifier, service operation identifier and parent service operation identifier in each service operation information; classify each service operation according to the request chain identifier to obtain a service operation set corresponding to each request chain; use a filter to deduplicate the service operation set corresponding to each request chain to obtain a deduplicated service operation set corresponding to each request chain; based on the deduplicated service operation set corresponding to each request chain, determine the operation call link corresponding to each request chain according to the service operation identifier and the parent service operation identifier.

[0121] In an exemplary embodiment, the determination module 406 is also used to obtain the service name identifier in the service operation information corresponding to each service operation; take each service operation in the operation call link corresponding to each request chain as a node, and take each call relationship of the operation call link as an edge; take the service name identifier of each service operation as the node identifier of the node corresponding to each service operation; and determine the service call link corresponding to each request chain based on the node, edge and node identifier.

[0122] In an exemplary embodiment, the service call link determination device further includes:

[0123] The analysis module is used to obtain the target task information corresponding to each service operation information, split the target task information to obtain the split target task information; integrate the split target task information with the service operation information to obtain service operation reorganization information; and analyze the service operations in each request chain according to the service operation reorganization information.

[0124] In an exemplary embodiment, the analysis module is also used to store the service operation reorganization information in a target database with preset dimensions, and determine the document identifier of each document in the target database in a target naming format; based on the target database, determine multiple indicators of the service application corresponding to each service operation.

[0125] In an exemplary embodiment, the parsing module 402 is also used to obtain a log file from the target queue; a data parser is used to parse the log file to extract multiple service operation information corresponding to each request chain; wherein each service operation information includes a service operation identifier of its corresponding service operation.

[0126] Each module in the above service call link determination device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0127] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 5As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store service operation data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a service call link determination method is implemented.

[0128] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0129] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0130] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0131] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0132] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0133] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0134] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for determining a service call link, characterized in that: The method comprises: Obtaining a log file, parsing the log file to obtain a plurality of service operation information corresponding to each request chain; wherein each service operation information corresponds to a service operation; Classify the service operations according to the plurality of service operation information, and determine the operation call link corresponding to each request chain; The service calling link corresponding to each request chain is determined according to the operation calling link corresponding to each request chain and the service operation information corresponding to each service operation.

2. The method according to claim 1, characterized in that The classifying and processing the service operations according to the plurality of service operation information to determine the operation call link corresponding to each request chain includes: Obtaining the request chain identifier, service operation identifier and parent service operation identifier corresponding to each of the service operation information; Classify each of the service operations according to the request chain identifier to obtain a service operation set corresponding to each of the request chains; Using a filter to perform deduplication processing on the service operation set corresponding to each of the request chains, to obtain a deduplication processing service operation set corresponding to each of the request chains; Based on the deduplicated service operation set corresponding to each of the request chains, the operation call link corresponding to each of the request chains is determined according to the service operation identifier and the parent service operation identifier.

3. The method according to claim 1, characterized in that The determining, according to the operation call link corresponding to each request chain and the service operation information corresponding to each service operation, the service call link corresponding to each request chain includes: Obtaining the service name identifier in the service operation information corresponding to each of the service operations; Each service operation in the operation call link corresponding to each request chain is regarded as a node, and each call relationship of the operation call link is regarded as an edge; Using the service name identifier of each service operation as the node identifier of the node corresponding to each service operation; The service call link corresponding to each of the request chains is determined according to the nodes, edges and node identifiers.

4. The method according to claim 1, characterized in that: The method further comprises: Obtaining target task information corresponding to each of the service operation information, splitting the target task information, and obtaining the split target task information; Integrate the split target task information with the service operation information to obtain service operation reorganization information; The service operations in each of the request chains are analyzed according to the service operation reorganization information.

5. The method according to claim 4, characterized in that The method further comprises: storing the service operation reorganization information in a target database with a preset dimension, and determining a document identifier of each document in the target database with a target naming format; According to the target database, a plurality of indicators of the service application corresponding to each of the service operations are determined.

6. The method according to claim 1, characterized in that Obtain a log file, and parse the log file to obtain multiple service operation information corresponding to each request chain, including: Get the log file from the target queue; The log file is parsed by a data parser to extract a plurality of service operation information corresponding to each request chain; wherein each of the service operation information includes a service operation identifier of the corresponding service operation.

7. A device for determining a service call link, characterized in that: The device comprises: A parsing module, used to obtain a log file, and parse the log file to obtain a plurality of service operation information corresponding to each request chain; wherein each service operation information corresponds to a service operation; A classification module, used to classify the service operations according to the plurality of service operation information, and determine the operation call link corresponding to each request chain; The determination module is used to determine the service call link corresponding to each request chain according to the operation call link corresponding to each request chain and the service operation information corresponding to each service operation.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images