A method and system for invoking data analytics
By filtering and matching the Span type in microservice call data, the number and volume of data aggregations are reduced, solving the problem of high cost of microservice call data analysis and achieving efficient data storage and analysis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-04
- Publication Date
- 2026-03-20
AI Technical Summary
In existing technologies, microservices call a large number of data, resulting in large aggregated data volumes and high storage and analysis costs.
By acquiring the link data at the current moment, we can filter out Spans of the following types: client-side Span, server-side Span, entry Span, and exit Span. We can also pair and aggregate Spans from the same call to reduce the number of data aggregations and the amount of data.
It reduces the memory required to store aggregated data, decreases analysis costs, and improves the efficiency of data analysis.
Smart Images

Figure CN116383257B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of Internet, more particularly, to a calling data analysis method and system. BACKGROUND
[0002] With the development of Internet technology, the concept of microservices is becoming more and more popular. Microservices refers to splitting a service that combines all businesses into multiple sub-services. Each sub-service can be expanded or compressed in cluster quantity. At this time, there is a dependent call between each sub-service, and a technician cannot view the real-time state of the service from a global perspective. Therefore, when an abnormality occurs in a business line, it is difficult to distinguish or even impossible to distinguish which sub-service is abnormal. The service topology can check the specific local abnormality from a global perspective.
[0003] The service topology is to display the microservices deployed in a scattered manner through a topology diagram. The start node in the diagram refers to the entrance of the traffic, and the termination node refers to the exit of the traffic. Therefore, when a service is abnormal, the service topology can quickly locate the sub-service that is abnormal.
[0004] The service topology data needs to be obtained by analyzing the calling data of the interface, and the number of calling data of the interface is relatively large. The existing method for obtaining the service topology data mainly analyzes the calling data of the interface through a big data framework. The big data framework analyzes the calling data by aggregating the calling data. Since the number of calling data is large, the data volume of the aggregated data obtained by aggregating the calling data is also large. Therefore, the memory required for storing the aggregated data is large, resulting in a high cost of using the big data framework to analyze the calling data. SUMMARY
[0005] Therefore, the present application provides a calling data analysis method and system to solve the problem that in the existing calling data analysis method, the number of calling data is large, the data volume of the aggregated data obtained by aggregating the calling data is also large, and therefore, the memory required for storing the aggregated data is large, resulting in a high cost of using the big data framework to analyze the calling data.
[0006] In order to achieve the above purpose, the present scheme is as follows:
[0007] A calling data analysis method, comprising:
[0008] obtaining link data at a current time, wherein the link data is calling data in a calling chain generated by mutual calling between business applications, and the calling chain is composed of multiple spans;
[0009] filtering the link data to obtain four types of Spans, namely, a client Span, a server Span, an entry Span and an exit Span;
[0010] pairing the client Span and the server Span belonging to the same call in the client Span and the server Span filtered at the current time;
[0011] aggregating the request information of the entry Span, the exit Span and the paired Span filtered at the current time to obtain aggregated data.
[0012] An invocation data analysis system, comprising: a data filtering layer, a data matching layer and a data aggregation layer;
[0013] The data filtering layer is configured to receive link data at a current time, identify four types of Spans, namely, a client Span, a server Span, an entry Span and an exit Span in the link data, and push the identified client Span and server Span to the data matching layer and push the identified entry Span and exit Span to the data aggregation layer.
[0014] The data matching layer is configured to receive the client Span and the server Span pushed by the data filtering layer at the current time, pair the client Span and the server Span belonging to the same call, and push the paired Span to the data aggregation layer.
[0015] The data aggregation layer is configured to receive the entry Span and the exit Span pushed by the data filtering layer and the paired Span pushed by the data matching layer at the current time, aggregate the request information of the entry Span, the exit Span and the paired Span to obtain aggregated data.
[0016] From the above technical solution can be seen, the application embodiment provides a calling data analysis method, before the data is aggregated, the link data of the current moment obtained is screened to obtain four types of Span, the client Span, the server Span, the entry Span and the exit Span, then the client Span and the server Span in the current moment are screened to obtain the client Span and the server Span belonging to the same call, and finally the request information of the entry Span, the exit Span and the paired Span screened in the current moment is aggregated to obtain the aggregated data. The paired Span can represent the calling relationship between the internal business applications of the system, and the exit Span and the entry Span can represent the calling relationship between the internal business applications of the system and the external system, so that the request information of the paired Span, the exit Span and the entry Span is aggregated, the request information of the calling between the internal business applications of the system and the external system in the business implementation process can be obtained. The application does not aggregate all data in the link data, reduces the number of calling data that needs to be aggregated, and the data amount of the aggregated data obtained by the calling data aggregation is also small, so that the memory required for storing the aggregated data is less, and the cost in the calling data analysis process can be reduced. BRIEF DESCRIPTION OF DRAWINGS
[0017] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals can represent the same or similar elements. It should be understood that the drawings are schematic, and the original and elements are not necessarily drawn according to the scale.
[0018] Figure 1 A calling data analysis method flow chart disclosed by the embodiment of the application;
[0019] Figure 2 A data flow diagram in the calling data analysis method provided by the embodiment of the application;
[0020] Figure 3 A data flow diagram in another calling data analysis method provided by the embodiment of the application;
[0021] Figure 4 A topology query method flow chart provided by the embodiment of the application;
[0022] Figure 5 A topology result display interface provided by the embodiment of the application;
[0023] Figure 6 A calling data analysis system structure schematic diagram disclosed by the embodiment of the application;
[0024] Figure 7 A hardware structure block diagram of a calling data analysis device disclosed in an embodiment of the present application. DETAILED DESCRIPTION
[0025] Embodiments of the present disclosure will be described in more detail with reference to the drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein, but rather, these embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are merely for exemplary purposes, and are not intended to limit the scope of protection of the present disclosure.
[0026] The term "comprising" and variations thereof as used herein are open-ended, that is "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related definitions are given throughout the description.
[0027] It should be noted that the "first", "second", and the like concepts mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.
[0028] It should be noted that the modification of "one" or "multiple" mentioned in the present disclosure is illustrative and not limiting, and those skilled in the art should understand that unless otherwise explicitly indicated in the context, it should be understood as "one or more".
[0029] The calling data analysis method provided by the present application can be implemented based on any terminal with data processing capability, which can be a mobile phone, a computer, a server, a cloud, etc.
[0030] First, the calling data analysis method provided by the present application will be described in conjunction with the accompanying drawings. Figure 1 The method can include the following steps: Figure 1
[0031] Step S01, acquiring link data at the current time.
[0032] Specifically, the business application can be connected to the link collection system, and the calling data generated by the mutual calling between the business applications can be collected through the link collection system, wherein the link data is all calling links involved in a distributed request, and the calling link is composed of multiple spans. The structure of the span is shown in Table 1:
[0033] Table 1
[0034]
[0035]
[0036] Wherein, the Spans with the same traceID can constitute a link, and the Span ID of each call is a globally unique ID; references.spanID is the Span ID of the upstream Span of the Span in the call, which can represent the calling relationship between the Spans; duration is the total delay of the Span from creation to destruction; tag.span.kind is the kind of the Span, and the enumeration values can be server, client, etc.; the component name of tag.component can be a custom component such as gateway, jdbc, etc.; and tag.error indicates the calling condition, such as true indicating an exception and false indicating normal.
[0037] Step S02, filtering the link data.
[0038] Specifically, the obtained link data at the current time instant can include Span data irrelevant to the application calling data analysis scheme, and therefore, the obtained link data at the current time instant can be filtered according to a preset filtering rule to obtain four types of Spans, i.e., client Spans, server Spans, entry Spans and exit Spans.
[0039] Wherein, the process that initiates a request serves as a client, and the process that provides a service serves as a server, and one cross-process call can be recorded as a Span, which can include client Spans and server Spans. The call between the internal business application of a system and an external system can be involved, and in this case, no client Span and server Span can be generated. In order to better analyze the calling data, when the call between the internal business application of a system and an external system is involved, special client Spans and server Spans, i.e., entry Spans and exit Spans, can be generated. For example, when a user or an external system calls the internal application of a system through an entry application, the entry application can serve as an entry Span, and when an external system or a database is called externally through an exit application, the exit application can serve as an exit Span. Therefore, the client Spans, server Spans, entry Spans and exit Spans can be filtered out, and the request information for the call between the internal business applications of a system and the call between the internal business application of a system and an external system in the process of business implementation can be obtained. The configurations required for filtering the link data are shown in Table 2:
[0040] Table 2
[0041]
[0042]
[0043] Among them, the enumerated Span types can include: ingress Span: ROOT, egress Span: EXTERNAL, client Span: CLIENT, and server Span: SERVER, etc. Moreover, when the enumerated type is egress Span, the name of the service also needs to be specified.
[0044] conditions.tagKey is the keyword in the tag field of the Span data structure. Keywords for each type of Span can include: entry Span: component, exit Span: component, client Span: span.kind, and server Span: span.kind, etc.
[0045] conditions.tagValue is the value of the tag field in the Span data structure. The values of various types of Span can include: entry Span: gateway, exit Span: component, client Span: client, and server Span: server, etc.
[0046] Therefore, different types of Spans can be identified based on the set conditions.tagKey and conditions.tagValue, and the client-side Spans, server-side Spans, egress Spans, and ingress Spans in the link data can be filtered out.
[0047] Step S03: Pair the client-side Spans and server-side Spans that belong to the same call from the client-side Spans and server-side Spans selected at the current moment.
[0048] Specifically, a remote call generates two spans: a client span and a server span. Client spans and server spans belonging to the same remote call span from the currently filtered client spans and server spans can be paired.
[0049] Step S04: Aggregate the request information of the current entry Span, exit Span, and successfully paired Span to obtain aggregated data.
[0050] Specifically, the request information of the entry Span, the exit Span and the matched Span filtered at the current moment can be aggregated to obtain the aggregated data. The request information can include the number of requests, the time delay length, the number of call occurrence errors, the number of slow requests, the number of extremely slow requests and the number of normal requests.
[0051] The calling data analysis method provided by the embodiment of the application filters the obtained link data at the current moment to obtain four types of Spans, i.e., the client Span, the server Span, the entry Span and the exit Span, and then matches the client Span and the server Span belonging to the same call in the client Span and the server Span filtered at the current moment, and finally aggregates the request information of the entry Span, the exit Span and the matched Span filtered at the current moment to obtain the aggregated data. The matched Span can represent the calling relationship between the internal business applications in the system, and the exit Span and the entry Span can represent the calling relationship between the internal business applications in the system and the external system. Therefore, the request information of the matched Span, the exit Span and the entry Span is aggregated to obtain the request information of the calling between the internal business applications in the system and between the internal business applications in the system and the external system in the business implementation process. The application does not aggregate all the data in the link data, reduces the number of calling data that needs to be aggregated, and the data amount of the aggregated data obtained by aggregating the calling data is also small. Therefore, the memory required for storing the aggregated data is small, and the cost in the calling data analysis process can be reduced.
[0052] The above Figure 1 The flowchart in the above Figure 1 For example, two consecutive blocks noted in the flowchart can actually be executed in parallel, and they can also be executed in reverse order, depending on the functions involved. It should also be noted that each block in the flowchart, and a combination of blocks in the flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of special-purpose hardware and computer instructions.
[0053] In some embodiments of the present application, step S03, the client Span and the server Span belonging to the same call in the client Span and the server Span screened at the current time are paired, and the process can include:
[0054] Step S030, determining whether there is a server Span in the first queue with the same parent Span ID as the Span ID of the client Span screened at the current time, and a client Span with the same Span ID as the parent Span ID of the server Span screened at the current time.
[0055] Specifically, the Span generated by a call has a unique Span ID, and the Span ID of the client Span and the parent Span ID of the server Span belonging to the same call are the same, so it can be determined whether there is a server Span in the first queue with the same parent Span ID as the Span ID of the client Span screened at the current time, and a client Span with the same Span ID as the parent Span ID of the server Span screened at the current time.
[0056] The references.spanID of the server Span in the first queue can be found according to the spanID of the client Span screened at the current time, and the spanID of the client Span in the first queue can be found according to the references.spanID of the server Span screened at the current time.
[0057] If there is a server Span in the first queue with the same parent Span ID as the Span ID of the client Span screened at the current time, steps S031-S032 are executed, and if there is no server Span in the first queue with the same parent Span ID as the Span ID of the client Span screened at the current time, step S033 is executed.
[0058] Step S031, determining that the client Span screened at the current time and the server Span in the first queue with the same parent Span ID as the Span ID of the client Span screened at the current time are successfully paired.
[0059] Specifically, if there is a server Span in the first queue with the same parent Span ID as the Span ID of the client Span screened at the current time, it can be determined that the client Span screened at the current time and the server Span in the first queue with the same parent Span ID as the Span ID of the client screened at the current time belong to the same call.
[0060] Step S032, deleting the successfully paired server Span from the first queue.
[0061] Specifically, in order to make better use of the memory, after the successfully paired server Span is obtained by screening the client Span and the server Span with the same parent Span ID and the Span ID of the client at the current time, the successfully paired server Span can be deleted from the first queue.
[0062] Step S033, storing the client Span screened at the current time to the first queue.
[0063] Specifically, if there is no server Span with the same parent Span ID and the Span ID of the client Span screened at the current time in the first queue, it proves that there is no server Span in the first queue which belongs to the same call as the client Span screened at the current time, and then the client Span screened at the current time can be stored to the first queue to wait for pairing with the screened server Span.
[0064] If there is a client Span with the same parent Span ID and the Span ID of the server Span screened at the current time in the first queue, steps S034-S035 are executed, and if there is no client Span with the same parent Span ID and the Span ID of the server Span screened at the current time in the first queue, step S036 is executed.
[0065] Step S034, determining that the server Span screened at the current time is successfully paired with the client Span with the same parent Span ID and the Span ID of the server Span screened at the current time in the first queue.
[0066] Step S035, deleting the successfully paired client Span from the first queue.
[0067] Step S036, storing the server Span screened at the current time to the first queue.
[0068] Steps S034-S036 are the same as the process of Steps S031-S033, in which the client Span is matched with the server Span with the same parent Span ID and the Span ID of the client Span in the first queue, and details are not repeated here.
[0069] Embodiments of the present application consider that the Span ID of each call is unique, and the Span ID of the client Span and the parent Span ID of the server Span of the same call are the same. The client Span and the server Span of the same call can be accurately determined by the Span ID, and the client Span and the server Span of the same call can be quickly paired.
[0070] It is considered that the Span in the first queue may not be successfully matched for a long period of time, but occupies a part of the memory of the first queue. Therefore, after the parent Span ID of the server Span in the first queue is the same as the Span ID of the client Span filtered at the current time, and the Span ID of the client Span is the same as the parent Span ID of the server Span filtered at the current time, the embodiment of the present application at step S030 can further include:
[0071] At step S037, it is determined whether the Span at the head of the first queue is successfully paired within a preset time.
[0072] Specifically, since the Span in the first queue is mainly sorted by storage time, the earlier the Span is stored, the more front the Span is located in the first queue, and therefore, the Span at the head of the first queue is the first stored, and therefore, it can be determined whether the Span at the head of the first queue is successfully paired within a certain time.
[0073] If the Span at the head of the first queue is not successfully paired within a preset time, step S038 is performed.
[0074] At step S038, the Span at the head of the first queue is deleted from the first queue.
[0075] Specifically, if the Span at the head of the first queue is not successfully paired within a preset time, the Span at the head of the first queue at the current time can be deleted from the first queue.
[0076] Embodiments of the present application determine whether the Span at the head of the first queue is successfully paired within a preset time, and if the Span at the head of the first queue is not successfully paired within a preset time, the Span at the head of the first queue is deleted from the first queue, so as to avoid the situation that the Span at the head of the first queue cannot be successfully paired, but occupies the memory, and the memory can be better utilized.
[0077] Some embodiments of the present application introduce the process of aggregating the request information of the entry Span, the exit Span and the successfully paired Span obtained by filtering at the current time, which can include:
[0078] At step S040, the request information of the entry Span, the exit Span and the successfully paired Span obtained by filtering at the current time is incrementally aggregated with the request information of each span obtained within the time window in which the current time is located.
[0079] Specifically, the request information of the entry Span, the exit Span and the successfully paired Span obtained in a period of time needs to be counted, so the request information of the entry Span, the exit Span and the successfully paired Span obtained by filtering at the current time can be incrementally aggregated with the request information of each Span obtained within the time window in which the current time is located. For example, if the time window is set to 1 minute, and the request information of the client Span obtained in 1 minute is aggregated to obtain a total number of requests m, then after incrementally aggregating the request information of n client Spans that are successfully paired at the current time, the total number of client Span requests becomes m+n.
[0080] The request information of the client Span and the server Span aggregated is shown in Table 3:
[0081] Table 3
[0082] Name Definition clientCount Total number of client Spans clientDurationSum Total duration of client Spans clientErrorCount Total number of client Span errors clientNormalCallCount Total number of client Span normal requests clientSlowCallCount Total number of client Span slow requests clientExtremelySlowCallCount Total number of client Span extremely slow requests serverCount Total number of server Spans serverDurationSum Total duration of server Spans serverErrorCount Total number of server Span errors serverNormalCallCount Total number of server Span normal requests serverSlowCallCount Total number of server Span slow requests
[0083] The request information of the client Span and the server Span can be determined as normal request, slow request or extremely slow request according to the delay duration. The request with a delay duration less than or equal to a first duration can be determined as normal request, the request with a delay duration greater than the first duration and less than or equal to a second duration can be determined as slow request, and the request with a delay duration greater than the second duration can be determined as extremely slow request. The first duration and the second duration are set according to requirements, so the first duration and the second duration are not fixed. For example, the first duration is set to 500ms and the second duration is set to 1500ms according to requirements, so the request with duration<=500ms can be determined as normal request, the request with duration>500ms&&duration<=1500ms can be determined as slow request, and the request with duration>1500ms can be determined as extremely slow request.
[0084] The total number of errors of the server Span and the client Span can be the total number of tag.error=true of the client Span and the server Span.
[0085] The embodiment of the present application aggregates the request information of the entry span, the exit span and the successfully paired span filtered at the current time with the request information of each span obtained in the time window in which the current time is located. The request information of the span obtained at the current time is aggregated with the request information of each span obtained in the time window in which the current time is located once a span is obtained at the current time. Therefore, the request information of the span obtained in the time window can be aggregated in real time, and the aggregation result of the request information of the entry span, the exit span and the successfully paired span obtained at any time can be obtained.
[0086] After the data aggregation, the topology analysis result of the aggregated data can not be viewed in time. Therefore, after the request information of the entry span, the exit span and the successfully paired span filtered at the current time is aggregated to obtain the aggregated data, the embodiment of the present application can further include the following steps.
[0087] In step S05, the aggregated data is stored in the database.
[0088] Specifically, after the aggregated data of the request information of the entry span, the exit span and the successfully paired span filtered at the current time is obtained, the aggregated data can be stored in the database. When the topology of the aggregated data is needed, the required aggregated data can be extracted from the database.
[0089] The embodiment of the present application stores the aggregated data in the database after the aggregated data is obtained. When the topology analysis of the aggregated data is needed, the aggregated data required for the topology can be extracted from the database. The aggregated data does not need to be topologically analyzed in real time, and the topology analysis of the previous aggregated data can be viewed.
[0090] The embodiment of the present application is combined with the accompanying drawings serverExtremelySlowCallCount The flow process of the data in the calling data analysis method provided by the present application is introduced, and the flow process is as shown in Total number of server Span extremely slow requests
[0091] Firstly, the link data can be pushed to the message queue, then the link data in the message queue is subjected to the topology analysis service, and finally the data obtained after the topology analysis service is stored in the database.
[0092] The topology analysis service mainly performs three operations of data filtering, data matching and data aggregation on the link data in the message queue, and finally stores the data obtained after the data aggregation in the database.
[0093] In the data filtering process, link data is mainly filtered to obtain four types of Span, i.e., client Span, server Span, entry Span and exit Span. In the data matching process, the client Span and the server Span filtered at the current time are mainly matched to the client and the server belonging to the same call. In the data aggregation process, the request information of the entry Span, the exit Span and the matched Span filtered at the current time is mainly aggregated.
[0094] Embodiments of the present application combine Figure 2 The process of obtaining the message queue in the flow process of the foregoing data is introduced, and the process is as follows:
[0095] The user can operate through the operation page of the business application, such as accessing the Web page or interface of the business application. The user can generate link data in the process of accessing the Web page or interface of the business application, collect the link data through the link data collection system, and then push the collected link data to the message queue.
[0096] Further combining Figure 2 The process of providing the topology analysis service for the link data is introduced, and the process includes:
[0097] Firstly, the process of data filtering is introduced, and the process can include:
[0098] The link data is filtered to obtain the client Span, the server Span, the exit Span and the entry Span in the link data. The Span other than the client Span, the server Span, the exit Span and the entry Span in the link data is discarded. The specific process can refer to the introduction of step S02 in the foregoing.
[0099] Then, the process of data matching of the client Span and the server Span filtered by the data filtering is introduced, and the process can include:
[0100] The client Span or the server Span filtered by the data filtering before the current time is temporarily stored in the queue. The client Span filtered by the data filtering at the current time can be matched to the server Span belonging to the same call in the temporarily stored Span in the queue. Similarly, the server Span filtered by the data filtering at the current time can be matched to the client Span belonging to the same call in the temporarily stored Span in the queue. The specific process can refer to the introduction of step S03 in the foregoing.
[0101] If the client-side Span obtained by filtering the data at the current time matches the server-side Span in the temporarily-stored Spans in the queue successfully, it can be determined that the client-side Span obtained by filtering the data at the current time is successfully paired with the server-side Span that matches successfully.
[0102] If the server-side Span obtained by filtering the data at the current time matches the client-side Span in the temporarily-stored Spans in the queue successfully, it can be determined that the server-side Span obtained by filtering the data at the current time is successfully paired with the client-side Span that matches successfully.
[0103] If the Span at the head of the queue is not successfully paired within a certain time, the Span at the head of the queue is discarded from the queue.
[0104] Finally, the process of data aggregation is introduced, which can include:
[0105] The request information of the entry Span and the exit Span obtained by filtering the data and the successfully-paired Span obtained by matching the data is incrementally aggregated with the request information of each Span obtained within the time window in which the current time is located. When the time corresponding to the time window ends, the time window expires. When the time window expires, no more Spans will be entered into the time window, and at this time, the data in the time window will no longer change. Therefore, the request information of the entry Span and the exit Span obtained by filtering the data and the successfully-paired Span obtained by matching the data obtained within the time corresponding to the time window can be aggregated to obtain aggregated data. The specific process can be referred to the introduction of step S04 above. For example, the time window can be set to 1 minute, and when the time exceeds 1 minute, the time window expires, and no more Spans will be entered into the time window, and at this time, the data in the time window will no longer change. The request information of the entry Span and the exit Span obtained by filtering the data and the successfully-paired Span obtained by matching the data obtained within the 1 minute corresponding to the time window can be aggregated to obtain aggregated data.
[0106] Considering that after obtaining the aggregated data, the user needs to view the topology analysis result of the aggregated data, therefore, on the basis of the foregoing calling the data analysis scheme, the embodiments of the present application can further include:
[0107] Step S06, in response to the operation of the user query, the aggregated data is topologically displayed.
[0108] Specifically, in response to the operation of the user querying the service topology, the aggregated data is topologically analyzed and displayed to the user in the form of a topology graph.
[0109] For example,Figure 3 As shown, after the operation of responding to the user query, the aggregated data can be extracted from the database, the aggregated data is topological to obtain topological data, the topological data of the user query is aggregated, and finally the final service topology display is performed through the Web page. The user can view the service topology result on the Web page.
[0110] The embodiment of the present application can respond to the operation of querying the service topology of the user, perform topological analysis on the aggregated data, and display the topological graph to the user. The user can intuitively view the running status of each sub-service.
[0111] Since the user may only want to query the topological analysis result of a part of the aggregated data, the embodiment of the present application can further include, before the operation of responding to the user query and topologically displaying the aggregated data in step S06, the following steps.
[0112] In step S07, the query condition is determined in response to the operation of setting the query condition of the user.
[0113] Specifically, the operation of setting the query condition of the user can be responded to, wherein the query condition includes at least one of a time range, an application name, and an application version number. The user can query the topological analysis result of the aggregated data in the set time range by setting the time range, query the topological analysis result of the aggregated data of the application corresponding to the set name by setting the application name, and query the topological analysis result of the aggregated data of the application corresponding to the set version by setting the application version number.
[0114] In step S08, it is determined whether the aggregated data needs to be aggregated again according to the query condition.
[0115] Specifically, since the Span request information obtained in the preset time window is aggregated when the data is aggregated, the minimum time unit of the query is the length of the preset time window. If the user sets a large time range in the query condition, the aggregated data in the time range set by the user needs to be aggregated again. In addition, if the user wants to query the topological analysis result of the aggregated data of an application with a certain name or a certain version, the aggregated data of the application with the certain name or the certain version set by the user needs to be aggregated again.
[0116] If the aggregated data needs to be aggregated again, the following step S08 is performed. If the aggregated data does not need to be aggregated again, the above step S06 is performed.
[0117] In step S09, the aggregated data is aggregated again according to the query condition, and the aggregated data after the aggregation is obtained, which is used for topological display when the user queries.
[0118] Specifically, according to the query condition set by the user, the aggregated data corresponding to the query condition is secondarily aggregated, and the aggregated data obtained after secondary aggregation is topologically analyzed and displayed to the user in the form of a topological graph.
[0119] The embodiment of the present application responds to the operation of setting the query condition by the user, judges whether the aggregated data needs secondary aggregation according to the query condition set by the user, and if the aggregated data needs secondary aggregation, topologically displays the aggregated data obtained after secondary aggregation to the user. The user can set the query condition according to his own needs, and topologically display the aggregated data according to the query condition set by the user. Therefore, the needs of the user can be better met.
[0120] Figure 3 A topological result display interface provided by the embodiment of the present application is shown as follows: Figure 4 The user sets the query condition as the last 5 minutes and all applications. The aggregated data of all applications in the last 5 minutes is topologically displayed. Through the topological graph, the total number of calls, the delay duration and the number of errors of each application can be seen.
[0121] The embodiment of the present application can respond to the operation of setting the query condition by the user, and judge whether the aggregated data needs secondary aggregation according to the query condition. If the aggregated data needs secondary aggregation, the aggregated data is secondarily aggregated according to the query condition, and then topologically displayed. Different needs of the user for topological display of the aggregated data can be met.
[0122] The names of the related information in the method embodiment of the present application are only for illustrative purposes, and are not used to limit the scope of the information.
[0123] Although the operations in the method embodiment of the present application are described in a specific order, this should not be understood as requiring the operations to be performed in the specific order or in a sequential order. In certain circumstances, multitasking and parallel processing can be advantageous.
[0124] It should be understood that each step recorded in the method embodiment of the present application can be performed in different order and / or in parallel. In addition, the method embodiment can include additional steps and / or omit the execution of the shown steps. The scope of the present disclosure is not limited in this respect.
[0125] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).
[0126] The calling data analysis system provided by the embodiments of the present application is described below. The calling data analysis system described below can be referred to in correspondence with the calling data analysis method described above.
[0127] First, in combination with Figure 5 The calling data analysis system is introduced as shown in Figure 5 The calling data analysis system can include a data filtering layer 100, a data matching layer 200, a data aggregation layer 300,
[0128] The data filtering layer 100 is configured to receive link data at the current time, identify four types of Spans in the link data, i.e., a client-side Span, a server-side Span, an entry-side Span and an exit-side Span, push the identified client-side Span and server-side Span to the data matching layer 200, and push the identified entry-side Span and exit-side Span to the data aggregation layer 300. The link data is calling data in a calling chain generated by mutual calling between business applications. The calling chain is composed of multiple Spans.
[0129] The data matching layer 200 is configured to receive the client-side Span and server-side Span pushed by the data filtering layer 100 at the current time, pair the client-side Span and server-side Span belonging to the same calling, and push the paired Span to the data aggregation layer 300.
[0130] The data aggregation layer 300 is configured to receive the entry-side Span and exit-side Span pushed by the data filtering layer 100 at the current time and the paired Span pushed by the data matching layer 200, aggregate the request information of the entry-side Span, the exit-side Span and the paired Span, and obtain aggregated data.
[0131] The embodiment of the application provides a kind of to call data analysis system, data filtering layer 100 can filter out the Span except client Span, server Span, export Span and entry Span in link data, and data matching layer 200 can filter out the Span that cannot be successfully matched in the client Span and server Span filtered by data filtering layer 100, so data aggregation layer 300 only needs to aggregate the request information of entry Span and export Span pushed by data filtering layer 100 and the Span of successfully matched pair pushed by data matching layer 200 at current time, reduce the number of call data that need to be aggregated, and the data amount of aggregation data obtained by calling data aggregation is also smaller, therefore, the memory required for storing aggregation data is less, and the cost in the process of calling data analysis can be reduced.
[0132] Next, the embodiment of the application introduces the process of pairing client Span and server Span belonging to the same call received by data matching layer 200 from data filtering layer 100 at current time, which can include:
[0133] Step S11, receiving client Span and server Span pushed by data filtering layer 100 at current time.
[0134] Specifically, data matching layer 200 can receive client Span and server Span obtained by data filtering layer 100 filtering link data at current time.
[0135] Step S12, judging whether there is server Span with parent Span ID same as Span ID of client Span received by data matching layer 200 at current time and client Span with Span ID same as parent Span ID of server Span received by data matching layer 200 at current time in the first queue.
[0136] Specifically, since the Span ID of client Span and the parent Span ID of server Span belonging to the same call are the same, after data matching layer 200 receives client Span and server Span pushed by data filtering layer 100 at current time, it is judged whether there is server Span with parent Span ID same as Span ID of client Span received at current time and client Span with Span ID same as parent Span ID of server Span received at current time in the first queue.
[0137] If there is a server-side Span in the first queue whose parent Span ID is the same as the Span ID of the client-side Span received by the current-time data matching layer 200, steps S13-S14 are performed, and if there is no server-side Span in the first queue whose parent Span ID is the same as the Span ID of the client-side Span received by the current-time data matching layer 200, step S15 is performed.
[0138] In step S13, it is determined that the client-side Span received by the current-time data matching layer 200 is successfully paired with the server-side Span in the first queue whose parent Span ID is the same as the Span ID of the client-side Span received by the current-time data matching layer 200.
[0139] Specifically, if there is a server-side Span in the first queue whose parent Span ID is the same as the Span ID of the client-side Span received by the current-time data matching layer 200, it is proved that the client-side Span received by the current-time data matching layer 200 and the server-side Span in the first queue whose parent Span ID is the same as the Span ID of the client-side Span received by the current-time data matching layer 200 belong to the same call, and thus it can be determined that the client-side Span received by the current-time data matching layer 200 is successfully paired with the server-side Span in the first queue whose parent Span ID is the same as the Span ID of the client-side Span received by the current-time data matching layer 200.
[0140] In step S14, the successfully paired server-side Span is deleted from the first queue.
[0141] Specifically, after the client-side Span received by the current-time data matching layer 200 is successfully paired with the server-side Span in the first queue whose parent Span ID is the same as the Span ID of the client-side Span received by the current-time data matching layer 200, the server-side Span successfully paired with the client-side Span received by the current-time data matching layer 200 can be deleted from the first queue.
[0142] In step S15, the client-side Span received by the current-time data matching layer 200 is stored in the first queue.
[0143] Specifically, if there is no service-side Span in the first queue whose parent Span ID is the same as the Span ID of the client-side Span received by the current time data matching layer 200, it proves that there is no service-side Span in the first queue that belongs to the same invocation as the client-side Span received by the current time data matching layer 200, and therefore the client-side Span received by the current time data matching layer 200 can be stored in the first queue, waiting for matching with a service-side Span received by the current time data matching layer 200 in the future.
[0144] If there is a client-side Span in the first queue whose Span ID is the same as the parent Span ID of the service-side Span received by the current time data matching layer 200, steps S16-S17 are performed, and if there is no client-side Span in the first queue whose Span ID is the same as the parent Span ID of the service-side Span received by the current time data matching layer 200, step S18 is performed.
[0145] Step S16: Determine whether the service-side Span received by the current time data matching layer 200 is successfully paired with the client-side Span in the first queue whose Span ID is the same as the parent Span ID of the service-side Span received by the current time data matching layer 200.
[0146] Step S17: Delete the successfully paired client-side Span from the first queue.
[0147] Step S18: Store the service-side Span received by the current time data matching layer 200 in the first queue.
[0148] The processes of matching the service-side Span received by the current time data matching layer 200 with the client-side Span in the first queue described in steps S16-S18 are the same as the processes of matching the client-side Span received by the current time data matching layer 200 with the service-side Span in the first queue described in steps S13-S15, and will not be described here.
[0149] The embodiment of the present application introduces the process of the data aggregation layer 300 receiving the entry Span and the exit Span pushed by the current time data filtering layer 100 and the successfully paired Span pushed by the data matching layer 200, and aggregating the request information of the entry Span, the exit Span, and the successfully paired Span. The process can include:
[0150] The data aggregation layer 300 incrementally aggregates the request information of the inlet Span and outlet Span pushed by the data filtering layer 100 and the successfully paired Span pushed by the data matching layer 200, which are received at the current moment, with the request information of each Span that the data aggregation layer 300 has already obtained within the time window at the current moment.
[0151] Specifically, since it is necessary to statistically analyze the request information of the ingress and egress spans pushed by the data filtering layer 100 and the successfully paired spans pushed by the data matching layer 200 received by the data aggregation layer 300 over a certain period, the request information of the ingress and egress spans pushed by the data filtering layer 100 and the successfully paired spans pushed by the data matching layer 200 at the current moment can be incrementally aggregated with the request information of each span already obtained by the data aggregation layer 300 within the current time window. For example, if the time window is set to 1 minute, and the total number of client span requests obtained by the data aggregation layer 300 within the current 1 minute is m, then after incrementally aggregating the request information of the n successfully paired client spans pushed by the data matching layer 200 received by the data aggregation layer 300 at the current moment, the total number of client span requests becomes m+n.
[0152] The data analysis method provided in this application can be applied to data analysis devices. Figure 6 A hardware structure block diagram of a data analysis device 600 is shown. The data analysis device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (e.g., vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The application shown calling the data analysis device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments disclosed herein.
[0153] like Figure 7 As shown, the data analysis device 600 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 606 into a random access memory (RAM) 603. The RAM 603 also stores various programs and data required for operating the data analysis device 600. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0154] Generally, the following devices can be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; a storage device 608 including, for example, a magnetic tape, a hard disk, and the like; and a communication device 609. The communication device 609 can allow the invocation data analysis apparatus 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 The invocation data analysis apparatus 600 is shown with various devices, but it should be understood that all of the shown devices are not required to be implemented or present. More or fewer devices can alternatively be implemented or present.
[0155] The embodiments of the present application also provide a computer readable medium carrying one or more programs, when the one or more programs are executed by the invocation data analysis apparatus, the invocation data analysis apparatus is caused to: after obtaining link data at a current time, filtering the link data to obtain four types of Spans, i.e., a client-side Span, a server-side Span, an entry-side Span, and an exit-side Span, wherein the link data is invocation data in an invocation chain generated by mutual invocations between business applications, the invocation chain is composed of multiple Spans, then pairing the client-side Span and the server-side Span belonging to the same invocation in the client-side Span and the server-side Span filtered at the current time, and finally aggregating request information of the entry-side Span, the exit-side Span, and the paired Span filtered at the current time to obtain aggregated data.
[0156] It should be noted that the computer readable medium described above in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination of the above.
[0157] The computer readable medium described above can be contained in the calling data analysis device described above; or can exist separately and not be assembled into the calling data analysis device.
[0158] In particular, according to embodiments of the present disclosure, the processes described in the calling data analysis method embodiments above can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program comprising program code for executing the calling data analysis method described above. In such embodiments, the computer program can be downloaded and installed from the network by the communication device 609 in the terminal 600, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above functions defined in the calling data analysis method of the embodiments of the present disclosure are executed. Figure 7 Figure 7 Figure 7
[0159] According to one or more embodiments of the present disclosure, example 1 provides a calling data analysis method, comprising:
[0160] obtain link data of a current moment, wherein the link data is calling data in a calling chain generated by mutual calling between business applications, and the calling chain is composed of multiple spans;
[0161] filter the link data to obtain four types of spans, namely, client-side spans, server-side spans, entry-side spans, and exit-side spans;
[0162] pair client-side spans and server-side spans belonging to the same calling in the client-side spans and the server-side spans filtered at the current moment;
[0163] aggregate request information of the entry-side spans, the exit-side spans, and the paired spans filtered at the current moment to obtain aggregated data.
[0164] According to one or more embodiments of the present disclosure, example 2 provides the method of example 1, and the pairing of the client-side spans and the server-side spans belonging to the same calling in the client-side spans and the server-side spans filtered at the current moment comprises:
[0165] determine whether there is a server-side span with a parent span ID same as a span ID of a client-side span filtered at the current moment in the first queue, and a client-side span with a span ID same as a parent span ID of a server-side span filtered at the current moment;
[0166] if there is a server-side span with a parent span ID same as a span ID of a client-side span filtered at the current moment in the first queue, it is determined that the pairing of the client-side span filtered at the current moment and the server-side span with the parent span ID same as the span ID of the client-side span filtered at the current moment in the first queue is successful;
[0167] delete the paired server-side span from the first queue;
[0168] if there is no server-side span with a parent span ID same as a span ID of a client-side span filtered at the current moment in the first queue, the client-side span filtered at the current moment is stored into the first queue;
[0169] If there is a client Span in the first queue with a Span ID same as the parent Span ID of the server Span filtered at the current time, it is determined that the server Span filtered at the current time is successfully paired with the client Span in the first queue with a Span ID same as the parent Span ID of the server Span filtered at the current time.
[0170] The successfully paired client Span is deleted from the first queue.
[0171] If there is no client Span in the first queue with a Span ID same as the parent Span ID of the server Span filtered at the current time, the server Span filtered at the current time is stored to the first queue.
[0172] According to one or more embodiments of the present disclosure, example 3 provides the method of example 2, after the judgment of whether there is a server Span in the first queue with a parent Span ID same as the Span ID of the client Span filtered at the current time, and the client Span with a Span ID same as the parent Span ID of the server Span filtered at the current time, further comprising:
[0173] It is judged whether the Span at the head of the first queue is successfully paired within a preset time.
[0174] If the Span at the head of the first queue is not successfully paired within the preset time, the Span at the head is deleted from the first queue.
[0175] According to one or more embodiments of the present disclosure, example 4 provides the method of example 1, and the aggregation of the request information of the entry Span, the exit Span and the successfully paired Span filtered at the current time comprises:
[0176] The request information of the entry Span, the exit Span and the successfully paired Span filtered at the current time is incrementally aggregated with the request information of each span obtained within the time window at the current time.
[0177] According to one or more embodiments of the present disclosure, example 5 provides the method of example 1, further comprising:
[0178] The aggregated data is stored to a database.
[0179] According to one or more embodiments of the present disclosure, example 6 provides the method of any one of examples 1-5, further comprising:
[0180] In response to the operation of the user query, the aggregated data is topologically displayed.
[0181] According to one or more embodiments of the present disclosure, example 7 provides the method of example 6, and before the operation of topologically displaying the aggregated data in response to the user query, further comprising:
[0182] In response to the operation of the user setting the query condition, the query condition is determined, the query condition comprising at least one of a time range, an application name, and an application version number;
[0183] According to the query condition, it is determined whether the aggregated data needs to be aggregated again;
[0184] If the aggregated data needs to be aggregated again, the aggregated data is aggregated again according to the query condition, to obtain the aggregated data after the secondary aggregation, for topological display when the user queries.
[0185] According to one or more embodiments of the present disclosure, example 8 provides a calling data analysis system, comprising: a data filtering layer, a data matching layer, and a data aggregation layer.
[0186] The data filtering layer is configured to receive link data at a current time, identify four types of Spans in the link data, namely client-side Spans, server-side Spans, entry Spans, and exit Spans, push the identified client-side Spans and server-side Spans to the data matching layer, and push the identified entry Spans and exit Spans to the data aggregation layer, wherein the link data is calling data in a calling chain generated by mutual calling between business applications, and the calling chain is composed of multiple Spans.
[0187] The data matching layer is configured to receive the client-side Spans and server-side Spans pushed by the data filtering layer at the current time, pair the client-side Spans and server-side Spans belonging to the same call, and push the paired Spans to the data aggregation layer.
[0188] The data aggregation layer is configured to receive the entry Spans and exit Spans pushed by the data filtering layer at the current time and the paired Spans pushed by the data matching layer, and aggregate the request information of the entry Spans, exit Spans, and paired Spans to obtain aggregated data.
[0189] According to one or more embodiments of the present disclosure, example 9 provides the system of example 8, and the process of pairing the client-side Spans and server-side Spans belonging to the same call by the data matching layer receiving the client-side Spans and server-side Spans pushed by the data filtering layer at the current time, comprises:
[0190] receiving a client-side span and a server-side span pushed by the data filtering layer at the current time point;
[0191] determining whether there is a server-side span in the first queue whose parent span ID is the same as the span ID of the client-side span received by the data matching layer at the current time point, and a client-side span whose span ID is the same as the parent span ID of the server-side span received by the data matching layer at the current time point;
[0192] if there is a server-side span in the first queue whose parent span ID is the same as the span ID of the client-side span received by the data matching layer at the current time point, it is determined that the client-side span received by the data matching layer at the current time point is successfully paired with the server-side span in the first queue whose parent span ID is the same as the span ID of the client-side span received by the data matching layer at the current time point;
[0193] deleting the successfully paired server-side span from the first queue;
[0194] if there is no server-side span in the first queue whose parent span ID is the same as the span ID of the client-side span received by the data matching layer at the current time point, the client-side span received by the data matching layer at the current time point is stored in the first queue;
[0195] if there is a client-side span in the first queue whose span ID is the same as the parent span ID of the server-side span received by the data matching layer at the current time point, it is determined that the server-side span received by the data matching layer at the current time point is successfully paired with the client-side span in the first queue whose span ID is the same as the parent span ID of the server-side span received by the data matching layer at the current time point;
[0196] deleting the successfully paired client-side span from the first queue;
[0197] if there is no client-side span in the first queue whose span ID is the same as the parent span ID of the server-side span received by the data matching layer at the current time point, the server-side span received by the data matching layer at the current time point is stored in the first queue.
[0198] According to one or more embodiments of the present disclosure, example 10 provides the system of example 8, wherein the data aggregation layer receives the ingress Span and the egress Span pushed by the data filtering layer and the matched successful Span pushed by the data matching layer at a current time, and the process of aggregating the request information of the ingress Span and the egress Span and the matched successful Span comprises:
[0199] The data aggregation layer incrementally aggregates the request information of the ingress Span and the egress Span pushed by the data filtering layer and the matched successful Span pushed by the data matching layer received at a current time with the request information of each Span obtained by the data aggregation layer within a time window in which the current time is located.
[0200] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
[0201] While several inventive embodiments have been described above, it should be understood that many modifications, additions and substitutions can be made to the above described embodiments without departing from the spirit and scope of this disclosure. Accordingly, many modifications, additions and substitutions are intended to be included within the scope of the disclosure, and all such modifications, additions and substitutions are intended to be within the scope of the claims.
[0202] The above description is merely illustrative of the application and the application should not be limited thereto as modifications of all described variants and similar arrangements will become apparent to those skilled in the art upon making reference to the foregoing description. For a better understanding of the application, reference should be made to the Drawing and to the above Disclosure of the Invention in which preferred embodiments of the application are described along with the accompanying claims.
Claims
1. A method for invoking data analysis, characterized in that, include: Obtain the link data at the current moment, wherein the link data is the call data in the call chain generated by mutual calls between business applications, and the call chain is composed of multiple Spans; The link data is filtered to obtain four types of spans: client span, server span, ingress span, and egress span. Pair the client-side and server-side spans that belong to the same call from the currently filtered client-side and server-side spans, wherein the pairing includes: Determine whether there exists a server Span in the first queue whose parent Span ID is the same as the Span ID of the client Span selected at the current time, and a client Span whose Span ID is the same as the parent Span ID of the server Span selected at the current time; If there is a server Span in the first queue whose parent Span ID is the same as the Span ID of the client Span selected at the current time, then it is determined that the client Span selected at the current time and the server Span in the first queue whose parent Span ID is the same as the Span ID of the client Span selected at the current time are successfully paired. Remove the successfully paired server Span from the first queue; If there is no server-side Span in the first queue whose parent Span ID is the same as the Span ID of the client-side Span selected at the current moment, then the client-side Span selected at the current moment is stored in the first queue. If there is a client Span in the first queue whose Span ID is the same as the parent Span ID of the server Span selected at the current time, then it is determined that the server Span selected at the current time and the client Span in the first queue whose Span ID is the same as the parent Span ID of the server Span selected at the current time are successfully paired. Remove the successfully paired client Span from the first queue; If there is no client Span in the first queue whose Span ID is the same as the parent Span ID of the server Span selected at the current time, then the server Span selected at the current time is stored in the first queue. The request information of the ingress Span, egress Span, and successfully paired Span obtained at the current moment is aggregated to obtain aggregated data.
2. The method according to claim 1, characterized in that, After determining whether there exists a server Span in the first queue whose parent SpanID is the same as the Span ID of the client Span selected at the current time, and a client Span whose Span ID is the same as the parent Span ID of the server Span selected at the current time, the process further includes: Determine whether the Span at the head of the first queue has been successfully paired within a preset time. If the Span at the head of the first queue fails to pair successfully within a preset time, then the Span at the head of the first queue is deleted from the first queue.
3. The method according to claim 1, characterized in that, The aggregation of request information for the currently filtered inlet Span, outlet Span, and successfully paired Spans includes: The request information of the ingress Span, the exit Span, and the successfully paired Span obtained at the current moment is incrementally aggregated with the request information of each span already obtained within the current time window.
4. The method according to claim 1, characterized in that, Also includes: The aggregated data is stored in a database.
5. The method according to any one of claims 1-4, characterized in that, Also includes: In response to user queries, the aggregated data is displayed in a topological manner.
6. The method according to claim 5, characterized in that, Before displaying the topology of the aggregated data in response to a user query, the method further includes: In response to the user's operation of setting query conditions, determine the query conditions, which include at least one of time range, application name, and application version number; Determine whether the aggregated data needs secondary aggregation based on the query conditions; If the aggregated data requires secondary aggregation, then the aggregated data is aggregated again according to the query conditions to obtain the aggregated data after secondary aggregation, which is used for topology display when users query.
7. A method for accessing a data analysis system, characterized in that, include: Data filtering layer, data matching layer, data aggregation layer; The data filtering layer is used to receive the link data at the current moment, identify four types of spans in the link data: client span, server span, entry span, and exit span, push the identified client span and server span to the data matching layer, and push the identified entry span and exit span to the data aggregation layer. The link data is the call data in the call chain generated by mutual calls between business applications, and the call chain is composed of multiple spans. The data matching layer is used to receive the client Span and server Span pushed by the data filtering layer at the current moment, match the client Span and server Span belonging to the same call, and push the successfully matched Span to the data aggregation layer. The process by which the data matching layer pairs client-side and server-side spans belonging to the same call includes: Receive the client-side Span and server-side Span pushed by the data filtering layer at the current moment; Determine whether there exists a server Span in the first queue whose parent Span ID is the same as the Span ID of the client Span received by the data matching layer at the current time, and a client Span whose Span ID is the same as the parent Span ID of the server Span received by the data matching layer at the current time; If there is a server Span in the first queue whose parent Span ID is the same as the Span ID of the client Span received by the data matching layer at the current time, then it is determined that the client Span received by the data matching layer at the current time and the server Span in the first queue whose parent Span ID is the same as the Span ID of the client Span received by the data matching layer at the current time are successfully paired. Remove the successfully paired server Span from the first queue; If there is no server-side Span in the first queue whose parent Span ID is the same as the Span ID of the client Span received by the data matching layer at the current moment, then the client Span received by the data matching layer at the current moment is stored in the first queue. If there is a client Span in the first queue whose Span ID is the same as the parent Span ID of the server Span received by the data matching layer at the current time, then it is determined that the server Span received by the data matching layer at the current time and the client Span in the first queue whose Span ID is the same as the parent Span ID of the server Span received by the data matching layer at the current time are successfully paired. Remove the successfully paired client Span from the first queue; If there is no client Span in the first queue whose Span ID is the same as the parent Span ID of the server Span received by the data matching layer at the current moment, then the server Span received by the data matching layer at the current moment is stored in the first queue. The data aggregation layer is used to receive the ingress Span and egress Span pushed by the data filtering layer at the current time and the successfully paired Span pushed by the data matching layer, and aggregate the request information of the ingress Span, egress Span and the successfully paired Span to obtain aggregated data.
8. The system according to claim 7, characterized in that, The data aggregation layer receives the ingress Span and egress Span pushed by the data filtering layer at the current moment, and the successfully paired Span pushed by the data matching layer. The process of aggregating the request information of the ingress Span, egress Span, and the successfully paired Span includes: The data aggregation layer incrementally aggregates the request information of the inlet Span and outlet Span pushed by the data filtering layer and the successfully paired Span pushed by the data matching layer at the current moment with the request information of each Span already obtained by the data aggregation layer within the current time window.
Citation Information
Patent Citations
Call link data processing method and device
CN111464352A