Data processing method and device, electronic equipment and computer readable storage medium
By generating and analyzing the information and logs of the call chain, the limitations of traditional technology application scenarios are solved, and a wider exception tracking and processing capabilities are achieved.
Patent Information
- Application Number
- CN202311592013.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-24
- Publication Date
- 2025-05-30
Smart Images
Figure CN120066936A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of call chain monitoring, and particularly to a data processing method, apparatus, electronic device, and computer-readable storage medium. Background Art
[0002] With the continuous development of software technology, the call chain technology, as a basic efficiency tool that can assist developers in software debugging and diagnosing software problems, is recognized by the industry and has also become an indispensable part of the infrastructure of the Internet industry. The call chain technology records the call information between various services in a complete call, connects the call information between various services into a chain structure, and can identify the information related to abnormal service calls, helping developers quickly locate the abnormal points, thereby improving the software optimization efficiency. However, most traditional call chain technologies can only perform abnormal tracking in specific environments, and there is a problem of limited application scenarios. Summary of the Invention
[0003] Embodiments of this application provide a data processing method, apparatus, electronic device, and computer-readable storage medium, which can be applicable to more application scenarios.
[0004] In a first aspect, this application provides a data processing method. The method includes:
[0005] Generating a call chain based on nodes corresponding to a target service;
[0006] When it is recognized that the call chain is an abnormal call chain, obtaining call chain information corresponding to the abnormal call chain;
[0007] Obtaining a call chain log corresponding to the abnormal call chain based on the call chain information;
[0008] Uploading the call chain information and the call chain log to a server to instruct the server to process the call chain information and the call chain log to obtain an abnormal analysis result.
[0009] In a second aspect, this application also provides a data processing apparatus. The apparatus includes:
[0010] A call chain generation module, configured to generate a call chain based on nodes corresponding to a target service;
[0011] An abnormal call chain module, configured to obtain call chain information corresponding to the abnormal call chain when it is recognized that the call chain is an abnormal call chain;
[0012] A call chain log module, configured to obtain a call chain log corresponding to the abnormal call chain based on the call chain information;
[0013] A data upload module, configured to upload the call chain information and the call chain log to a server, so as to instruct the server to process based on the call chain information and the call chain log to obtain an exception analysis result.
[0014] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0015] Generate a call chain based on nodes corresponding to a target service;
[0016] When it is recognized that the call chain is an abnormal call chain, obtain the call chain information corresponding to the abnormal call chain;
[0017] Obtain the call chain log corresponding to the abnormal call chain based on the call chain information;
[0018] Upload the call chain information and the call chain log to a server, so as to instruct the server to process based on the call chain information and the call chain log to obtain an exception analysis result.
[0019] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:
[0020] Generate a call chain based on nodes corresponding to a target service;
[0021] When it is recognized that the call chain is an abnormal call chain, obtain the call chain information corresponding to the abnormal call chain;
[0022] Obtain the call chain log corresponding to the abnormal call chain based on the call chain information;
[0023] Upload the call chain information and the call chain log to a server, so as to instruct the server to process based on the call chain information and the call chain log to obtain an exception analysis result.
[0024] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:
[0025] Generate a call chain based on nodes corresponding to a target service;
[0026] When it is recognized that the call chain is an abnormal call chain, obtain the call chain information corresponding to the abnormal call chain;
[0027] Obtain the call chain log corresponding to the abnormal call chain based on the call chain information;
[0028] Upload the call chain information and the call chain log to the server, so as to instruct the server to process based on the call chain information and the call chain log to obtain an exception analysis result.
[0029] The above data processing method, device, electronic device, computer-readable storage medium and computer program product generate a call chain based on nodes corresponding to a target service. When it is recognized that the call chain is an abnormal call chain, obtain the call chain information corresponding to the abnormal call chain, obtain the call chain log corresponding to the abnormal call chain based on the call chain information, and upload the call chain information and the call chain log to the server, so as to instruct the server to process based on the call chain information and the call chain log to obtain an exception analysis result. That is, the call chain information of the abnormal call chain and the corresponding call chain log can be uploaded to the server, and the server performs unified processing and analysis, which is not limited to the node type in the call chain or the network type corresponding to the node, so as to realize the acquisition and analysis of data corresponding to various types of nodes in the call chain and increase the usage scenarios of the call chain.
[0030] In a sixth aspect, the present application provides a data processing method. The method includes:
[0031] Obtain the call chain information and the call chain log uploaded by the terminal, and process the call chain information and the call chain log to obtain an exception analysis result; wherein,
[0032] The call chain information is the information of the abnormal call chain obtained by the node when generating a call chain based on the node corresponding to the target service and recognizing that the call chain is an abnormal call chain;
[0033] The call chain log is the log corresponding to the abnormal call chain obtained by the node based on the call chain information.
[0034] In a seventh aspect, the present application provides a data processing device. The device includes:
[0035] A data analysis module, configured to obtain the call chain information and the call chain log uploaded by the terminal, and process the call chain information and the call chain log to obtain an exception analysis result; wherein, the call chain information is the information of the abnormal call chain obtained by the node when generating a call chain based on the node corresponding to the target service and recognizing that the call chain is an abnormal call chain; the call chain log is the log corresponding to the abnormal call chain obtained by the node based on the call chain information.
[0036] In an eighth aspect, the present application provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0037] Obtain the call chain information and call chain logs uploaded by the terminal, and process the call chain information and the call chain logs to obtain an anomaly analysis result; wherein,
[0038] The call chain information is the information of the abnormal call chain obtained by the node when generating a call chain based on the nodes corresponding to the target service and identifying that the call chain is an abnormal call chain;
[0039] The call chain log is the log corresponding to the abnormal call chain obtained by the node based on the call chain information.
[0040] In a ninth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the following steps are implemented:
[0041] Obtain the call chain information and call chain logs uploaded by the terminal, and process the call chain information and the call chain logs to obtain an anomaly analysis result; wherein,
[0042] The call chain information is the information of the abnormal call chain obtained by the node when generating a call chain based on the nodes corresponding to the target service and identifying that the call chain is an abnormal call chain;
[0043] The call chain log is the log corresponding to the abnormal call chain obtained by the node based on the call chain information.
[0044] In a tenth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:
[0045] Obtain the call chain information and call chain logs uploaded by the terminal, and process the call chain information and the call chain logs to obtain an anomaly analysis result; wherein,
[0046] The call chain information is the information of the abnormal call chain obtained by the node when generating a call chain based on the nodes corresponding to the target service and identifying that the call chain is an abnormal call chain;
[0047] The call chain log is the log corresponding to the abnormal call chain obtained by the node based on the call chain information.
[0048] The above data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product obtain the call chain information and call chain logs uploaded by the terminal, and process the call chain information and call chain logs to obtain an exception analysis result. Among them, the call chain information is the information of the abnormal call chain obtained by the node when generating a call chain based on the node corresponding to the target service and identifying the call chain as an abnormal call chain. The call chain log is the log corresponding to the abnormal call chain obtained by the node based on the call chain information. It can realize the unified processing of the call chain information and call chain logs uploaded by the terminal, so it is not limited to the type of the terminal or the network type to which the terminal is connected. It can obtain the data of abnormal call chains in more scenarios, and at the same time improve the exception handling efficiency corresponding to the abnormal call chain. Brief Description of the Drawings
[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.
[0050] Figure 1 It is an application environment diagram of the data processing method in an embodiment;
[0051] Figure 2 It is a flowchart of the data processing method in an embodiment;
[0052] Figure 3 It is a schematic diagram of the abnormal log acquisition process in an embodiment;
[0053] Figure 4 It is a flowchart of step 202 in an embodiment;
[0054] Figure 5 It is a schematic diagram of the node identification group of each node in the call chain in an embodiment;
[0055] Figure 6 It is a schematic diagram of business calls in an embodiment;
[0056] Figure 7 It is a schematic diagram of the call chain information processing process in an embodiment;
[0057] Figure 8 It is a flowchart of step 206 in an embodiment;
[0058] Figure 9 It is a hierarchical architecture diagram of the server system in an embodiment;
[0059] Figure 10Schematic diagram of the call chain information integration process in an embodiment;
[0060] Figure 11 Call chain topology diagram in an embodiment;
[0061] Figure 12 Call chain flame graph in an embodiment;
[0062] Figure 13 Schematic diagram of the call chain log acquisition process in an embodiment;
[0063] Figure 14 Flowchart of the data processing method in another embodiment;
[0064] Figure 15 Structural block diagram of the data processing device in an embodiment;
[0065] Figure 16 Internal structure diagram of the computer device in an embodiment. Detailed implementation manners
[0066] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0067] The data processing method provided by the embodiments of the present application can be applied to the application environment as Figure 1 shown. Among them, the terminal 102 communicates with the server 104 through the network. The terminal 102 generates a call chain based on the nodes corresponding to the target service. When it is recognized that the call chain is an abnormal call chain, the call chain information corresponding to the abnormal call chain is obtained, the call chain log corresponding to the abnormal call chain is obtained based on the call chain information, and the call chain information and the call chain log are uploaded to the server 104. The server 104 receives the call chain information and the call chain log uploaded by the terminal 102, and processes the call chain information and the call chain log to obtain an abnormal analysis result. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The terminal 102 can be a terminal on a cloud architecture. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.
[0068] It should be noted here that the data processing method provided in the embodiments of the present application is applicable not only to the scenario of interaction between a terminal and a server, but also to the scenario of interaction between different servers. For example, if the call chain nodes in the call chain corresponding to the target service include terminal nodes and server nodes, then it involves the scenario of interaction between a terminal node and a server and the scenario of interaction between a server node and a server. The two servers interacting can be different servers. The terminal 102 can also be a device or a server.
[0069] In one embodiment, as Figure 2 shown, a data processing method is provided. Taking the terminal in Figure 1 as an example, the method includes the following steps 202 to 208.
[0070] Step 202, generate a call chain based on the nodes corresponding to the target service.
[0071] A service refers to the complete business call process corresponding to a request. A service can also be a process. The target service can be any service or process. Usually, to complete a service, multiple business calls are often required. During the business call process, calls or data transfer may be involved between multiple nodes. The nodes involved in the business call process may be terminals, devices, or functional modules, etc. It can be understood that the business call process may be realized between a terminal node and a server node, or between a terminal node and a terminal node, or between a device node and a device node, or between a service node and a service node.
[0072] In this embodiment, a call chain is generated according to the nodes corresponding to the target service, that is, a corresponding call chain is generated according to the nodes involved in the target service. One target service corresponds to at least one call chain, and the nodes involved in the target service are the call chain nodes. Optionally, the call chain includes a tracking identifier of the call chain, and the tracking identifier of the call chain can be generated according to the node information of the first node of the call chain, where the first node of the call chain is the starting node corresponding to the target service. Optionally, through the buried points on the nodes corresponding to the target service, the call chain can obtain the node status of each node corresponding to the target service, so as to monitor the running status of the target service.
[0073] In an optional embodiment, a call chain can be generated according to the node information of the target service. Any node in the call chain is used as the target node, and the node information of the target node can include a trace identifier, a target node identifier, an identifier of the previous node of the target node, a target node name, a call start timestamp of the target node, a call end timestamp of the target node, a node status of the target node, error information generated by the target node, and the like. Among them, the trace identifiers corresponding to all nodes in the same call chain are the same, the target node name can be named according to the function of the node, and the node status includes a normal state and an abnormal state.
[0074] Step 204, in the case where it is recognized that the call chain is an abnormal call chain, obtain the call chain information corresponding to the abnormal call chain.
[0075] In this embodiment, when the terminal recognizes that the call chain is an abnormal call chain, it obtains the call chain information corresponding to the abnormal call chain. For example, when abnormal reporting information is obtained through the buried points set in the nodes corresponding to the target service, it is recognized that the call chain is an abnormal call chain, and the call chain information corresponding to the abnormal call chain is obtained; or, when the end node of the target service is not the preset tail node, but other nodes except the tail node, it is recognized that the call chain is an abnormal call chain. In this case, it means that the business call of the target service is aborted and the complete business call process is not completed, resulting in a call abort. Or, when the obtained buried point data is preset abnormal data, it is recognized that the call chain is an abnormal call chain; or, when one or more nodes in the call chain call error buried points, the call chain is recognized as an abnormal call chain.
[0076] The call chain information can include a trace identifier, a node identifier, a start time of the target service, an end time of the target service, a node running state, and a system version corresponding to the node. Among them, the node identifier refers to the node identifiers corresponding to all nodes in the call chain.
[0077] Optionally, the call chain information corresponding to the abnormal call chain can be obtained by obtaining the buried point information corresponding to each node in the abnormal call chain and according to the buried point information corresponding to each node in the abnormal call chain.
[0078] Step 206, obtain the call chain log corresponding to the abnormal call chain based on the call chain information.
[0079] In this embodiment, the call chain log corresponding to the abnormal call chain can be obtained based on the call chain information, and the call chain log associated with the abnormal call chain can be obtained. Optionally, the initial log corresponding to each service thread of all services can be obtained, and the initial log corresponding to each service thread is associated with the call chain information of the abnormal call chain, so as to obtain the call chain log associated with the call chain information.
[0080] In one example, the abnormal log acquisition process is as follows Figure 3 shown. During the operation or call of the service's corresponding business process, corresponding logs are generated. The generated corresponding logs are printed and stored through logcat. Here, logcat is a command-line tool in the Android system and can be used to obtain the log information of a program. When the data processing module of the terminal identifies an abnormal log, the corresponding abnormal log is sent to the log storage file, and the logs associated with the abnormal log in the log storage file are packaged and uploaded to the distributed database in the cloud. Subsequently, the server can obtain the corresponding abnormal log from the distributed database.
[0081] Step 208: Upload the call chain information and call chain logs to the server to instruct the server to process based on the call chain information and call chain logs to obtain an abnormal analysis result.
[0082] The terminal uploads the obtained call chain information and call chain logs to the server to instruct the server to process based on the call chain information and call chain logs to obtain an abnormal analysis result. Optionally, the terminal can perform a persistence process on the obtained call chain information to obtain persistent call chain information and upload the persistent call chain information to the server. For example, create a local file to store the call chain information, cache the local file, and delete the local file after a preset time delay after the corresponding call chain tracking is completed. Optionally, the terminal can upload the obtained call chain logs to the cloud. The cloud sorts out its metadata information to obtain a log index corresponding to the call chain logs and synchronizes the log index to the server. The server can query and download the call chain logs in the cloud according to the call chain information.
[0083] The server can sort out based on the call chain information to obtain structured data, thereby generating visual call chain information. The visual call chain information can be, for example, a call chain list, call chain details, call chain topology diagram, call chain flame graph, log list, or alarm list, etc.
[0084] In the above data processing method, a call chain is generated based on the nodes corresponding to the target service. When it is recognized that the call chain is an abnormal call chain, the call chain information corresponding to the abnormal call chain is obtained. Based on the call chain information of the abnormal call chain, the call chain log corresponding to the abnormal call chain is obtained, and the call chain information and the call chain log are uploaded to the server to instruct the server to process based on the call chain information and the call chain log to obtain an abnormal analysis result. Different types of abnormal data corresponding to the nodes of the target service can be obtained, and then the server can uniformly analyze the call chain information and the call chain log of the abnormal call chain, regardless of the type of call chain nodes and the network type to which the nodes are connected. This can improve the positioning speed of call chain anomalies and the analysis speed of call chain logs, thereby improving the efficiency of anomaly handling.
[0085] In some embodiments, as Figure 4 shown, step 202 of generating a call chain based on the nodes corresponding to the target service includes the following steps 402 to 406.
[0086] Step 402: Use the starting node corresponding to the target service as the first node of the call chain, and generate a tracking identifier for the call chain according to the node information of the first node.
[0087] A service often includes multiple business call processes, which also include multiple nodes. Use the starting node corresponding to the target service as the first node of the call chain, and generate a tracking identifier for the call chain according to the node information of the first node. The node information of the first node of each call chain is unique, and the corresponding tracking identifier for each call chain is also unique. It can be understood that the business call process of the target service is set in advance, that is to say, the nodes corresponding to the target service are also preset, and the starting node and the ending node can also be determined by the operating system on the terminal. Therefore, the terminal can know in advance the starting node corresponding to the target service, use the starting node corresponding to the target service as the first node of the call chain, and generate a tracking identifier for the call chain according to the node information of the first node. The node information of the first node includes at least the address corresponding to the first node, the running timestamp, the process number corresponding to the target service, or may also include an incrementing serial number, etc. Among them, the address corresponding to the first node can be an IP (Internet Protocol) address, a MAC (Media Access Control) address, etc., and the running timestamp refers to the timestamp when the node starts running.
[0088] In one example, the trace identifier of the first node in the call chain includes the address corresponding to the first node, the running timestamp, the incrementing sequence number, and the process ID corresponding to the target service. For example, a 30-digit hexadecimal string is generated as 0ad1348f1403169275002100356696. Among them, the 0th to 7th digits are the IP address corresponding to the first node, such as 127.0.0.1; the 8th to 20th digits are the running timestamp; the 21st to 24th digits are the incrementing sequence number; the 25th to 29th digits are the process ID corresponding to the target service.
[0089] Step 404, generate a node identifier group corresponding to each node in the call chain according to the trace identifier, where the node identifier group includes the trace identifier, the current node identifier, and the previous node identifier.
[0090] In this embodiment, a node identifier group corresponding to each node in the call chain is generated according to the trace identifier. The node identifier group may include the trace identifier, the current node identifier, and the previous node identifier. It can be understood that for the first node of the call chain, the previous node identifier in the corresponding node identifier group is empty. Among them, the current node identifier may be, for example, the process identifier or service identifier corresponding to the current node. Different nodes have different corresponding node identifiers, that is, the node identifier of the current node is unique, and the previous node identifier is determined according to the node identifier of the previous node.
[0091] In one example, the node identifier groups of each node in the call chain are as Figure 5 shown. When the first node 502 of the call chain calls the corresponding buried point interface, a corresponding trace identifier is generated. Assume that the node identifier group corresponding to the first node 502 includes: the trace identifier is wert289dfi6rx7g1, the current node identifier is s_000, and the previous node identifier is empty, which is represented by -1 in this example; correspondingly, node 1 is the next node of the first node 502, that is, the previous node of node 1 is the first node 502. The node identifier group corresponding to node 1 includes: the trace identifier is wert289dfi6rx7g1, the current node identifier is s_001, and the previous node identifier is s_000; the node identifier group corresponding to node 2 includes: the trace identifier is wert289dfi6rx7g1, the current node identifier is s_002, and the previous node identifier is s_001; the node identifier group corresponding to the tail node 504 includes: the trace identifier is wert289dfi6rx7g1, the current node identifier is s_003, and the previous node identifier is s_002.
[0092] Step 406, generate a call chain according to the node identifier group.
[0093] Generate a corresponding call chain according to the node identifier group corresponding to each node. Optionally, the node identifier groups corresponding to each node can be constituted into corresponding call chain information.
[0094] In this embodiment, by using the starting node corresponding to the target service as the first node of the call chain, generating a tracking identifier of the call chain according to the node information of the first node, generating a node identifier group corresponding to each node in the call chain according to the tracking identifier, and generating the call chain according to the node identifier group, the tracking identifiers of all nodes in the generated call chain are the same. At the same time, each node can also be distinguished according to its node identifier, and a relationship is established between the previous node identifier and the current node identifier. Thus, the information of each node can be tracked and distinguished through the call chain, and it can be determined whether it belongs to the content of the same call chain through the tracking identifier, so as to facilitate the determination of cross-device or cross-terminal calls of the same call chain and achieve tracking between different terminals or devices.
[0095] In some embodiments, obtaining the call chain information corresponding to the abnormal call chain includes:
[0096] Obtaining the buried point information corresponding to each node of the abnormal call chain; using the buried point information corresponding to each node in the abnormal call chain as the call chain information.
[0097] The buried point information refers to the data obtained by the corresponding buried point. Each node is provided with a corresponding buried point, and node information or node operation status can be obtained through the corresponding buried point. The buried points corresponding to each node can be the same or different, and the specific position of the buried point can be set according to requirements. The buried point information corresponding to each node in the abnormal call chain can be used as the call chain information. For example, obtaining the buried point information of the corresponding node for the buried points set at the start execution position, end execution position, and error reporting position corresponding to each node. Exemplarily, the buried point information includes the node status, and the node statuses of all nodes in the call chain can be used as the call chain information. Among them, the node status includes a normal status and an abnormal status. If a node calls a wrong buried point or receives abnormal reporting information, the node is identified as an abnormal node, and the corresponding node status is the abnormal status.
[0098] In this embodiment, by obtaining the buried point information corresponding to each node of the abnormal call chain and using the buried point information corresponding to each node in the abnormal call chain as the call chain information, the integrity of the call chain information can be enhanced. At the same time, by setting the buried points, the buried point information corresponding to each node can be quickly obtained, thereby accelerating the speed of obtaining the call chain information and further improving the efficiency of locating abnormal points based on the call chain information.
[0099] In some embodiments, obtaining the buried point information corresponding to each node of the abnormal call chain includes:
[0100] Obtaining the information of the start buried point, end buried point, and error reporting buried point corresponding to each node of the target service in the abnormal call chain, and using the information of the start buried point, end buried point, and error reporting buried point corresponding to each node in the abnormal call chain as the node information.
[0101] In this embodiment, each node calls the start point marking, end point marking, and error point marking respectively at the start of operation, end of operation, and corresponding error positions. The information of the start point marking, end point marking, and error point marking corresponding to each node in the abnormal call chain of the target service is obtained as node information. Among them, the start point marking indicates the start of the current node of the target service, that is, the start point marking is called before the start of this node. The end point marking indicates the end of the current node of the target service, that is, the end point marking is called when the node receives the operation. The error point marking indicates that an error has occurred during the operation of the current node of the target service, that is, the error point marking is called when an error occurs during the operation of the node. Optionally, when one or more nodes in the call chain call the error point marking, the call chain is regarded as an abnormal call chain. It can be understood that when an error occurs or a scenario that meets the preset analysis conditions is recognized, the error point marking is called.
[0102] Optionally, when calling the point marking, a call timestamp is generated, and the call chain information further includes the call timestamp of the called point marking.
[0103] In one example, when there is a cross-process business call corresponding to the target service, there may be nested calls to the corresponding point markings. As Figure 6 shown, Node 1 is the head node of the call chain, Node 6 is the tail node of the call chain, and Node 3 needs to nest the start point marking corresponding to Node 3 in the start point marking called by Node 2. Figure 5 The scenarios shown include business calls between Terminal A and Terminal B. Case 1 belongs to business calls implemented within the same terminal, that is, the call between Node 1 and Node 2 in Terminal A. Case 2 belongs to the nested call between Node 3 and Node 2. Case 3 belongs to cross-terminal calls, that is, the business call between Terminal A and Terminal B.
[0104] In the above embodiment, by using the information of the start point marking, end point marking, and error reporting point marking corresponding to each node in the abnormal call chain of the target service as node information, the integrity of the node information can be ensured, and the start position, end position, and error position of each node can be accurately marked, and the point marking information at the corresponding position can be obtained, which can improve the accuracy of the call chain information.
[0105] In an optional embodiment, when the terminal recognizes that the call chain is an abnormal call chain, it obtains the call chain information corresponding to the abnormal call chain, processes the call chain information, and uploads it to the distributed database in the cloud. The process schematic diagram of call chain information processing is as Figure 7As shown in the figure, the business process collects the buried point data in the business code through the observation SDK (Software Development Kit). Device A performs processing such as persistence and cleaning on the buried point data generated by Device A through the corresponding data processing process, obtains the call chain information corresponding to Device A, and uploads the call chain information corresponding to Device A to the distributed database in the cloud. Correspondingly, Device B performs processing such as persistence and cleaning on the buried point data generated by Device B through the corresponding data processing process, obtains the call chain information corresponding to Device B, and uploads the call chain information corresponding to Device B to the distributed database in the cloud. In this embodiment, the processing processes of the buried point data for different devices can be the same or different, but the data processing process for the buried point data does not affect the normal business call process of the target service. Different devices upload the corresponding call chain information to the same distributed database in the cloud, so that the call chain information corresponding to different devices can be obtained through this distributed database, and the call chain information corresponding to different devices can be integrated to achieve the synchronization of the call chain information across devices.
[0106] In some embodiments, as Figure 8 shown, step 206 of obtaining the call chain log corresponding to the abnormal call chain based on the call chain information includes the following steps 802 to 806.
[0107] Step 802, obtain the initial log corresponding to each service thread.
[0108] The running process of the target service may include multiple service threads, and each service thread corresponds to a corresponding initial log. Obtain the initial log corresponding to each service thread. For example, the initial log corresponding to each service thread can be obtained from the operating system or the application background corresponding to the terminal.
[0109] Step 804, associate with the call chain information according to the log thread identifier of the initial log.
[0110] The log thread identifier of the initial log can be the log name, log number, etc. corresponding to the initial log. The log thread identifier of the initial log can uniquely identify the corresponding initial log, that is, the log thread identifiers corresponding to different initial logs are different.
[0111] The call chain information includes the thread identifier of the corresponding initial log, and the initial log can be associated with the corresponding call chain information through the thread identifier of the initial log. Optionally, identify whether the call chain information includes the thread identifier corresponding to the initial log, and associate the initial log corresponding to the thread identifier included in the call chain information with the corresponding call chain information.
[0112] Step 806, if an abnormal log is detected, determine the target call chain information associated with the log thread identifier of the abnormal log, and use the initial logs corresponding to all the log thread identifiers associated with the target call chain information as the call chain logs.
[0113] If the terminal detects an abnormal log, it determines the target call chain information associated with the log thread identifier of the abnormal log, and uses the initial logs corresponding to all the log thread identifiers associated with the target call chain information as the call chain logs. In one example, if the terminal detects an abnormal log, the log thread identifier of the abnormal log is A, and the target call chain information associated with the log thread identifier A is L. If all the log thread identifiers associated with the target call chain information L are A, B, C, and D, then the initial logs corresponding to the log thread identifiers A, B, C, and D are used as the call chain logs.
[0114] In this embodiment, by obtaining the initial logs corresponding to each service thread, associating them with the call chain information according to the log thread identifiers of the initial logs, if an abnormal log is detected, determining the target call chain information associated with the log thread identifier of the abnormal log, and using the initial logs corresponding to all the log thread identifiers associated with the target call chain information as the call chain logs, the call chain logs corresponding to the call chain information can be accurately determined according to the log thread identifiers, so as to accurately obtain the call chain logs corresponding to the abnormal call chain.
[0115] In one embodiment, a data processing method is provided. Taking the server in Figure 1 as an example for illustration, it includes:
[0116] Obtain the call chain information and call chain logs uploaded by the terminal, and process the call chain information and call chain logs to obtain an abnormal analysis result; wherein, the call chain information is the information of the abnormal call chain obtained by the node when generating a call chain based on the nodes corresponding to the target service and identifying that the call chain is an abnormal call chain; the call chain logs are the logs corresponding to the abnormal call chain obtained by the node based on the call chain information.
[0117] In this embodiment, the server obtains the call chain information and call chain logs uploaded by the terminal, and processes the call chain information and call chain logs respectively to obtain an abnormal analysis result. Optionally, the server can sort out the call chain information to obtain structured data, generate visual call chain information from the structured data. The visual call chain information can be, for example, a call chain list, call chain details, call chain topology diagram, call chain flame graph, log list, or alarm list, etc. The server can also generate a log index and display the log index, and provide a log download interface, so that users can query and download the call chain logs according to the call chain information.
[0118] In an optional embodiment, the hierarchical architecture diagram of the server system is as follows Figure 9 shown. The server obtains the call chain information and call chain logs uploaded by the terminal, and the server integrates and processes the call chain information and call chain logs based on the cloud base and displays them. Among them, the cloud base includes a container environment, a distributed file system, and MongDB (a distributed document storage database). The call chain service and log analysis service in the business layer are supported based on the cloud base. Among them, the call chain analysis service includes call point pulling, call point parsing, log query pulling, alarm management, etc., and the log analysis service may include log script analysis. Analysis based on the business layer can display a call chain list, call chain topology, log list, log analysis, and alarm list, etc. in the user layer.
[0119] In an optional embodiment, the call chain information integration process is as follows Figure 10 shown. The terminal synchronizes the obtained call chain information to the distributed database, and the server can obtain the call chain information in the distributed database at preset intervals. For example, the preset interval is 30 minutes. The server saves the obtained call chain information to the MongDB database corresponding to the server. The server provides a call chain service and can query the call chain information from the MongDB database according to preset query conditions. For example, the preset query conditions are trace identifier, device identifier, user identifier, start timestamp, end timestamp, etc. The server queries according to the preset query conditions, obtains the corresponding query results, and displays the query results in the form of a list. If the overall call chain information is queried, such as call chain details, the server can display a topology diagram and a flame graph on the front-end page. In this embodiment, by storing the call chain information obtained from the terminal in the MongDB database of the server, fast query of the call chain information can be realized, so as to quickly locate the anomalies in the call chain information, and the efficiency of anomaly handling can be improved.
[0120] In this embodiment, by separately processing the obtained call chain information and call chain logs uploaded by the terminal, an anomaly analysis result is obtained. According to the analysis result corresponding to the call chain information, anomalies in the target service can be quickly identified, the anomalies in the target service can be quickly located, and the logs related to the anomalies can be quickly obtained, thereby improving the efficiency of anomaly resolution.
[0121] In some embodiments, processing the call chain information and call chain logs to obtain an anomaly analysis result includes: structuring the call chain information to obtain call chain data in a tree structure; generating at least one of a topology diagram and a flame graph according to the call chain data in the tree structure.
[0122] Structuring refers to the process of organizing or sequencing data. The call chain information in this embodiment includes the node information and buried point information corresponding to each node in the call chain, and there is no specific order. The nodes in the call chain can be sorted according to the call order of the nodes, so as to realize the structuring of the call chain information. For example, sorting can be performed according to the node identifiers of the nodes.
[0123] It can be understood that if a node calls a wrong buried point or receives an abnormal reporting message, the node is identified as an abnormal node, and the corresponding node status is an abnormal status. The call chain information includes node status information, and the node status includes a normal status and an abnormal status. Therefore, based on the call chain data in a tree structure, a topology graph or a flame graph can be generated, and abnormal nodes can be quickly identified from the topology graph or the flame graph.
[0124] In one example, the call chain topology graph is as Figure 11 shown, which can represent the call chain structure involved in the target service and the node status of the nodes in the call chain. Among them, the solid dot represents that the node status of the corresponding node is an abnormal status, and the corresponding node belongs to an abnormal node; the hollow dot represents that the node status of the corresponding node is a normal status, and the corresponding node is a normal node. Figure 11 The call chain topology graph shown also includes node names. The node name can be, for example, the name of the application program where the abnormality occurs, such as Systemui (system interface), desktop, etc. For example, Application 1 Node 1102 is a normal node, and Application 4 Node 1104 is an abnormal node. Through the call chain topology graph, the position corresponding to the node and whether the node is abnormal can be quickly and accurately determined.
[0125] In one example, the call chain flame graph is as Figure 12 shown, which can represent the running duration and node status of each node in the call chain. Among them, the solid box represents that the node status of the corresponding node is an abnormal status, and the corresponding node is an abnormal node; the hollow box represents that the node status of the corresponding node is a normal status, and the corresponding node is a normal node. Here, it should be noted that the forms representing the node status are not limited to the solid dots, hollow dots, solid boxes, and hollow boxes in the embodiments of the present application, and can be set as needed, as long as the normal nodes and abnormal nodes can be distinguished. For example, UMS Node 1202 is an abnormal node, and the running duration is 134 ms (milliseconds); UMS Node 1204 is a normal node, and the running duration is 131 ms (milliseconds).
[0126] In this embodiment, by structuring the call chain information, call chain data in a tree structure is obtained; according to the call chain data in the tree structure, at least one of a topology graph and a flame graph is generated, which can visually display the data related to the call chain, so that abnormal nodes in the call chain can be quickly identified, the abnormal situation can be quickly located, and the efficiency of abnormal handling can be improved.
[0127] In some embodiments, when processing the call chain information and call chain logs to obtain an abnormal analysis result, it further includes: sorting out the call chain logs according to the metadata information of the call chain logs to obtain a log index corresponding to the call chain logs.
[0128] Metadata refers to the data that describes data and its environment. In this embodiment, the metadata information may include the call chain log generation timestamp, the log source node identifier, the system environment version when the log is generated, the node model when the log is generated, etc. Sorting out the call chain logs according to the metadata information of the call chain logs can obtain a log index corresponding to the call chain logs, and the corresponding call chain logs can be queried and downloaded according to the log index.
[0129] In an example, the call chain log acquisition process is as Figure 13 shown. When a node identifies an abnormal log, it uploads the abnormal log to the log storage system. The log storage system can sort out the logs to obtain a log index corresponding to the call chain logs, and synchronize the log index to the server. The server can query the abnormal logs in the log storage system through the call chain information, and can download and save the abnormal logs to the distributed file system. Through the mounting method of the distributed file system, all the servers in the server cluster can read the disk logs. The front-end page of the server can provide an interface for users to download logs, and users can download the abnormal logs stored in the distributed file system from this download interface.
[0130] In this embodiment, sorting out the call chain logs according to the metadata information of the call chain logs to obtain a log index corresponding to the call chain logs can achieve the quick query and acquisition of the call chain logs, thereby improving the efficiency of log analysis and processing.
[0131] In one embodiment, as Figure 14 shown, a data processing method is provided, including the following steps 1402 to 1408.
[0132] Step 1402, the terminal generates a call chain based on the node corresponding to the target service.
[0133] Step 1404, when the terminal identifies that the call chain is an abnormal call chain, it obtains the call chain information corresponding to the abnormal call chain.
[0134] Step 1406: The terminal obtains the call chain log corresponding to the abnormal call chain based on the call chain information.
[0135] Step 1408: The terminal uploads the call chain information and the call chain log to the server.
[0136] Step 1410: The server receives the call chain information and the call chain log uploaded by the terminal.
[0137] Step 1412: The server processes the call chain information and the call chain log to obtain an abnormal analysis result.
[0138] In this embodiment, the terminal generates a call chain based on the nodes corresponding to the target service. When it is recognized that the call chain is an abnormal call chain, the terminal obtains the call chain information corresponding to the abnormal call chain, obtains the call chain log corresponding to the abnormal call chain based on the call chain information, uploads the call chain information and the call chain log to the server, the server receives the call chain information and the call chain log uploaded by the terminal, and processes the call chain information and the call chain log to obtain an abnormal analysis result. There is no limitation on the type of nodes and the network type accessed by the nodes, and it is possible to track, obtain, and analyze data corresponding to various types of nodes, which is applicable to more types of abnormal handling scenarios.
[0139] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are displayed in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.
[0140] Based on the same inventive concept, the embodiments of the present application also provide a data processing device for implementing the above-mentioned data processing method. The solution provided by this device for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more of the following data processing device embodiments can refer to the limitations on the data processing method in the above text, and will not be repeated here.
[0141] In one embodiment, as Figure 15As shown, a data processing device is provided, which is applied to a terminal and includes: a call chain generation module 1502, an abnormal call chain module 1504, a call chain log module 1506, and a data upload module 1508, where:
[0142] The call chain generation module 1502 is used to generate a call chain based on nodes corresponding to a target service;
[0143] The abnormal call chain module 1504 is used to obtain call chain information corresponding to the abnormal call chain when it is recognized that the call chain is an abnormal call chain;
[0144] The call chain log module 1506 is used to obtain a call chain log corresponding to the abnormal call chain based on the call chain information;
[0145] The data upload module 1508 is used to upload the call chain information and the call chain log to a server, so as to instruct the server to process based on the call chain information and the call chain log to obtain an abnormal analysis result.
[0146] In one embodiment, the call chain generation module 1502 is further used to: use the starting node corresponding to the target service as the head node of the call chain, generate a tracking identifier of the call chain according to the node information of the head node; generate a node identifier group corresponding to each node in the call chain according to the tracking identifier; the node identifier group includes the tracking identifier, the current node identifier, and the previous node identifier; generate the call chain according to the node identifier group.
[0147] In one embodiment, the abnormal call chain module 1504 is further used to: obtain the buried point information corresponding to each node of the abnormal call chain; use the buried point information corresponding to each node in the abnormal call chain as the call chain information.
[0148] In one embodiment, the abnormal call chain module 1504 is further used to: obtain the information of the start buried point, end buried point, and error reporting buried point corresponding to each node of the target service in the abnormal call chain, and use the information of the start buried point, end buried point, and error reporting buried point corresponding to each node in the abnormal call chain as the node information.
[0149] In one embodiment, the call chain log module 1506 is further used to: obtain an initial log corresponding to each service thread; associate with the call chain information according to the log thread identifier of the initial log; if an abnormal log is detected, determine the target call chain information associated with the log thread identifier of the abnormal log, and use the initial logs corresponding to all log thread identifiers associated with the target call chain information as the call chain log.
[0150] In one embodiment, a data processing device is provided, which is applied to a server and includes a data analysis module. The data analysis module is configured to: obtain call chain information and call chain logs uploaded by a terminal, and process the call chain information and the call chain logs to obtain an anomaly analysis result; wherein, the call chain information is information of an abnormal call chain obtained by the node when generating a call chain based on a node corresponding to a target service and identifying the call chain as an abnormal call chain; and the call chain logs are logs corresponding to the abnormal call chain obtained by the node based on the call chain information.
[0151] The data analysis module is further configured to: perform structured processing on the call chain information to obtain call chain data in a tree structure; and generate at least one of a topology graph and a flame graph based on the call chain data in the tree structure.
[0152] The data analysis module is further configured to: sort out the call chain logs according to metadata information of the call chain logs to obtain a log index corresponding to the call chain logs.
[0153] Each module in the above data processing device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in hardware form or be independent of the processor, or can be stored in the memory in the computer device in software form, so that the processor can call and execute operations corresponding to the above modules.
[0154] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 16 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store call chain information and call chain log data. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with an external terminal through a network connection. When the computer program is executed by the processor, it implements a data processing method.
[0155] Those skilled in the art can understand that Figure 16The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.
[0156] An embodiment of this application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, when the computer-executable instructions are executed by one or more processors, cause the processors to execute the steps of the data processing method.
[0157] An embodiment of this application also provides a computer program product containing instructions, which when run on a computer, causes the computer to execute the data processing method.
[0158] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.
[0159] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.
[0160] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.
[0161] The above-described embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.
Claims
1. A data processing method, characterized in that, it includes: generating a call chain based on nodes corresponding to a target service; when it is recognized that the call chain is an abnormal call chain, obtaining call chain information corresponding to the abnormal call chain; obtaining a call chain log corresponding to the abnormal call chain based on the call chain information; uploading the call chain information and the call chain log to a server to instruct the server to process based on the call chain information and the call chain log to obtain an abnormal analysis result.
2. The method according to claim 1, characterized in that, the generating a call chain based on nodes corresponding to a target service includes: using the starting node corresponding to the target service as the first node of the call chain, and generating a tracking identifier of the call chain according to the node information of the first node; generating a node identifier group corresponding to each node in the call chain according to the tracking identifier; the node identifier group includes the tracking identifier, the current node identifier, and the previous node identifier; generating the call chain according to the node identifier group.
3. The method according to claim 1, characterized in that, the obtaining call chain information corresponding to the abnormal call chain includes: obtaining buried point information corresponding to each node of the abnormal call chain; using the buried point information corresponding to each node in the abnormal call chain as the call chain information.
4. The method according to claim 3, characterized in that, the obtaining buried point information corresponding to each node of the abnormal call chain includes: obtaining information on the start buried point, end buried point, and error reporting buried point corresponding to each node of the target service in the abnormal call chain, and using the information on the start buried point, end buried point, and error reporting buried point corresponding to each node in the abnormal call chain as the node information.
5. The method according to claim 1, characterized in that, the obtaining a call chain log corresponding to the abnormal call chain based on the call chain information includes: obtaining an initial log corresponding to each service thread; associating the log thread identifier of the initial log with the call chain information; if an abnormal log is detected, determining target call chain information associated with the log thread identifier of the abnormal log, and using the initial logs corresponding to all log thread identifiers associated with the target call chain information as the call chain log.
6. A data processing method, characterized in that, the method includes: obtaining call chain information and a call chain log uploaded by a terminal, and processing the call chain information and the call chain log to obtain an abnormal analysis result; wherein, the call chain information is the information of the abnormal call chain obtained when the node generates a call chain based on nodes corresponding to a target service and recognizes that the call chain is an abnormal call chain; the call chain log is the log corresponding to the abnormal call chain obtained by the node based on the call chain information.
7. The method according to claim 6, characterized in that, the processing the call chain information and the call chain log to obtain an abnormal analysis result includes: Structurally process the call chain information to obtain call chain data in a tree structure; Generate at least one of a topology graph and a flame graph based on the call chain data in the tree structure.
8. The method according to claim 7, wherein, the processing of the call chain information and the call chain log to obtain an exception analysis result further includes: Sort out the call chain log according to the metadata information of the call chain log to obtain a log index corresponding to the call chain log.
9. A data processing device, wherein, it includes: a call chain generation module for generating a call chain based on nodes corresponding to a target service; an abnormal call chain module for obtaining call chain information corresponding to the abnormal call chain when it is recognized that the call chain is an abnormal call chain; a call chain log module for obtaining a call chain log corresponding to the abnormal call chain based on the call chain information; a data upload module for uploading the call chain information and the call chain log to a server to instruct the server to process based on the call chain information and the call chain log to obtain an exception analysis result.
10. An electronic device includes a memory and a processor, and a computer program is stored in the memory, wherein, when the computer program is executed by the processor, the processor is caused to execute the steps of the method according to any one of claims 1 to 8.
11. A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.