Data processing method, computing device, storage medium and computer program product

Abstract

Provided in the embodiments of the present disclosure are a data processing method, a computing device, a storage medium and a computer program product. The data processing method comprises: acquiring a call chain data span generated by a service node; searching for a target attribute template matching the call chain data span, and determining difference data between the call chain data span and the target attribute template; and sending the difference data and identification information of the target attribute template to a server, wherein the target attribute template and the difference data are used for restoring the call chain data span. The technical solution provided by the embodiments of the present disclosure achieves lossless compression of a call chain data span.

Description

[0001] The present disclosure claims priority to Chinese patent application No. 2024108535821, filed on June 27, 2024 with the Chinese Patent Office, entitled "Data processing, computing device, computer storage medium and program product", the entire content of which is incorporated herein by reference. TECHNICAL FIELD Embodiments of the present disclosure relate to the technical field of data processing, and in particular to a data processing method, a computing device, a storage medium and a computer program product. BACKGROUND Distributed tracing system is a software tool used for monitoring and diagnosing request processing in a distributed system. In a microservice architecture, a request may pass through multiple service nodes, and the distributed tracing system can help developers understand the flow of the request in the system, so as to locate performance bottlenecks or fault points. In the distributed tracing system, each service node generates a span when processing a request, and the span can record the request processing operation of the node, such as an HTTP request, an RPC call, a database query, etc. After generating the span, the span is usually recorded and reported to the server to help developers and system administrators diagnose and optimize the performance and stability of the distributed system. The inventors have found in the process of implementing the present disclosure that each service call in a distributed system generates a span, and if the generated large number of spans are directly reported to the server, it will generate a large network and storage overhead. INVENTION The present disclosure provides a data processing method, a computing device, a storage medium and a computer program product. In a first aspect, a data processing method is provided in the present disclosure, which is applied to a probe, and the method comprises: collecting a span generated by a service node; finding a target attribute template matched with the span, and determining difference data between the span and the target attribute template; and sending the difference data and identification information of the target attribute template to a server, wherein the target attribute template and the difference data are used to restore the span.In a second aspect, a data processing method is provided in embodiments of the present disclosure, and is applied to a server. The method comprises: receiving difference data from a probe and identification information of an attribute template, wherein the difference data comprises difference data between a call chain data segment and a target attribute template, the call chain data segment is generated by a service node and obtained by the probe, and the attribute template matches the call chain data segment; searching for the target attribute template corresponding to the identification information from a plurality of attribute templates; and generating the call chain data segment based on the target attribute template and the difference data. In a third aspect, a computing device is provided in embodiments of the present disclosure, comprising a processing component and a storage component. The storage component stores one or more computer instructions. The one or more computer instructions are used to be invoked and executed by the processing component to implement the data processing method provided in embodiments of the present disclosure. In a fourth aspect, a computer readable storage medium is provided in embodiments of the present disclosure, and a computer program is stored on the computer readable storage medium. When the computer program is executed by a processing component, the data processing method provided in embodiments of the present disclosure is implemented. In a fifth aspect, a computer program product is provided in embodiments of the present disclosure, comprising computer programs / instructions. When the computer programs / instructions are executed by a processing component, the data processing method provided in embodiments of the present disclosure is implemented. In embodiments of the present disclosure, after the call chain data segment is collected on the service node side, the target attribute template matching the call chain data segment is searched for and the difference data between the call chain data segment and the target attribute template is determined, and then only the identification information of the target attribute template and the difference data are sent to the server, thereby reducing the data transmission amount between the service node and the server. After the identification information and the difference data are obtained by the server, the call chain data segment can be restored and generated, and lossless compression of the call chain data segment is implemented. These aspects and other aspects of the present disclosure will be more apparent in the following description of embodiments. BRIEF DESCRIPTION OF DRAWINGS In order to more clearly illustrate the technical solutions of embodiments of the present disclosure or the prior art, hereinafter, a brief introduction will be given to the drawings needed in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present disclosure, and for those skilled in the art, other drawings can also be obtained without creative labor. FIG. 1 shows a system architecture diagram to which a technical solution of an embodiment of the present disclosure can be applied. FIG. 2 schematically shows a flowchart of one embodiment of a data processing method provided by the present disclosure. FIG. 3 schematically shows a schematic diagram of a data processing method provided by embodiments of the present disclosure. FIG. 4 schematically shows a flowchart of another embodiment of a data processing method provided by the present disclosure.FIG. 5 schematically shows a schematic diagram of determining an attribute template according to an embodiment of the present disclosure. FIG. 6 schematically shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure. FIG. 7 schematically shows a block diagram of a data processing apparatus according to another embodiment of the present disclosure. In order for those skilled in the art to better understand the present disclosure, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. In some processes described in the specification, claims and the above drawings of the present disclosure, a plurality of operations appearing in a specific order are included, but it should be clearly understood that these operations can be executed or performed in parallel without the order appearing in the present disclosure, and the serial number of the operation is only used to distinguish different operations, and the serial number itself does not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be executed in sequence or in parallel. It should be noted that the "first", "second", etc. in the present disclosure are used to distinguish different messages, devices, modules, etc., and do not represent the order, nor do "first" and "second" represent different types. In order to facilitate understanding of the technical solutions of the present disclosure, the technical terms that can be involved in the present disclosure will be described below: A distributed tracing system is a system used to monitor and debug request flows in a distributed computing environment. It can track the path of a single request between multiple service nodes, collect and record the processing information of each service node. The system helps developers and operation and maintenance personnel locate performance bottlenecks, detect errors and optimize system performance by generating and analyzing request call chain data. A service node is a node that provides specific services or functions in a computer network or distributed system. The service node can be a server, a computing device or other devices in the network, responsible for performing specific service tasks or functions such as data processing, storage, communication, routing, etc. Service nodes work together in the entire system to ensure efficient operation of the system and reliability of services. An agent is a small program or code fragment used to collect data and monitor information in a distributed link tracing system. The agent is embedded in the code of the application program, responsible for capturing request flows, generating spans and sending data to the tracing system. The agent can be automatically or manually deployed on different service nodes to provide real-time system status and performance data.

[0002] A trace (call chain data) is a key concept in distributed tracing systems, referring to the complete path of a single request across various service nodes throughout its entire lifecycle. A trace consists of multiple spans, each representing an independent unit of work or service call within the trace. T-races help developers understand the flow of requests throughout the system.

[0003] A Span (call chain data fragment) is a basic unit in a Trace, representing a specific service call or job task. Each Span can contain attribute data and topology information. Below is an example of a call chain data fragment: Span ID: Identifies a unique Span instance.

[0004] Service Name: The service name, indicating the service to which this Span belongs.

[0005] Span Name: The name of the span, describing the operation or task performed by the span.

[0006] Duration: The duration of a Span, i.e., the execution time of the operation or task.

[0007] Status Code: The status code for the operation or task execution.

[0008] Attributes: May contain multiple key-value pairs describing other properties of the Span, such as request parameters, response data, thread information, etc.

[0009] Trace ID: Call identifier, which identifies the entire call chain to which this Span belongs.

[0010] Parent Span ID: The ID of the parent Span, indicating the parent Span of this Span.

[0011] Start Time: The start time of the Span.

[0012] Host Name / IP: The hostname or IP address of the Span.

[0013] Telemetry Information: Metadata related to tracing, such as SDK version, language, etc. Attributes can be attribute data, which can include multiple key-value pairs, where the key can refer to the attribute type of the attribute, and the value can be the attribute value corresponding to the attribute type. Attribute types can include, for example, request parameters, response data, environment information, service information, error information, performance indicators, log information, and tracing information.

[0014] Parent Span ID can be topology information, which can be used to indicate the calling relationship between this call chain data segment and other call chain data segments.

[0015] Spans are identified as being associated together by a call, forming a complete Trace o It should be noted that the embodiments of the present disclosure can involve the use of user data, and in actual application, user-specific personal data can be used in the schemes described herein within the scope permitted by applicable laws and regulations of the country where the user is located (for example, the user has given explicit consent, the user has been effectively notified, etc.). It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose authorization or refusal. It should be noted that the technical solutions of the embodiments of the present disclosure are applicable to a network virtual environment, and the users described are generally referred to as "virtual users", and real users can register user accounts in the service end through registration to obtain user identities in the network environment. In the embodiments of the present disclosure, the same user account can be logged into the service end through different types of clients, so that the service end can identify the same user, and of course different user accounts can be logged into the service end through different types of clients, and the service end stores the binding relationship of different user accounts, so that different user accounts with a binding relationship can be considered as the same user. The technical solutions in the embodiments of the present disclosure will be described clearly and completely in combination with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present disclosure. FIG. 1 shows a system architecture diagram to which a technical solution of an embodiment of the present disclosure can be applied, which can include a service node 101 and a service end 102o Among them, the service node 101 and the service end 102 are connected through a network. The network provides a medium for the communication link between the service node 101 and the service end 102. The network can include various connection types, such as wired, wireless communication links or optical cables, etc.Optionally, the server can be connected with the service node through a mobile network, and accordingly, the network standard of the mobile network can be any one of the following: second generation mobile communication technology 2G (GSM), second generation semi-mobile communication technology 2.5G (GPRS), third generation mobile communication technology 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), fourth generation mobile communication technology 4G (LTE), enhanced fourth generation mobile communication technology 4G+ (LTE+), fifth generation mobile communication technology 5G, and global microwave interconnection WiMax. Optionally, the user end can also establish a communication connection with the server through Bluetooth, WiFi (Wireless Fidelity), infrared, and the like. The service node 101 can interact with the server 102 through the network to receive or send messages and the like. The service node 101 can be an APP (Application), or a web application such as an H5 (HyperText Markup Language, version 5) application, or a light application (also known as a small program, a lightweight application), or a cloud application, and the like. The service node 101 can be deployed in an electronic device and needs to be dependent on the device to run or run through some app in the device. The electronic device can have a display screen and support information browsing, and the like, and can be a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, a desktop computer, a smart speaker, a smart watch, and the like. In order to facilitate understanding, the service node is mainly represented in the form of a device in FIG. 1. Various other applications can also be configured in the electronic device, such as man-machine dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, and the like. The electronic device can be a device used by a user and having functions of computing, networking, and communication required by the user, such as a mobile phone, a tablet computer, a personal computer, a wearable device, and the like. The electronic device can generally include at least one processing component and at least one storage component. The electronic device can also include a network card chip, a USB (Universal Serial Bus), an audio and video component, and the like, which are not limited in the present disclosure. Optionally, according to the implementation form of the electronic device, some peripheral devices can also be included, such as a keyboard, a mouse, an input pen, a printer, and the like, which are not limited in the present disclosure. The server 102 can include a server providing various services, such as a server for background training supporting a model used on the service node 101, or a server processing interactive information sent by the user end, and the like.It should be noted that the server 102 can be implemented as a distributed server cluster composed of multiple servers, or as a single server. The server can also be a server of a distributed system, or a server combined with a blockchain. The server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. It should be understood that the number of service nodes and servers in FIG. 1 is only illustrative. According to the implementation needs, there can be any number of user terminals and servers. The implementation details of the technical solutions of the embodiments of the present disclosure are described in detail below. FIG. 2 schematically shows a flowchart of one embodiment of a data processing method provided by the present disclosure, and the technical solution of the present embodiment can be executed by a probe deployed on a service node. As shown in FIG. 2, the data processing method can include the following steps.

[0016] 201: Collecting the span data generated by the service node. The service node can include a node in a computer network or distributed system that provides a specific service or function. These nodes can be servers, computing devices, or other devices in a network that are responsible for performing certain service tasks or functions, such as data processing, storage, communication, routing, etc. The span can be a basic unit of trace data, representing a specific service call or work task. The span can include real-time span data generated by the service node collected by the probe. In embodiments of the present disclosure, the service node can generate a span when it receives a request from a client or another service node, which can record the start time of the request, the request type, and related context information; the service node can also generate a span when it starts processing the request, which can be used to record various stages and steps in the request processing process, such as database queries, data processing, and other operations; the service node can also generate a span when it forwards the request to another node, such as an inter-service call in a microservice structure, which can be used to record detailed information of the forwarding operation; the service node can also generate a span when it completes the request processing and sends a response, to record the time and result of the response, which can include, for example, a success or failure status, response data, etc. After the service node generates the span, the probe can collect the span data generated by the service node.

[0017] 202: find a target attribute template matching the call chain data segment, and determine difference data of the call chain data segment and the target attribute template. In an embodiment of the present disclosure, a template library can be defined locally in the service node, and the template library can include a plurality of attribute templates. After the probe collects the call chain data segment, the call chain data segment can be compared with the plurality of attribute templates included in the template library, so as to find a target attribute template matching the call chain data segment. The call chain data segment can include a plurality of attribute data, which can be used to describe the request processing process of the call chain data segment. The attribute template can also include a plurality of template attributes, and the attribute template can be used to describe the processing process of a request through the plurality of template attributes. The target attribute template matching the call chain data segment can be, for example, an attribute template accurately describing the request processing process of the call chain data segment. After the target attribute template is found, the difference data of the call chain data segment and the target attribute template can be further determined, and the difference data can include attribute data in the plurality of attribute data included in the call chain data segment, which is different from the plurality of template attributes included in the target attribute template.

[0018] 203: send the difference data and the identification information of the target attribute template to the server, wherein the target attribute template and the difference data are used to restore the call chain data segment. In some embodiments, the identification information of the target attribute template can be used to uniquely determine the target attribute template. After the probe collects the call chain data segment and determines the difference data between the call chain data segment and the target attribute template, only the difference data and the target attribute template can be sent to the server, without sending the complete call chain data segment to the server. After the server receives the identification information of the target attribute template and the difference data, the server can obtain the corresponding target attribute template through the identification information. Since the attribute data contained in the target attribute template is the same as the attribute data contained in the call chain data segment, and the part that is different between the call chain data segment and the attribute data of the target attribute template is the difference data, the server can restore the call chain data segment collected by the probe by combining the target attribute template and the difference data. In some embodiments of the present disclosure, the server can obtain the target attribute template corresponding to the identification information in the template library defined locally on the server through the identification information, but is not limited thereto. The server can also obtain the target attribute template from the data storage system through the identification information. In some embodiments of the present disclosure, the call chain data segment is collected on the service node side, the target attribute template that matches the call chain data segment is found, the difference data between the call chain data segment and the target attribute template is determined, and then only the identification information of the target attribute template and the difference data are sent to the server, thereby reducing the data transmission amount between the service node and the server. After the server obtains the identification information and the difference data, the server can restore and generate the call chain data segment, thereby achieving lossless compression of the call chain data segment. In some embodiments, finding the attribute template that matches the call chain data segment can be implemented by extracting the plurality of attribute data of the call chain data segment, respectively determining the matching relationship between the plurality of attribute data of the call chain data segment and the template attribute of the plurality of attribute templates, and determining the target attribute template from the plurality of attribute templates based on the matching relationship. In some embodiments of the present disclosure, the data structure of the call chain data segment can be defined in advance, and the meaning and field name of each attribute of the call chain data segment can be defined. In the process of extracting the attribute data of the call chain data segment, the call chain data segment can be first parsed into a processable data structure, and then the plurality of attribute data can be extracted from the processable data structure according to the meaning and field name of each attribute of the call chain data segment.In some embodiments, the difference data between the call chain data segment and the target attribute template can be determined by: determining at least one target attribute data in the multiple attribute data of the call chain data segment that matches the template attribute of the attribute template; and determining attribute data other than the at least one target attribute data in the multiple attribute data as difference data. In some embodiments, the at least one target attribute data in the multiple attribute data of the call chain data segment that matches the template attribute of the attribute template can be determined as the target attribute data, and then the attribute data other than the target attribute data in the multiple attribute data can be determined as the difference data. The difference data can include attribute data that is in the call chain data segment but not in the target attribute template, but is not limited thereto. The difference data can also include different attribute values corresponding to the same attribute type in the call chain data segment and the target attribute template. In some embodiments of the present disclosure, the template attribute in the target attribute template that is not in the call chain data segment can also be recorded as difference data.In some embodiments, in addition to containing the same template attribute as the attribute data of the call chain data segment, the target attribute template can also contain a difference template attribute that is not in the call chain data segment. In this case, when the call chain data segment is restored based on the difference data and the target attribute template, the restored call chain data segment will have the difference template attribute. Therefore, in addition to determining the attribute data other than the target attribute data of the call chain data segment as the difference data, the difference template attribute can also be determined as the difference data. In some embodiments, the data processing method can further include: extracting topology information of the call chain data segment and a call identifier of the call chain data segment, wherein the topology information represents the calling relationship between the call chain data segment and other call chain data segments; finding a target topology template matching the topology information; and sending the target topology template and the call identifier to the server, so that the server generates a call chain of the call chain data segment based on the topology template. The topology information can represent the calling relationship between the call chain data segment and other call chain data segments, which can be obtained by a probe of the current service node, but is not limited thereto and can also be obtained by a probe of another service node. A single request can have multiple specific service calls during its entire life cycle, thereby generating multiple call chain data segments, which can have the same call identifier, such as the identification information of the call chain data. In an embodiment of the present disclosure, a template library can be defined locally in the service node, and the template library can contain multiple topology templates. After the probe collects the call chain data segment and extracts the topology information of the call chain data segment, a target topology template matching the topology information can be found from the multiple topology templates contained in the template library. The target topology template can include the calling relationship between the multiple call chain data segments generated during the entire life cycle of a single request, and the multiple calling relationships have the same call identifier. After the target topology template and the call identifier are sent to the server, the server can restore the call chain of the call chain data segment according to the target topology template and the call identifier. In some embodiments, finding a topology template matching the topology information can be specifically implemented as follows: based on the topology information and the call identifier, finding other call chain data segments having a calling relationship with the call chain data segment, the call chain data segment and the other call chain data segments having the same call identifier; based on the calling relationship, generating a call link including the call chain data segment and the other call chain data segments; and finding the target topology template based on the call link.In some embodiments, the determining the target topology template based on the call link can be specifically implemented as: determining a matching relationship of the call link with call paths indicated by the plurality of topology templates respectively; and determining the target topology template from the plurality of topology templates based on the matching relationship. The call link can refer to a set of a series of call data segments with a calling relationship, and describes a complete call path from a starting node to a terminal node. The plurality of topology templates included in the template library can respectively describe different call paths. After the call path described by the call link is determined, the topology template, of which the call path described by the topology template is the same as the call path described by the call link, can be determined as the target topology template from the plurality of topology templates in the template library. In the embodiments of the present disclosure, the plurality of topology templates included in the template library can be generated by the server and sent to the service node. The plurality of attribute templates included in the template library can also be generated by the server and sent to the service node. In some embodiments, the data processing method can further include: obtaining a sampling rule for the call chain data segment, the sampling rule being used to indicate a processing manner of the probe for the collected call chain data segment; in some embodiments, the sending the difference data and the identification information of the attribute template to the server includes: in a case where the call chain data segment hits the sampling rule, sending the difference data and the identification information of the attribute template to the server, otherwise, recording the identification information of the attribute template. The sampling rule can be a condition for determining which call chain data segment needs to be further processed, analyzed or sent to the server. The sampling rule can include, for example, a sampling rule based on a service name, that is, sampling the call chain data segment of a specific service node; or a sampling rule based on a corresponding time, that is, sampling the call chain data segment whose corresponding time exceeds a certain threshold; or a sampling rule based on an error or an exception, that is, sampling the call chain data segment with an error or an exception; or a random sampling rule, that is, sampling according to a certain random proportion. In the embodiments of the present disclosure, by sending only the difference data and the identification information of the attribute template of the call chain data segment that hits the sampling to the server, the amount of data that needs to be sent to the server can be further reduced on the premise of lossless compression. For the call chain data segment that does not hit the sampling, the identification information of the attribute template can be recorded and stored locally, so as to restore the call chain data segment to a certain extent based on the identification information when the call chain data segment needs to be analyzed. FIG. 3 schematically shows a schematic diagram of a data processing method provided by the embodiments of the present disclosure.As shown in FIG. 3, the service node 301 can include a separator 302, an attribute encoder 303, a topology builder 304, and a topology encoder 305, or a probe 306 can send the call chain data segment generated by the service node 301 to the separator 302, and the separator 302 can be used to extract attribute data and topology information from the call chain data segment. Among them, the attribute data can continue to be sent to the attribute encoder 303. The attribute encoder 303 can first determine whether the call chain data segment hits the sampling rule. For the call chain data segment that hits the sampling rule, the target attribute template matching the attribute data of the call chain data segment can be found, and the difference data between the attribute data and the target attribute template can be determined, and the identification information of the target attribute template and the difference data can be uploaded to the service end 307. The topology information can include the identification information of the call chain data segment, and the identification information and the call identification of the parent call chain data segment of the call chain data segment. The topology information can be sent to the topology builder 304, and the topology builder 304 can be used to construct a complete call chain according to the identification information of the call chain data segment and the identification information and the call identification of the parent call chain data segment of the call chain data segment contained in the topology information. The generated call chain can be sent to the topology encoder 305, and the topology encoder 305 can match the call chain to the target topology template according to the topology structure thereof, and upload the target topology template and the call identification to the service end 307. FIG. 4 schematically shows a flowchart of another embodiment of a data processing method provided by the present disclosure, and the technical scheme of the present embodiment can be executed by a service end. As shown in FIG. 4, the data processing method can include the following steps.

[0019] 401 : receiving difference data and identification information of an attribute template from a service node, wherein the difference data includes difference data between a call chain data segment and a target attribute template, the call chain data segment is generated by the service node and obtained by a probe, and the attribute template matches the call chain data segment;

[0020] 402 : finding a target attribute template corresponding to the identification information in a plurality of attribute templates;

[0021] 403 : Based on the target attribute template and the difference data, generate a span of the trace. The service node can include a node in a computer network or a distributed system that provides a specific service or function. These nodes can be servers, computing devices, or other devices in a network that are responsible for performing certain service tasks or functions, such as data processing, storage, communication, routing, etc. A span of the trace can be a basic unit in the trace of the call chain, representing a specific service call. The span of the call chain data generated in real time by the probe can be collected by the probe. In an embodiment of the present disclosure, the service node can generate a span of the call chain data when receiving a request from a client or another service node, which can record the start time of the request, the type of the request, and the relevant context information; the service node can also generate a span of the call chain data when starting to process the request, which can be used to record each stage and step in the request processing process, such as database query, data processing, etc.; the service node can also generate a span of the call chain data when forwarding the request to another node, such as inter-service call in a micro-service architecture, which can be used to record detailed information of the forwarding operation; the service node can also generate a span of the call chain data when the service node completes the request processing and sends a response, to record the time and result of the response, which can include, for example, the success or failure status, response data, etc. After the service node generates the span of the call chain data, the probe can collect the span of the call chain data generated by the service node. In an embodiment of the present disclosure, a template library can be predefined locally in the service node, and the template library can include a plurality of attribute templates. After the probe collects the span of the call chain data, the span of the call chain data and the plurality of attribute templates included in the template library can be compared to find a target attribute template matching the span of the call chain data. The span of the call chain data can include a plurality of attribute data, which can be used to describe the request processing process of the span of the call chain data. After the target attribute template is found, the difference data between the span of the call chain data and the target attribute template can be further determined, which can include attribute data in the plurality of attribute data included in the span of the call chain data that is different from the plurality of template attributes included in the target attribute template. The identification information of the target attribute template can be used to uniquely determine the target attribute template. After the probe collects the span of the call chain data and determines the difference data between the span of the call chain data and the target attribute template, only the difference data and the target attribute template can be sent to the server, without sending the complete span of the call chain data to the server.The service end can obtain the target attribute template corresponding to the identification information after receiving the identification information of the target attribute template and the difference data. Since the attribute data included in the target attribute template is the same as the attribute data included in the call chain data segment, and the different parts of the call chain data segment and the attribute data of the target attribute template are the difference data, the service end can restore the call chain data segment collected by the probe by combining the target attribute template and the difference data. In the embodiments of the present disclosure, the service end can obtain the target attribute template corresponding to the identification information in the template library defined in the service end in advance through the identification information, but is not limited thereto, and the service end can also obtain the target attribute template in the data storage system through the identification information. In the embodiments of the present disclosure, the call chain data segment is collected on the service node side, the target attribute template matched with the call chain data segment is found, and the difference data between the call chain data segment and the target attribute template is determined, and then only the identification information of the target attribute template and the difference data are sent to the service end, thereby reducing the data transmission amount between the service node and the service end. After the service end obtains the identification information and the difference data, the call chain data segment can be restored and generated, and lossless compression of the call chain data segment is realized. In some embodiments, the data processing method further includes: obtaining a plurality of historical call chain data segments; generating a plurality of attribute templates according to the plurality of historical call chain data segments; and sending the plurality of attribute templates to the service node, so that the service node finds the target attribute template matched with the call chain data segment from the plurality of attribute templates. The historical call chain data segment can include the call chain data segment recorded and generated by the system during the past running period. These historical call chain data segments can be used for analyzing and generating attribute templates. The plurality of historical call chain data segments generated during the past running period can be obtained from logs, databases or other data storage systems. The inventors have found that the call chain data segments with the same number of attribute types of attribute data have a higher probability of having the same attribute data, and therefore, in some embodiments of the present disclosure, generating attribute templates according to a plurality of historical call chain data segments can be specifically implemented as: extracting historical attribute data of the plurality of historical call chain data segments; respectively determining the number of types of attribute types involved in the plurality of historical call chain data segments; dividing the historical call chain data segments with the same number of types into a group based on the number of types to obtain a plurality of first historical call chain data segment groups; respectively determining a first historical call chain data segment from each first historical call chain data segment group; and respectively determining the historical attribute data of the plurality of first historical call chain data segments as attribute templates.In the plurality of first historical call chain data segment groups, the number of attribute types of the plurality of call chain data segments in each first historical call chain data segment group is the same. In an embodiment of the present disclosure, one first historical call chain data segment can be randomly determined from the plurality of call chain data segments of each first historical call chain data segment group, and the historical attribute data of the determined first historical call chain data segment is determined as an attribute template. After determining one attribute template from each of the plurality of first historical call chain data segment groups, the remaining historical call chain data segments can be further divided to determine more attribute templates. The plurality of historical call chain data segments included in the first historical call chain data segment only have the same number of attribute types, for example, all include five attribute types, but the attribute values of the five attribute types can be different, thereby the plurality of historical call chain data segments can be further divided. In some embodiments, the data processing method further comprises: for any first historical call chain data segment group, comparing the historical call chain data segments other than the first historical call chain data segment with the first historical call chain data segment to generate a difference list, wherein the difference list includes a plurality of second historical call chain data segments, and the difference historical attribute data of each second historical call chain data segment and the first historical call chain data segment; grouping the plurality of second historical call chain data segments in the difference list according to the number of different attribute types involved in the difference historical attribute data to obtain a plurality of second historical call chain data segment groups; determining one second historical call chain data segment from each second historical call chain data segment group, and determining the historical attribute data of the determined second historical call chain data segment as an attribute template. FIG. 5 schematically shows a schematic diagram for determining an attribute template provided by an embodiment of the present disclosure. In step 1 shown in FIG. 5, set 1 can include a plurality of historical call chain data segments, each historical call chain data segment can include a plurality of historical attribute data, and each historical attribute data can have a specific attribute type and attribute value. In step 2, the plurality of historical call chain data segments included in set 1 can be divided into different first historical call chain data segment groups according to the first number of attribute types, for example, set 2, set 3 and set 4 can be divided, wherein the historical call chain data segments in set 2 all include k attribute types, the historical call chain data segments in set 3 all include m attribute types, and the call chain data segments in set 4 all include n attribute types.For set 2, set 3 and set 4, a historical call chain data segment can be randomly determined from the multiple historical call chain data segments included in set 2, set 3 and set 4 respectively as a first historical call chain data segment, and the historical attribute data of the determined first historical call chain data segment is determined as the attribute template. Further, in each set, a second historical call chain data segment other than the first historical call chain data segment can be compared with the first historical call chain data segment in attribute value, and a difference attribute data list can be generated based on the multiple historical call chain data segments different from the first historical call chain data segment in attribute value. For example, in a set, the first historical call chain data segment has 5 attribute types, and the corresponding other historical call chain data segments also have 5 attribute types, wherein the historical call chain data segment 1 has 3 attribute types different from the attribute value of the first historical call chain data segment, and then for the historical call chain data segment 1, a second historical call chain data segment group 1 can be generated, which includes the 3 attribute types of the historical call chain data segment 1 different from the attribute value of the first historical call chain data segment. For each set, the multiple second historical call chain data segment groups can also be divided into different sets according to the number of attribute types, for example, set 2 can be further divided into set 5 and set 6; set 3 can be further divided into set 7 and set 8; set 4 can be further divided into set 9 and set 10. For set 5, set 6, set 7, set 8, set 9 and set 10, a second historical call chain data segment group can be randomly determined from the multiple second historical call chain data segment groups included in set 5, set 6, set 7, set 8, set 9 and set 10 respectively, and the historical attribute data of the determined second historical call chain data segment group is determined as the attribute template. As can be seen in step 5 in FIG. 5, the multiple sets are divided in a tree shape, wherein set 1 is the first depth, set 2, set 3 and set 4 are the second depth, and set 5, set 6, set 7, set 8, set 9 and set 10 are the third depth.The iteration stopping condition may be, for example, that the depth of the tree reaches a preset depth. If the depth of the tree reaches the preset depth, the BJT can stop further set partitioning. If the depth of the tree does not reach the preset depth, the set partitioning is continued according to the above process for each set including the maximum depth until the preset depth is reached. The preset depth may be flexibly set by a person skilled in the art according to application requirements. In some embodiments, the data processing method further includes: determining topology information of the call chain data segment, the topology information representing the calling relationship between the call chain data segment and other call chain data segments; and generating a call chain including the call chain data segment and other call chain data segments based on the topology information. In an embodiment of the present disclosure, the server can receive a large number of call chain data segments from a plurality of service nodes, and each call chain data segment can have its own topology information. After obtaining the call chain data segments, the server can determine the calling relationship between the call chain data segments based on the call chain data segments of each call chain data segment, thereby generating a complete call chain. In some embodiments, the data processing method further includes: extracting historical topology information of a plurality of historical call chain data segments; and generating a topology template based on the historical topology information, the topology template being used to generate a call chain of the call chain data segment. The historical topology information may refer to the calling relationship between historical call chain data segments recorded by the system during past operation. The topology template may be composed of a plurality of nodes and edges connecting the nodes, for example. Each node may represent a call chain data segment, and the edge may be a directed edge representing the calling relationship between two call chain data segments connected by the directed edge. In some embodiments, generating the topology template based on the historical topology information can be implemented as follows: determining the calling relationship between each historical call chain data segment based on the historical topology information; generating at least one call chain based on the calling relationship; and determining each call chain as a topology template. In some embodiments, the data processing method further includes: extracting topology information of the call chain data segment; finding a target topology template matching the topology information; and restoring the call chain of the call chain data segment based on the topology template. The target topology template including the topology information can be found from a plurality of topology templates included in a template library, and the target topology template is determined as the call chain of the call chain data segment.Figure 6 schematically shows a block diagram of a data processing apparatus provided by an embodiment of the present disclosure. As shown in Figure 6, the data processing apparatus can include: a collection module 601, configured to collect a call chain data segment generated by a service node; a first search module 602, configured to search for a target attribute template matched with the call chain data segment, and determine difference data of the call chain data segment and the target attribute template; and a compression module 603, configured to send the difference data and identification information of the target attribute template to a service end, so that the target attribute template and the difference data are used to restore the call chain data segment. In some embodiments, the first search module 602 is specifically configured to: extract a plurality of attribute data of the call chain data segment; respectively determine a matching relationship between the plurality of attribute data of the call chain data segment and template attributes of a plurality of attribute templates; and determine the target attribute template from the plurality of attribute templates based on the matching relationship. In some embodiments, the first search module 602 is specifically configured to: determine at least one target attribute data matched with the template attributes of the attribute template from the plurality of attribute data of the call chain data segment; and determine attribute data other than the at least one target attribute data from the plurality of attribute data as the difference data. In some embodiments, the data processing apparatus further includes: an extraction module, configured to extract topology information of the call chain data segment and a call identifier of the call chain data segment, wherein the topology information represents a calling relationship between the call chain data segment and other call chain data segments; a third search module, configured to search for a target topology template matched with the topology information; and a data sending module, configured to send the target topology template and the call identifier to the service end, so that the service end generates a call chain of the call chain data segment based on the topology template. In some embodiments, the third search module is specifically configured to: search for other call chain data segments having a calling relationship with the call chain data segment based on the topology information and the call identifier, the call chain data segment and the other call chain data segments having the same call identifier; generate a call link including the call chain data segment and the other call chain data segments based on the calling relationship; and search for the target topology template based on the call link. In some embodiments, the third search module is specifically configured to: respectively determine a matching relationship between the call link and calling paths indicated by a plurality of topology templates; and determine the target topology template from the plurality of topology templates based on the matching relationship.In some embodiments, the data processing apparatus further includes: a rule obtaining module configured to obtain a sampling rule for the call chain data segment, the sampling rule being used to indicate a processing manner of the probe for the collected call chain data segment; in some embodiments, the compression module 603 is specifically configured to: in a case where the call chain data segment hits the sampling rule, send the difference data and the identification information of the target attribute template to the server, otherwise, record the identification information of the target attribute template. The data processing apparatus shown in FIG. 6 can perform the data processing method described in the embodiment shown in FIG. 2, and the implementation principle and technical effects will not be described again. For the data processing apparatus in the above embodiments, the specific manner in which each module or unit performs an operation has been described in detail in the embodiments related to the method, and will not be described in detail here. FIG. 7 schematically shows a block diagram of a data processing apparatus provided by another embodiment of the present disclosure, as shown in FIG. 7, the data processing apparatus can include: a receiving module 701 configured to receive difference data and identification information of an attribute template from a service node, wherein the difference data includes difference data of a call chain data segment and a target attribute template, the call chain data segment is generated by the service node and collected by the probe, and the attribute template matches the call chain data segment; a second searching module 702 configured to search for a target attribute template corresponding to the identification information from a plurality of attribute templates; and a generating module 703 configured to generate a call chain data segment based on the target attribute template and the difference data. In some embodiments, the data processing apparatus can further include: a data obtaining module configured to obtain a plurality of historical call chain data segments; a first template generating module configured to generate a plurality of attribute templates according to the plurality of historical call chain data segments; and a first template sending module configured to send the plurality of attribute templates to the service node, so that the service node searches for a target attribute template matching the call chain data segment from the plurality of attribute templates. In some embodiments, the first template generating module is specifically configured to: extract historical attribute data of the plurality of historical call chain data segments; respectively determine a type number of attribute types involved in the plurality of historical call chain data segments; based on the type number, divide historical call chain data segments with the same type number into a group to obtain a plurality of first historical call chain data segment groups; respectively determine a first historical call chain data segment from each first historical call chain data segment group; and respectively determine the historical attribute data of the plurality of first historical call chain data segments as attribute templates.In some embodiments, the data processing apparatus can further include: a list generation module configured to, for any one first historical call chain data segment group, compare historical call chain data segments other than the first historical call chain data segment with the first historical call chain data segment to generate a difference list, wherein the difference list includes a plurality of second historical call chain data segments, and each second historical call chain data segment has difference historical attribute data from the first historical call chain data segment; a sub-rental module configured to group the plurality of second historical call chain data segments in the difference list according to a number of difference attribute types involved in the difference historical attribute data to obtain a plurality of second historical call chain data segment groups; and a template determination module configured to determine one second historical call chain data segment from each second historical call chain data segment group, and determine historical attribute data of the determined second historical call chain data segment as the attribute template. In some embodiments, the data processing apparatus can further include: a topology determination module configured to determine topology information of the call chain data segment, wherein the topology information represents a calling relationship between the call chain data segment and other call chain data segments; and a call chain generation module configured to generate a calling chain including the call chain data segment and the other call chain data segments based on the topology information. In some embodiments, the data processing apparatus can further include: a first topology extraction module configured to extract historical topology information of the plurality of historical call chain data segments; and a topology template generation module configured to generate a topology template based on the historical topology information, wherein the topology template is used to generate a calling chain of the call chain data segment. In some embodiments, the topology template generation module is specifically configured to: determine a calling relationship between each of the historical call chain data segments based on the historical topology information; generate at least one calling chain based on the calling relationship; and determine each calling chain as a topology template. In some embodiments, the data processing apparatus can further include: a second topology extraction module configured to extract topology information of the call chain data segment; a topology template searching module configured to search for a topology template matching the topology information; and a call chain restoration module configured to restore a calling chain of the call chain data segment based on the topology template. The data processing apparatus shown in FIG. 7 can perform the data processing method shown in the embodiment of FIG. 4, and the implementation principle and technical effects will not be repeated here. For the data processing apparatus in the above embodiments, the specific operation of each module and unit has been described in detail in the embodiments related to the method, and will not be described in detail here.The embodiments of the present disclosure further provide a computing device, which can include a storage component and a processing component; the storage component can include one or more computer instructions, wherein the one or more computer instructions are called by the processing component to be executed, so as to implement the data processing method provided by the embodiments of the present disclosure. Of course, the computing device can also include other components, such as an input / output interface, a display component, a communication component, etc. The input / output interface provides an interface between the processing component and a peripheral interface module, and the peripheral interface module can be an output device, an input device, etc. The communication component is configured to facilitate wired or wireless communication between the computing device and other devices, etc. The processing component can include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component can also be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components, for executing the above method. The storage component is configured to store various types of data to support the operation of the terminal. The storage component can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The display component can be an electroluminescence (EL) element, a liquid crystal display or a micro display with a similar structure, or a retina display or a similar laser scanning display. It should be noted that, in the case of the computing device implementing the data processing method shown in FIG. 4, it can be a physical device or an elastic computing host provided by a cloud computing platform. It can be realized as a distributed cluster composed of multiple servers or terminal devices, or as a single server or a single terminal device. In the case of the computing device implementing the data processing method shown in FIG. 2, it can be specifically realized as an electronic device, which can be a device used by a user and having functions of computing, surfing the Internet, communication and the like required by the user, such as a mobile phone, a tablet computer, a personal computer, a wearable device, etc. It should be noted that the computing device can be a physical device or an elastic computing host provided by a cloud computing platform.It can be implemented as a distributed cluster composed of multiple servers or terminal devices, or as a single server or single terminal device. The embodiments of the present disclosure also provide a computer-readable storage medium storing a computer program, and the computer program can implement the data processing method of the embodiment shown in FIG. 2 or implement the data processing method of the embodiment shown in FIG. 4 when executed by a computer. The computer-readable storage medium can be included in the computing device described in the above embodiments; or it can exist separately and not be assembled into the computing device. The embodiments of the present disclosure also provide a computer program product comprising a computer program loaded on a computer-readable storage medium, and the computer program can implement the data processing method of the embodiment shown in FIG. 2 or implement the data processing method of the embodiment shown in FIG. 4 when executed by a computer. In such embodiments, the computer program can be downloaded and installed from a network, and / or installed from a removable medium. When the computer program is executed by a processor, various functions defined in the system of the present disclosure are executed. It should be noted that the embodiments of the present disclosure can involve the use of user data, and in actual applications, user-specific personal data can be used in the schemes described herein in a manner that complies with applicable laws and regulations of the country (for example, with the explicit consent of the user, with the actual notification of the user, etc.), within the scope permitted by applicable laws and regulations. Those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be described here. The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. According to actual needs, some or all of the modules can be selected to achieve the purpose of the embodiments of the present disclosure. Those skilled in the art can understand and implement without creative labor. Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by software with a necessary general hardware platform, and of course, can also be implemented by hardware.Based on such an understanding, the technical solutions described above can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, or the like, and includes a number of instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments or some parts of the embodiments. It should be finally pointed out that: the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced equivalently; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

Claims 1. A data processing method, wherein, Applied to probes, the method includes: collecting call chain data fragments generated by service nodes; finding a target attribute template that matches the call chain data fragments, and determining the difference data between the call chain data fragments and the target attribute templates; sending the difference data and the identification information of the target attribute templates to the server, wherein the target attribute templates and the difference data are used to reconstruct the call chain data fragments.

2. The method according to claim 1, wherein, The step of finding the attribute template that matches the call chain data fragment includes: extracting multiple attribute data from the call chain data fragment; determining the matching relationship between the multiple attribute data of the call chain data fragment and the template attributes of multiple attribute templates; and determining the target attribute template from the multiple attribute templates based on the matching relationship.

3. The method according to claim 2, wherein, The step of determining the difference data between the call chain data fragment and the target attribute template includes: determining at least one target attribute data among multiple attribute data of the call chain data fragment that matches the template attribute of the attribute template; and determining the attribute data other than the at least one target attribute data among the multiple attribute data as the difference data.

4. The method according to claim 1 or 3, wherein, The method further includes: extracting topology information of the call chain data segment and a call identifier of the call chain data segment, wherein the topology information represents the call relationship between the call chain data segment and other call chain data segments; finding a target topology template that matches the topology information; and sending the target topology template and the call identifier to the server so that the server generates a call chain of the call chain data segment based on the topology template.

5. The method according to claim 4, wherein, The process of finding a topology template that matches the topology information includes: based on the topology information and the call identifier, finding other call chain data segments that have a call relationship with the call chain data segment, wherein the call chain data segment and the other call chain data segments have the same call identifier; based on the call relationship, generating a call chain that includes the call chain data segment and the other call chain data segments; and finding the target topology template based on the call chain.

6. The method according to claim 5, wherein, The step of finding the target topology template based on the call chain includes: determining the matching relationship between the call chain and the call paths indicated by multiple topology templates; Based on the matching relationship, the target topology template is determined from multiple topology templates.

7. The method according to claim 1 or 6, wherein, The method further includes: obtaining sampling rules for the call chain data segment, the sampling rules being used to instruct the probe on how to process the collected call chain data segment; sending the difference data and the identification information of the target attribute template to the server includes: if the call chain data segment matches the sampling rules, sending the difference data and the identification information of the target attribute template to the server; otherwise, recording the identification information of the target attribute template.

8. A data processing method, wherein, Applied to the server side, the method includes: receiving difference data from a probe and identification information of an attribute template, wherein the difference data includes difference data between a call chain data fragment and a target attribute template, the call chain data fragment being generated by a service node and collected by the probe, and the attribute template matching the call chain data fragment; searching for a target attribute template corresponding to the identification information among multiple attribute templates; and generating a call chain data fragment based on the target attribute template and the difference data.

9. The method according to claim 8, wherein, The method further includes: acquiring multiple historical call chain data fragments; generating multiple attribute templates based on the multiple historical call chain data fragments; and sending the multiple attribute templates to the service node.

10. The method according to claim 9, wherein, The step of generating multiple attribute templates based on the multiple historical call chain data fragments includes: extracting historical attribute data from the multiple historical call chain data fragments; determining the number of attribute types involved in each of the multiple historical call chain data fragments; grouping historical call chain data fragments with the same number of types into multiple first historical call chain data fragment groups based on the number of types to obtain multiple first historical call chain data fragment groups; determining one first historical call chain data fragment from each first historical call chain data fragment group; and determining the historical attribute data of the multiple first historical call chain data fragments as attribute templates.

11. The method according to claim 10, wherein, The method further includes: for any first historical call chain data segment group, comparing second historical call chain data segments other than the first historical call chain data segments with the first historical call chain data segments to generate a difference list, wherein the difference list includes multiple second historical call chain data segments, and historical attribute data of differences between each second historical call chain data segment and the first historical call chain data segment; grouping the multiple second historical call chain data segments in the difference list according to the number of difference attribute types involved in the historical attribute data to obtain multiple second historical call chain data segment groups; determining one second historical call chain data segment from each second historical call chain data segment group, and determining the historical attribute data of the determined second historical call chain data segment as the attribute template.

12. The method according to claim 8 or 11, wherein, The method further includes: Determine the topology information of the call chain data segment, wherein the topology information characterizes the call relationship between the call chain data segment and other call chain data segments; based on the topology information, generate a call chain including the call chain data segment and the other call chain data segments.

13. The method according to claim 9 or 11, wherein, The method further includes: extracting historical topology information of the plurality of historical call chain data fragments; generating a topology template based on the historical topology information, wherein the topology template is used to generate the call chain of the call chain data fragments.

14. The method according to claim 13, wherein, The step of generating a topology template based on the historical topology information includes: determining the call relationship between multiple historical call chain data fragments based on the historical topology information; generating at least one call chain based on the call relationship; and determining each call chain as a topology template.

15. The method according to claim 12 or 13, wherein, The method further includes: extracting topology information of the call chain data segment; finding a topology template that matches the topology information; and reconstructing the call chain of the call chain data segment based on the topology template.

16. A computing device, wherein, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement the data processing method as described in any one of claims 1 to 7, or to implement the data processing method as described in any one of claims 8 to 15.

17. A computer-readable storage medium, wherein, It stores a computer program thereon, which, when executed by a processing component, implements the data processing method as described in any one of claims 1 to 7, or implements the data processing method as described in any one of claims 8 to 15.

18. A computer program product, wherein, Includes a computer program / instruction, which, when executed by a processing component, implements the data processing method as described in any one of claims 1 to 7, or implements the data processing method as described in any one of claims 8 to 15.

Images