Data processing method and device for distributed link tracking, equipment and storage medium

Abstract

The embodiment of the invention relates to a data processing method and device for distributed link tracking, equipment and a storage medium, and the method comprises the steps: receiving a service request, processing the service request, caching the generated first tracking data locally, constructing a sub-request of the service request, and transmitting the sub-request to a next-hop service node, receiving second tracking data fed back by the next-hop service node through the response message, locally caching the second tracking data, and processing the response message to locally cache generated third tracking data, the first tracking data, the second tracking data and the third tracking data obtained from the local cache are subjected to tracking data splicing to generate the full-link tracking data generated by processing the service request, and the full-link tracking data are stored in the target storage device, so that the full-link tracking data generated by processing the service request are stored in the same node. And each node does not need to upload the tracking data to the log service, so that the dependence of each node on the log service is avoided.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to a data processing method, device, equipment and storage medium for distributed link tracking. Background Art

[0002] Data tracking refers to the process of collecting, monitoring and analyzing specific data through technical means. It is widely used in the field of data processing, especially in software anomaly location, which can help developers quickly locate the specific link where the anomaly occurs and improve the reliability of the software system. Since data tracking technology needs to obtain a large amount of tracking data, data transmission and reporting analysis takes up a lot of resources, which can seriously affect the timeliness of data tracking analysis. Therefore, how to track data efficiently and with low overhead is a technical problem that needs to be solved. Summary of the invention

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a data processing method, device, equipment and storage medium for distributed link tracking.

[0004] In a first aspect, an embodiment of the present disclosure provides a data processing method for distributed link tracing, the method comprising:

[0005] Receiving a service request, processing the service request and caching the generated first tracking data locally, constructing a sub-request of the service request and transmitting it to a next-hop service node;

[0006] receiving second tracing data generated by the next hop service node through processing a sub-request of the service request fed back through a response message, and caching the second tracing data locally;

[0007] Process the response message and cache the generated third tracing data locally, obtain the first tracing data, the second tracing data and the third tracing data from the local cache and concatenate the tracing data to generate the full-link tracing data generated by processing the business request, and store the full-link tracing data to the target storage device for analyzing the abnormality of the business request processing link.

[0008] In an optional implementation manner, the constructing of the sub-request of the service request further includes:

[0009] A sub-request of the service request is constructed, and a tracking identifier of the service request is encapsulated in the sub-request. The tracking identifier is transmitted along the service node processing path, and the tracking identifier is used to identify tracking data related to the service request.

[0010] In an optional implementation manner, before storing the full-link tracking data in the target storage device, the method further includes:

[0011] Based on the full-link tracking data, determining whether the processing of the service request meets a timeout condition;

[0012] When it is determined that the processing of the service request meets the timeout condition, the full-link tracking data is stored in the target storage device.

[0013] In an optional implementation, the next-hop service node is an intermediate node that processes the service request, and the second tracking data in the response message includes the tracking data generated by the next-hop service node when processing the sub-request of the service request and the tracking data generated by all other service nodes after the next-hop service node when processing other sub-requests of the service request.

[0014] In an optional implementation manner, the method further includes:

[0015] If no response message fed back by the next-hop service node is received within a preset time, an abnormal notification is reported to the target storage device to notify that other tracking data of the service request is lost.

[0016] In an optional implementation, the next-hop service node is a terminal service node that processes the service request, and the second tracking data in the response message includes tracking data generated by the terminal service node processing a sub-request of the service request.

[0017] In an optional implementation, the terminal service node is used to directly store the tracking data that has not been successfully fed back to the previous hop service node into the target storage device, for splicing to obtain the full-link tracking data generated by processing the business request.

[0018] In an optional implementation manner, before local caching, the method further includes:

[0019] The tracking data is encoded and compressed according to a first preset mapping table storing character strings and coding symbols having a corresponding relationship, and the encoded and compressed tracking data is then cached locally.

[0020] In an optional implementation manner, before local caching, the method further includes:

[0021] The tracking data is encoded and compressed according to a second preset mapping table storing character strings and encoding values ​​having a corresponding relationship, and the encoded and compressed tracking data is cached locally, wherein the encoding value is obtained after encoding processing based on the corresponding character string.

[0022] In a second aspect, the present disclosure provides a data processing device for distributed link tracing, the device comprising:

[0023] A first receiving module is used to receive a service request, process the service request and cache the generated first tracking data locally, process a sub-request of the service request and transmit it to a next-hop service node;

[0024] A second receiving module is used to receive second tracing data generated by the next hop service node through processing the sub-request of the service request through a response message feedback, and cache the second tracing data locally;

[0025] The first storage module is used to process the response message and cache the generated third tracking data locally, obtain the first tracking data, the second tracking data and the third tracking data from the local cache, and perform tracking data splicing to generate the full-link tracking data generated by processing the business request, and store the full-link tracking data in a target storage device for analyzing the abnormality of the business request processing link.

[0026] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement a data processing method for distributed link tracking as provided in an embodiment of the present disclosure.

[0027] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the data processing method for distributed link tracking as provided in an embodiment of the present disclosure.

[0028] In a fifth aspect, the present disclosure provides a computer program product, wherein the computer program product comprises a computer program / instructions, and the computer program / instructions implement the above method when executed by a processor.

[0029] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0030] In the data processing method for distributed link tracking provided by the embodiment of the present disclosure, first, a business request is received, the business request is processed and the generated first tracking data is cached locally, a sub-request of the business request is constructed and passed to a next-hop service node, second tracking data generated by the sub-request of the business request fed back by the next-hop service node through a response message is received, and the second tracking data is cached locally, the response message is processed and the generated third tracking data is cached locally, the first tracking data, the second tracking data and the third tracking data are obtained from the local cache and the tracking data are spliced ​​to generate full-link tracking data generated by processing the business request, and the full-link tracking data is stored in a target storage device for analyzing abnormalities in the business request processing link.

[0031] The disclosed embodiment achieves unified storage of the full-link tracking data generated by processing the business request on the same node by caching the first tracking data generated by processing the business request locally, caching the second tracking data generated by the sub-request of the processing business request fed back by the next-hop service node through the response message locally, and caching the third tracking data generated by processing the response message locally. This eliminates the need for each node to upload its own tracking data to the log service separately, effectively avoiding the system design where each node is strongly dependent on the log service. The data processing method for distributed link tracking provided by the disclosed embodiment can achieve data tracking of business requests, thereby facilitating rapid problem location. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0033] Figure 1 A schematic diagram of a data processing process for distributed link tracing provided by an embodiment of the present disclosure;

[0034] Figure 2 A schematic diagram of a flow chart of a data processing method for distributed link tracing provided by an embodiment of the present disclosure;

[0035] Figure 3 A schematic diagram of another process of data processing for distributed link tracing provided by an embodiment of the present disclosure;

[0036] Figure 4 A schematic diagram of the structure of a data processing device for distributed link tracking provided by an embodiment of the present disclosure;

[0037] Figure 5 A schematic diagram of the structure of a data processing device for distributed link tracking provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0039] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0040] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0041] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0042] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0043] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0044] Distributed link data tracking refers to the comprehensive monitoring and recording of every link of data flow from the beginning to the end of data processing in a distributed system to ensure the visibility and traceability of data when it is transmitted between multiple systems and services. It is extremely important for diagnosing system problems, optimizing data processes and ensuring data accuracy.

[0045] In one implementation, taking a distributed I / O system as an example, refer to Figure 1 , which is a schematic diagram of a process processing of distributed link tracking data provided by an embodiment of the present disclosure.

[0046] from Figure 1 It can be seen that when receiving business request a, service node A, service node B and service node C process business request a and its sub-requests, and upload the tracking data generated in the processing to the log service. That is, each node must upload tracking data to the log service separately, resulting in each node being highly dependent on the log service. How to effectively avoid the system design of each node being highly dependent on the log service and realize data tracking of business requests is a technical problem that needs to be solved at present.

[0047] To this end, an embodiment of the present disclosure provides a data processing method for distributed link tracking, specifically, receiving a business request, processing the business request and caching the generated first tracking data locally, building a sub-request of the business request and passing it to a next-hop service node, receiving the second tracking data generated by the sub-request of the business request fed back by the next-hop service node through a response message, and caching the second tracking data locally, processing the response message and caching the generated third tracking data locally, obtaining the first tracking data, the second tracking data, and the third tracking data from the local cache and generating tracking data splicing to generate full-link tracking data generated by executing the business request, and storing the full-link tracking data in a target storage device for analyzing abnormalities in the business request processing link.

[0048] The disclosed embodiment achieves unified storage of the full-link tracking data generated by processing the business request on the same node by caching the first tracking data generated by processing the business request locally, caching the second tracking data generated by the sub-request of the processing business request fed back by the next-hop service node through the response message locally, and caching the third tracking data generated by processing the response message locally. This eliminates the need for each node to upload its own tracking data to the log service separately, effectively avoiding the system design where each node is strongly dependent on the log service. The data processing method for distributed link tracking provided by the disclosed embodiment can achieve data tracking of business requests, thereby facilitating rapid problem location.

[0049] Based on this, an embodiment of the present disclosure provides a data processing method for distributed link tracking, which is introduced below in conjunction with a specific embodiment.

[0050] Figure 2 A flow chart of a data processing method for distributed link tracking provided in an embodiment of the present disclosure is provided. The method can be executed by a data processing device for distributed link tracking, wherein the device can be implemented using software and / or hardware and can generally be integrated in an electronic device.

[0051] like Figure 2 As shown, the method includes:

[0052] S201: Receive a service request, process the service request and cache the generated first tracking data locally, construct a sub-request of the service request and transmit it to a next-hop service node.

[0053] The data processing method for distributed link tracing provided by the embodiment of the present disclosure can be applied in a distributed system application with at least two service nodes, wherein the processing node can refer to a relatively independent processing module, an independent process, etc.

[0054] The service request in the embodiment of the present disclosure is a service request received by the starting service node. When receiving the service request, the service request is processed, and the first tracking data generated during the processing is cached locally, wherein the first tracking data may include data generated when the starting service node processes the service request, such as a timestamp.

[0055] In an optional implementation, a toolkit is deployed for the current application system, and when a service request is received, the toolkit is started to collect relevant data generated when processing the service request as the first tracking data.

[0056] In the embodiment of the present disclosure, based on the first tracing data generated by processing the business request, the first tracing data is cached locally.

[0057] In an optional implementation, a local cache area may be allocated to the current service node, and the local cache area is used to store the first tracking data generated by the current service node processing the service request. The local cache area has a corresponding relationship with the service node, that is, each service node is allocated a corresponding local cache area.

[0058] In the disclosed embodiment, the current starting service node processes the service request and locally caches the generated first tracking data, constructs a sub-request of the service request, and sends the sub-request to the next-hop service node, wherein the next-hop service node is the node that processes the sub-request of the service request.

[0059] The next hop service node may include an intermediate node in the service node processing path, and may also include a terminal service node, that is, an end service node. The next hop service node may be an intermediate node or a terminal service node, as described in detail below, and will not be discussed here.

[0060] In actual applications, since a service node can process multiple business requests, in order to identify the ownership of the tracking data, the tracking identifier needs to be encapsulated in a sub-request of the business request in the embodiment of the present disclosure.

[0061] Specifically, a sub-request of the service request is constructed, and a tracking identifier of the service request is encapsulated in the sub-request. The tracking identifier is transmitted along a service node processing path, and is used to identify tracking data related to the service request.

[0062] That is to say, the sub-request of the business request carries a tracking identifier, which is used to indicate which business request the tracking data generated by the sub-request on the next-hop service node belongs to, wherein the tracking data generated by processing the sub-request on the next-hop service node is partial tracking data of the business request to which it belongs on the tracking link.

[0063] S202: Receive second tracing data generated by the next hop service node through a response message when processing a sub-request of the service request, and cache the second tracing data locally.

[0064] In the embodiment of the present disclosure, when a response message returned by the next-hop service node is received, the second tracking data generated by the sub-request of the service request fed back through the response message can be received. That is, the second tracking data is encapsulated in the response message returned by the next-hop service node to the starting service node, wherein the second tracking data is the tracking data generated when the next-hop service node processes the sub-request of the service request. That is, the second tracking data generated by the next-hop service node processing the sub-request is fed back through the response message.

[0065] Based on the acquired second tracking data, the second tracking data is cached locally.

[0066] S203: Process the response message and cache the generated third tracing data locally, obtain the first tracing data, the second tracing data and the third tracing data from the local cache and concatenate the tracing data to generate the full-link tracing data generated by processing the business request, and store the full-link tracing data in the target storage device for analyzing the abnormality of the business request processing link.

[0067] The third tracking data in the embodiment of the present disclosure is the tracking data generated by the current starting service node processing related logical operations based on the data in the response message returned by the next-hop service node.

[0068] In the disclosed embodiment, the tracking data generated by the entire service node processing path processing the business request is collected at the starting processing node, the first tracking data, the second tracking data and the third tracking data are obtained from the local cache of the starting processing node, and all the above tracking data are spliced ​​to generate full-link tracking data for processing the business request. The full-link tracking data of the business request is the tracking data generated by processing the business request on the complete processing path. Specifically, the above tracking data can be spliced ​​based on the processing order of the business request on the service node processing path.

[0069] Based on the full-link tracking data generated by processing the processing request, the full-link tracking data is stored in the target storage device, and the full-link tracking data is used to locate the abnormal processing node of the business request. Among them, the target storage device is used to store the full-link tracking data, and the target storage device can be a device with storage function, such as a local disk, a log service platform, etc.

[0070] In an optional implementation, if the target storage device is a local disk, after storing the full-link tracking data generated by processing the business request in the local disk, the full-link tracking data in the local disk can also be uploaded to the log service for subsequent abnormality location. To facilitate abnormality location, if the full-link tracking data is encoded tracking data, the encoded tracking data can be decoded before uploading to the log service, and then the decoded tracking data can be uploaded to the log service.

[0071] In practical applications, in order to avoid storing full-link tracking data without abnormalities in the target storage device, the embodiment of the present disclosure can also filter the full-link tracking data generated by processing business requests.

[0072] In an optional implementation, a delay judgment is performed on the service request. Specifically, based on the full-link tracking data generated by processing the service request, it is determined whether the processing of the service request meets the timeout condition. The timeout condition is whether the full-link data processing time for processing the service request exceeds a preset time threshold, and the preset time threshold can be the minimum time value for a timeout to occur.

[0073] Specifically, based on the full-link tracking data generated by processing the business request, the full-link data processing time for processing the business request is obtained from the full-link tracking data, and it is determined whether the full-link data processing time meets the preset timeout condition; if it is determined that the full-link data processing time meets the timeout condition, the full-link tracking data for processing the business request is stored in the target storage device.

[0074] Among them, the full-link data processing time for processing the business request is obtained. Specifically, the start timestamp and end timestamp of processing the business request can be obtained from the full-link tracking data of executing the business request, and the full-link data processing time is obtained by subtracting the two.

[0075] If it is determined that the full-link tracking data for processing the business request does not meet the timeout condition, that is, the processing of the business request has not timed out, the full-link tracking data for processing the business request will be discarded.

[0076] The disclosed embodiment provides a data processing method for distributed link tracking, specifically, receiving a business request, processing the business request and caching the generated first tracking data locally, constructing a sub-request of the business request and passing it to a next-hop service node, receiving the second tracking data generated by the sub-request of the business request fed back by the next-hop service node through a response message, and caching the second tracking data locally, processing the response message and caching the generated third tracking data locally, obtaining the first tracking data, the second tracking data, and the third tracking data from the local cache and generating tracking data splicing to generate full-link tracking data generated by executing the business request, and storing the full-link tracking data in a target storage device for analyzing the abnormality of the business request processing link.

[0077] The disclosed embodiment achieves unified storage of the full-link tracking data generated by processing the business request on the same node by caching the first tracking data generated by processing the business request locally, caching the second tracking data generated by the sub-request of the processing business request fed back by the next-hop service node through the response message locally, and caching the third tracking data generated by processing the response message locally. This eliminates the need for each node to upload its own tracking data to the log service separately, effectively avoiding the system design where each node is strongly dependent on the log service. The data processing method for distributed link tracking provided by the disclosed embodiment can achieve data tracking of business requests, thereby facilitating rapid problem location.

[0078] In actual applications, network delays and other issues may cause abnormalities in the tracking data generated by processing business requests. In order to prompt that the tracking data of the business request is abnormal, the embodiment of the present disclosure can set a preset time to prompt that the tracking data generated by processing the business request is abnormal.

[0079] In an optional implementation, a preset time is set for the service node, and if no response message fed back by the next-hop service node is received within the preset time, an abnormal notification is reported to the target storage device, and the abnormal notification is used to notify that other tracking data of the service request is lost. The other tracking data may include the tracking data encapsulated in the current response message.

[0080] In an optional implementation, when a service request is received, the service request is processed and the generated first tracking data is cached locally, and a sub-request of the service request is constructed and transmitted to a next-hop service node.

[0081] The second tracking data generated by the next-hop service node through processing the sub-request of the service request fed back through a response message is received, and the second tracking data is cached locally.

[0082] Among them, the next-hop service node can be an intermediate node for processing the business request, and the second tracking data carried in the response message includes the tracking data generated by the next-hop service node processing the sub-request of the business request and the tracking data generated by other service nodes after the next-hop service node processing other sub-requests of the business request.

[0083] That is, the second tracing data carried in the response message includes tracing data generated by the current intermediate node processing the sub-request of the service request, and tracing data generated by other service nodes after the current intermediate node processing other sub-requests of the service request.

[0084] Then, based on the response message, the response message is processed and the generated third tracking data is cached locally, the first tracking data, the second tracking data and the third tracking data are obtained from the local cache, and the tracking data are spliced ​​to generate the full-link tracking data generated by processing the business request, and the full-link tracking data is stored in the target storage device for abnormal analysis of the business request processing link.

[0085] In another optional implementation, when a service request is received, the service request is processed and the generated first tracking data is cached locally, and a sub-request of the service request is constructed and transmitted to a next-hop service node.

[0086] The second tracking data generated by the next-hop service node through processing the sub-request of the service request fed back through a response message is received, and the second tracking data is cached locally.

[0087] Among them, the next-hop service node may be a terminal service node that processes the service request, and the terminal service node may specifically be a tail service node. The second tracking data in the response message includes tracking data generated by the terminal service node processing the sub-request of the service request.

[0088] That is, the second tracking data carried in the response message includes the tracking data generated when the current terminal service node processes the sub-request of the service request.

[0089] Then, based on the response message, the response message is processed and the generated third tracking data is cached locally, the first tracking data, the second tracking data and the third tracking data are obtained from the local cache and the tracking data are spliced ​​to generate the full-link tracking data generated by processing the business request, and the full-link tracking data is stored in the target storage device for abnormal analysis of the business request processing link.

[0090] On the basis of the contents of the above embodiments, in order to prevent the terminal service node from failing to successfully feed back the tracking data to the previous-hop service node, that is, the tracking data is lost, the embodiment of the present disclosure can also directly store the tracking data in the target storage device. Specifically, when it is detected that the second tracking data is not successfully encapsulated in the response message of the terminal service node, the second tracking data is directly stored in the target storage device, and then the second tracking data is obtained in the target storage device to splice and obtain the full-link tracking data generated by processing the business request.

[0091] In practical applications, in order to reduce the storage volume of full-link tracking data, the embodiments of the present disclosure may also perform data encoding on the tracking data of each node in the full-link data generated by processing business requests to achieve data compression.

[0092] In an optional implementation, the tracking data is encoded and compressed according to a first preset mapping table storing character strings and coding symbols having a corresponding relationship, and then the encoded and compressed tracking data is cached locally. The character string is a fixed character string in the tracking data.

[0093] In an application scenario, when a business request is received, the business request is executed, and the first tracking data generated by executing the business request is obtained. Based on a first preset mapping table, the character string in the first tracking data is mapped to a corresponding coding symbol to obtain the encoded first tracking data, and the encoded first tracking data is cached locally.

[0094] Similarly, the same encoding method can be used for the tracking data generated on other service nodes of the business request, and the character strings in the tracking data can be mapped to corresponding encoding symbols, and then cached locally.

[0095] In actual applications, there are non-fixed character strings, that is, character strings with limited changes, in the tracking data generated by executing business requests. The embodiment of the present disclosure can obtain the encoding value corresponding to this type of character string based on the second preset mapping table to compress the data volume.

[0096] In an optional implementation, the tracking data is encoded and compressed according to a second preset mapping table storing corresponding character strings and encoding values, and the encoded and compressed tracking data is cached locally. The encoding value is obtained after encoding processing based on the corresponding character string.

[0097] In an application scenario, based on a second preset mapping table, a coding value corresponding to a string in first tracking data generated by executing a business request is queried in the second preset mapping table, and the string in the first tracking data is replaced with the corresponding coding value to obtain the encoded first tracking data, and the encoded first tracking data is cached locally.

[0098] In another optional implementation, if the coding value corresponding to the string in the tracking data is not found in the second preset mapping table, the string can be converted into a corresponding coding value based on the coding method corresponding to the second preset mapping table to obtain the encoded tracking data, and the correspondence between the string and the coding value is stored in the second preset mapping table.

[0099] In order to facilitate understanding of the content of the above embodiments, the distributed I / O system is taken as an example to introduce the data processing method of distributed link tracking provided by the present disclosure. Figure 3 , which is a schematic diagram of the data processing process of a distributed link tracking provided in an embodiment of the present disclosure.

[0100] First, when service node A receives a processing instruction for business request a, service node A processes business request a, that is, executes relevant processing logic, and locally caches the tracking data TracePart1 generated by processing the business request a. The tracking data TracePart1 is part of the tracking data on the tracking link of business request a.

[0101] Then, a sub-request b of the business request a is constructed, and the tracking identifier corresponding to the business request a is encapsulated in the sub-request b, and the sub-request b is sent to the service node B. When receiving the sub-request b of the business request a, the service node B processes the sub-request b and locally caches the tracking data TracePart2 generated by processing the sub-request b. The tracking data TracePart2 is partial tracking data on the tracking link of the business request a.

[0102] Then, a sub-request c of the business request a is constructed, the tracing identifier corresponding to the business request a is encapsulated in the sub-request c of the business request a, and the sub-request c of the business request a is sent to the service node C. Upon receiving the sub-request c, the service node C processes the sub-request c and caches the tracing data TracePart3 generated by processing the sub-request c locally. When the service node C completes the relevant logic processing of the sub-request c, the tracing data TracePart3 is encapsulated in the response message of the sub-request c, and the response message of the sub-request c of the business request a is returned to the service node B.

[0103] When service node B receives the response message of sub-request c, it obtains the tracing data TracePart3 generated by processing sub-request c from the response message, and caches the tracing data TracePart3 locally. Based on the data processing logic in the response message, service node B caches the tracing data TracePart4 generated by processing the response message locally. Then, it obtains the tracing data TracePart2, TracePart3 and TracePart4 from the local cache of service node B, encapsulates the above tracing data TracePart2, TracePart3 and TracePart4 in the response message returned to service node A, and returns the response message of sub-request b of business request a to service node A.

[0104] When service node A receives a response message to sub-request b, it obtains tracing data TracePart2, TracePart3 and TracePart4 from the response message, and caches the tracing data TracePart2, TracePart3 and TracePart4 locally. Service node A processes the relevant logic based on the data in the response message, and caches the tracing data TracePart5 generated by processing the response message locally.

[0105] Service node A obtains the tracking data TracePart1, TracePart2, TracePart3, TracePart4, and TracePart5 generated by processing business request a from the local cache, and concatenates the tracking data to generate the full-link tracking data generated by processing the business request a. Based on the full-link tracking data, it determines whether the processing of the business request a meets the timeout condition. When the processing of the business request a meets the timeout condition, the full-link tracking data is stored in the target storage device, and the processing logic execution for the business request a is completed.

[0106] In order to implement the above embodiments, the present disclosure also proposes a data processing device for distributed link tracking. Figure 4 This is a schematic diagram of the structure of a data processing device for distributed link tracking provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated into an electronic device. Figure 4 As shown, the device comprises:

[0107] The first receiving module 401 is used to receive a service request, process the service request and cache the generated first tracking data locally, construct a sub-request of the service request and pass it to a next-hop service node;

[0108] The second receiving module 402 is used to receive the second tracking data generated by the next hop service node through the response message feedback when processing the sub-request of the service request, and cache the second tracking data locally;

[0109] The first storage module 403 is used to process the response message and cache the generated third tracing data locally, obtain the first tracing data, the second tracing data and the third tracing data from the local cache and perform tracing data splicing to generate the full-link tracing data generated by processing the business request, and store the full-link tracing data in the target storage device for analyzing the abnormality of the business request processing link.

[0110] In an optional implementation manner, the first receiving module includes a building module, the building module being specifically configured to:

[0111] A sub-request of the service request is constructed, and a tracking identifier of the service request is encapsulated in the sub-request. The tracking identifier is transmitted along the service node processing path, and the tracking identifier is used to identify tracking data related to the service request.

[0112] In an optional implementation, the device further includes:

[0113] A determination module, configured to determine whether the processing of the service request satisfies a timeout condition based on the full-link tracking data;

[0114] The second storage module is used to store the full-link tracking data to a target storage device when it is determined that the processing of the business request meets a timeout condition.

[0115] In an optional implementation, the next-hop service node is an intermediate node that processes the service request, and the second tracking data in the response message includes the tracking data generated by the next-hop service node when processing the sub-request of the service request and the tracking data generated by all other service nodes after the next-hop service node when processing other sub-requests of the service request.

[0116] In an optional implementation, the device further includes:

[0117] The reporting module is used to report an abnormal notification to the target storage device if no response message fed back by the next-hop service node is received within a preset time, so as to notify that other tracking data of the service request is lost.

[0118] In an optional implementation, the next-hop service node is a terminal service node that processes the service request, and the second tracking data in the response message includes tracking data generated by the terminal service node processing a sub-request of the service request.

[0119] In an optional implementation, the terminal service node is used to directly store the tracking data that has not been successfully fed back to the previous hop service node into the target storage device, for splicing to obtain the full-link tracking data generated by processing the business request.

[0120] In an optional implementation, the device further includes:

[0121] The first encoding module is used to encode and compress the tracking data according to a first preset mapping table storing character strings and encoding symbols having a corresponding relationship, and then cache the encoded and compressed tracking data locally.

[0122] In an optional implementation, the device further includes:

[0123] The second encoding module is used to encode and compress the tracking data according to a second preset mapping table storing character strings and encoding values ​​with corresponding relationships, and then cache the encoded and compressed tracking data locally, wherein the encoding value is obtained after encoding processing based on the corresponding character strings.

[0124] In the data processing device for distributed link tracking provided by the embodiment of the present disclosure, first, a business request is received, the business request is processed and the generated first tracking data is cached locally, a sub-request of the business request is constructed and passed to a next-hop service node, second tracking data generated by the sub-request of the business request fed back by the next-hop service node through a response message is received, and the second tracking data is cached locally, the response message is processed and the generated third tracking data is cached locally, the first tracking data, the second tracking data and the third tracking data are obtained from the local cache and the tracking data are spliced ​​to generate full-link tracking data generated by processing the business request, and the full-link tracking data is stored in a target storage device for analyzing abnormalities in the business request processing link.

[0125] The disclosed embodiment achieves unified storage of the full-link tracking data generated by processing the business request on the same node by caching the first tracking data generated by processing the business request locally, caching the second tracking data generated by the sub-request of the processing business request fed back by the next-hop service node through the response message locally, and caching the third tracking data generated by processing the response message locally. This eliminates the need for each node to upload its own tracking data to the log service separately, effectively avoiding the system design where each node is strongly dependent on the log service. The data processing method for distributed link tracking provided by the disclosed embodiment can achieve data tracking of business requests, thereby facilitating rapid problem location.

[0126] The data processing device for distributed link tracking provided in the embodiments of the present disclosure can execute the data processing method for distributed link tracking provided in any embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.

[0127] In order to implement the above embodiments, the present disclosure also proposes a computer program product, including a computer program / instruction, which implements the data processing method for distributed link tracking in the above embodiments when executed by a processor.

[0128] In addition to the above-mentioned methods and apparatus, the embodiments of the present disclosure also provide a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device implements the data processing method for distributed link tracking described in the embodiments of the present disclosure.

[0129] In addition, the embodiment of the present disclosure also provides a data processing device for distributed link tracking, see Figure 5 As shown, it may include:

[0130] Processor 501, memory 502, input device 503 and output device 504. The number of processors 501 in the data processing device of distributed link tracking can be one or more. Figure 5 In some embodiments of the present disclosure, the processor 501, the memory 502, the input device 503 and the output device 504 may be connected via a bus or other means, wherein: Figure 5 The example of connecting through bus is taken in the following.

[0131] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing of the data processing device of distributed link tracking by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The input device 503 may be used to receive input digital or character information, and to generate signal input related to user settings and function control of the data processing device of distributed link tracking.

[0132] Specifically in this embodiment, the processor 501 will load the executable files corresponding to the processes of one or more applications into the memory 502 according to the following instructions, and the processor 501 will run the applications stored in the memory 502, thereby realizing the various functions of the above-mentioned distributed link tracking data processing device.

[0133] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0134] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing method for distributed link tracing, characterized in that: include: Receiving a service request, processing the service request and caching the generated first tracking data locally, constructing a sub-request of the service request and transmitting it to a next-hop service node; receiving second tracing data generated by the next hop service node through processing a sub-request of the service request fed back by a response message, and caching the second tracing data locally; Process the response message and cache the generated third tracing data locally, obtain the first tracing data, the second tracing data and the third tracing data from the local cache and concatenate the tracing data to generate the full-link tracing data generated by processing the business request, and store the full-link tracing data to the target storage device for analyzing the abnormality of the business request processing link.

2. The method according to claim 1, characterized in that The sub-request for constructing the business request includes: A sub-request of the service request is constructed, and a tracking identifier of the service request is encapsulated in the sub-request. The tracking identifier is transmitted along the service node processing path, and the tracking identifier is used to identify tracking data related to the service request.

3. The method according to claim 1, characterized in that Before storing the full-link tracking data in the target storage device, the method further includes: Based on the full-link tracking data, determining whether the processing of the service request meets a timeout condition; When it is determined that the processing of the service request meets the timeout condition, the full-link tracking data is stored in the target storage device.

4. The method according to claim 1, characterized in that: If the next-hop service node is an intermediate node that processes the service request, the second tracking data in the response message includes the tracking data generated by the next-hop service node when processing the sub-request of the service request and the tracking data generated by all other service nodes after the next-hop service node when processing other sub-requests of the service request.

5. The method according to claim 1, characterized in that The method further comprises: If no response message fed back by the next-hop service node is received within a preset time, an abnormal notification is reported to the target storage device to notify that other tracking data of the service request is lost.

6. The method according to claim 1, characterized in that If the next-hop service node is a terminal service node that processes the service request, the second tracking data in the response message includes tracking data generated by the terminal service node when processing a sub-request of the service request.

7. The method according to claim 6, characterized in that The terminal service node is used to directly store the tracking data that has not been successfully fed back to the previous hop service node into the target storage device, so as to obtain the full-link tracking data generated by processing the service request through splicing.

8. The method according to claim 1, characterized in that Before local caching, the method further includes: The tracking data is encoded and compressed according to a first preset mapping table storing character strings and coding symbols having a corresponding relationship, and the encoded and compressed tracking data is then cached locally.

9. The method according to claim 1, characterized in that: Before local caching, the method further includes: The tracking data is encoded and compressed according to a second preset mapping table storing character strings and encoding values ​​having a corresponding relationship, and the encoded and compressed tracking data is cached locally, wherein the encoding value is obtained after encoding processing based on the corresponding character string.

10. A data processing device for distributed link tracking, characterized in that: The device comprises: A first receiving module is used to receive a service request, process the service request and cache the generated first tracking data locally, construct a sub-request of the service request and pass it to a next-hop service node; A second receiving module is used to receive second tracing data generated by the next hop service node through processing the sub-request of the service request through a response message feedback, and cache the second tracing data locally; The first storage module is used to process the response message and cache the generated third tracking data locally, obtain the first tracking data, the second tracking data and the third tracking data from the local cache, and perform tracking data splicing to generate the full-link tracking data generated by processing the business request, and store the full-link tracking data in a target storage device for analyzing the abnormality of the business request processing link.

11. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the data processing method for distributed link tracking described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the data processing method for distributed link tracking described in any one of claims 1-9.

13. A computer program product, characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images