Data tracking method, device, intelligent body and electronic device

Abstract

The present application discloses a data tracking method, device, intelligent body and electronic device, which belongs to the field of data tracking. The method includes: obtaining a span identifier of a first span; the first span is a span generated by a process of obtaining a second data packet by processing a first microservice to receive a first data packet; injecting the tracking context information including the span identifier of the first span and the tracking identifier of the target tracking to which the first span belongs into the serialized Protobuf data of the second data packet to obtain a third data packet; instructing the first microservice to send the third data packet to the second microservice, and realizing data packet-level tracking in terms of tracking granularity, which is conducive to quickly locating abnormalities or faults.

Description

Technical Field

[0001] The present application belongs to the field of data tracking, and in particular, relates to a data tracking method, device, intelligent body and electronic device. Background Art

[0002] The tracing process in OpenTelemetry (OTEL for short) refers to the collection of request (or transaction) execution information to help users understand the propagation path of requests throughout the distributed system.

[0003] The tracing process needs to transfer trace context information between different systems. In a distributed system, the trace context in OTEL is generally placed in the header of the http request and is only passed once along with the request header when the http connection is initialized. That is, the tracing process of the related technology is at the http request level, and the data packet is the basic unit of data in network transmission. The tracing process at the http request level cannot locate faults at the data packet level and cannot accurately locate faults. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a data tracking method, device, intelligent body and electronic device, which realizes data packet level tracking, and data packet level tracking is conducive to quickly locating anomalies or faults.

[0005] In a first aspect, the present application provides a data tracking method, the method comprising:

[0006] Obtain a span identifier of a first span; the first span is a span generated by a process in which the first microservice processes a received first data packet to obtain a second data packet;

[0007] Injecting the trace context information including the span identifier of the first span and the trace identifier of the target trace to which the first span belongs into the serialized Protobuf data of the second data packet to obtain a third data packet;

[0008] Instruct the first microservice to send a third data packet to the second microservice.

[0009] According to the data tracing method of the present application, by injecting the tracing context information including the span identifier of the first span and the tracing identifier of the target trace to which the first span belongs into the serialized Protobuf data of the second data packet, a third data packet is obtained, and the first microservice is instructed to send the third data packet to the second microservice. In terms of tracing granularity, tracing at the data packet level is achieved, and tracing at the data packet level is conducive to quickly locating anomalies or faults.

[0010] According to an embodiment of the present application, before obtaining the span identifier of the first span, the method further includes:

[0011] Get the tracking ID of the target tracking to which the first span belongs based on the following method:

[0012] In the case where the second data packet is obtained based on the first data packet and other data packets, selecting a target data packet from the first data packet and other data packets; using the tracking identifier included in the Protobuf data of the target data packet as the tracking identifier of the target tracking to which the first span belongs; or

[0013] Generate a new tracking identifier, and use the new tracking identifier as the tracking identifier of the target tracking to which the first span belongs;

[0014] When the second data packet is obtained based on the first data packet and not based on other data packets, the tracking identifier contained in the Protobuf data of the first data packet is used as the tracking identifier of the target tracking belonging to the first span; the tracking identifiers contained in the Protobuf data of the first data packet and the other data packets are different.

[0015] According to one embodiment of the present application, selecting a target data packet from a first data packet and other data packets includes:

[0016] sorting the data packet names of the first data packet and the other data packets;

[0017] The data packet with the target bit sequence in the sorting result is taken as the target data packet.

[0018] According to one embodiment of the present application, after selecting a target data packet from the first data packet and the other data packets, the method further includes:

[0019] When the target data packet is the first data packet, creating a span link between the tracking identifier included in the Protobuf data of the other data packets and the span identifier of the first span;

[0020] When the target data packet is not the first data packet, a span link is created between the tracking identifier included in the Protobuf data of the first data packet and the span identifier of the first span.

[0021] According to one embodiment of the present application, after generating a new tracking mark, the method further includes:

[0022] Create span links between the new tracking identifier and the tracking identifiers included in the Protobuf data of the first data packet and other data packets.

[0023] According to one embodiment of the present application, the serialized Protobuf data of the first data packet includes a span identifier of the second span generated by the third microservice; the first data packet is generated by the third microservice;

[0024] After obtaining the span identifier of the first span, the method further includes:

[0025] Creating a parent-child relationship between the span identifier of the first span and the span identifier of the second span; wherein the parent-child relationship indicates that the first span is a child span and the second span is a parent span;

[0026] Creating an inclusion relationship between the tracking identifier of the target tracking and the span identifier of the first span;

[0027] Based on the tracking identifier of the target tracking and the inclusion relationship between each span, as well as the parent-child relationship between each span, a tracking tree for target tracking is constructed.

[0028] According to an embodiment of the present application, a tracking tree for target tracking is constructed based on the tracking identifier of the target tracking and the inclusion relationship between the spans, and the parent-child relationship between the spans, including:

[0029] Based on the tracking identifier of the target tracking and the inclusion relationship between the spans, each span included in the target tracking is obtained;

[0030] Based on the parent-child relationship between each span, each span is linked to obtain a tracking tree for target tracking. In a second aspect, the present application provides a data tracking device, which includes:

[0031] A first processing module is used to obtain a span identifier of a first span; the first span is a span generated by a process of obtaining a second data packet by processing a first microservice to receive a first data packet;

[0032] A second processing module is used to inject the tracking context information including the span identifier of the first span and the tracking identifier of the target tracking to which the first span belongs into the serialized Protobuf data of the second data packet to obtain a third data packet;

[0033] The third processing module is used to instruct the first microservice to send a third data packet to the second microservice.

[0034] According to the data tracking device of the present application, by injecting the tracing context information including the span identifier of the first span and the tracing identifier of the target trace to which the first span belongs into the serialized Protobuf data of the second data packet, a third data packet is obtained, and the first microservice is instructed to send the third data packet to the second microservice. In terms of tracing granularity, tracing at the data packet level is achieved, and tracing at the data packet level is conducive to quickly locating anomalies or faults.

[0035] In a third aspect, the present application provides an intelligent agent, which is used to implement the data tracking method provided in the first aspect above.

[0036] In a fourth aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the data tracking method provided in the first aspect described above is implemented.

[0037] In a fifth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data tracking method provided in the first aspect above.

[0038] In a sixth aspect, the present application provides a chip, the chip comprising a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the data tracking method provided in the first aspect.

[0039] In a seventh aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the data tracking method provided in the first aspect above.

[0040] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0041] The tracing context information including the span identifier of the first span and the tracing identifier of the target trace to which the first span belongs is injected into the serialized Protobuf data of the second data packet to obtain a third data packet, thereby instructing the first microservice to send the third data packet to the second microservice. In terms of tracing granularity, tracing at the data packet level is implemented, and tracing at the data packet level is conducive to quickly locating anomalies or faults.

[0042] In addition, when the second data packet is obtained based on the first data packet and other data packets, a target data packet is selected from the first data packet and the other data packets; the tracing identifier contained in the Protobuf data of the target data packet is used as the tracing identifier of the target tracing belonging to the first span; or a new tracing identifier is generated, and the new tracing identifier is used as the tracing identifier of the target tracing belonging to the first span, and a span link is created between the tracing identifier contained in the Protobuf data of the first data packet or other data packets and the tracing identifier contained in the Protobuf data in the target data packet, so that the integrity of the trace is guaranteed in the multi-path aggregation scenario.

[0043] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0045] Figure 1 This is one of the flow charts of the data tracking method provided in the embodiment of the present application;

[0046] Figure 2 It is a schematic diagram of a DAG graph constructed for each microservice provided in an embodiment of the present application;

[0047] Figure 3 It is a schematic diagram of data tracking using the prior art provided in the embodiments of the present application;

[0048] Figure 4 is a connection diagram of various modules of the data tracking system provided in an embodiment of the present application;

[0049] Figure 5 is a schematic diagram of data tracking based on the data tracking method provided in an embodiment of the present application;

[0050] Figure 6 is a schematic diagram of the parent-child relationship between spans generated in each microservice provided in an embodiment of the present application;

[0051] Figure 7 It is a schematic diagram of the inclusion relationship between target tracking and each span, and the parent-child relationship between spans provided in an embodiment of the present application;

[0052] Figure 8 is a schematic diagram of a tracking tree provided in an embodiment of the present application;

[0053] Figure 9a is a schematic diagram of obtaining a tracking mark for target tracking based on method 1 provided in an embodiment of the present application;

[0054] Figure 9b is a schematic diagram of obtaining a tracking identifier for target tracking based on method 2 provided in an embodiment of the present application;

[0055] Fig.10 is a schematic diagram of the structure of a data tracking device provided in an embodiment of the present application;

[0056] Fig.11 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0058] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0059] In the open source world, distributed tracing solutions are usually based on the OpenTelemetry (OTEL) framework. OTEL is an open source framework for building distributed tracing, monitoring and log collection systems. It provides a set of standard APIs (Application Programming Interface), SDKs (Application Development Kits) and other tools to help developers collect, process and transmit observable data from distributed systems, such as traces, metrics and logs, and send them to different backend analysis systems for storage and processing.

[0060] However, the tracing of the OTEL framework is at the http request level or the grpc single request level. Even for the grpc stream tracing, it is only performed once when the stream is established.

[0061] This is because distributed tracing needs to transmit tracing context information tracecontext between different systems. Trace context is generally placed in the http header and transmitted along with the request. grpc is also built on http / 2. Due to the binary framing characteristics of http / 2, trace context will be transmitted along with the Headers Frame, and it is only transmitted once when the http link is initialized. It cannot be tracked afterwards. In TongOS business, if you want to track the status of each streampackage flow data packet, it is not possible under the existing OpenTelemetry framework.

[0062] See also Figure 3 , which shows a schematic diagram of data tracing using existing technologies. The client and server are connected via http / 2, and the trace context is passed once through the Headers Frame only during initialization. However, during the remote call process of the stream data packet Data Frame, the Data Frame does not pass the trace context, so data packet-level tracing cannot be performed.

[0063] The data tracking method provided in the embodiment of the present application can track each transmission of the grpc streaming data packet, can expand the OTEL SDK, support the injection of trace data at the grpc package level, and transmit it in microservices, and supports three languages ​​​​such as c++ / go / python.

[0064] The data tracking method, data tracking device, electronic device and readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0065] The data tracking method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.

[0066] The data tracking method provided in the embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the data tracking method. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, wearable devices, etc.

[0067] The data tracking method provided in the embodiment of the present application may be executed by a link tracking system or a link tracking platform with a data tracking function, such as an OTEL platform.

[0068] The data tracking method provided in the embodiment of the present application may be executed by an intelligent agent or a functional module or functional entity in the intelligent agent that can implement the data tracking method. The intelligent agent is, for example, TongAI, and the operating system where TongAI is located is the TongOS system.

[0069] A real-time link tracking system built for general intelligent agents. The link tracking system can track the real-time data flow, service dependency, time consumption, anomalies, etc. of each unit of the intelligent agent (TongOS, language, perception, cognition, planning, etc.).

[0070] The intelligent agent TongAI is multimodal and can receive visual and language signals from the environment, which then flow through a complex DAG network to achieve the capabilities of perceiving the environment, recognizing speech, inferring intent, action planning, and dialogue interaction. Such a large and complex intelligent agent system requires a corresponding tracking system that can track the data processing status of each module of the system in real time.

[0071] The reaction of the intelligent agent TongAI to the real world can essentially be seen as the continuous processing of real-world data streams. Therefore, it is necessary to track the flow and processing of data streams to facilitate the rapid location of system problems and provide support for system iteration and tuning.

[0072] like Figure 4 As shown, an embodiment of the present application provides a connection diagram of each module of a data tracking system, and the data tracking system includes a data generation module, a data export module, a data storage module and a data display / retrieval module.

[0073] Among them, the data generation module can generate and collect observable data, which includes logs, metrics and traces. The observable data can be collected through the SDK and API of the OTEL platform.

[0074] The data export module is used to export the above observable data, and the above observable data can be exported through the Exporter of the OTEL platform.

[0075] The data storage module is used to store the observable data exported by the data export module. In practical applications, the data storage module can be mature tempo, jagger, etc.

[0076] The data display / retrieval module is used to display observable data. In practical applications, the data display / retrieval module can use Grafana.

[0077] like Figure 1 As shown, the data tracking method includes: step 110, step 120 and step 130.

[0078] Step 110: Obtain a span identifier of a first span; the first span is a span generated by a process in which a first microservice processes a received first data packet to obtain a second data packet.

[0079] Distributed systems usually contain multiple microservices. Microservices is an architectural style that decomposes complex applications into a group of small, independent services. Each microservice implements specific business functions, performs specific operations, runs specific functions or codes, is an independent process, and has an independent life cycle. Microservices can be divided based on any standard or granularity, and there is no restriction on the division standard or granularity.

[0080] In a distributed system, when processing requests or data packets, a directed acyclic graph (DAG) can be constructed based on the relationship between microservices (such as call relationships). Specifically, microservices can be connected in series based on the relationship between them to obtain a DAG composed of microservices. The nodes of the DAG represent microservices, and the edges represent the relationship between microservices.

[0081] like Figure 2 As shown, an embodiment of the present application provides a schematic diagram of a DAG graph constructed for each microservice, as shown in the figure, including a user user, microservices A, B, C, D and E, wherein the user can directly interact with A, trigger a request requestX to A, A belongs to the front end Frontend, B and C belong to the middle tier Middle tier, D and E belong to the backend backend, A remote calls B to generate a remote call rpc1, A remote calls C to generate a remote call rpc2, C remote calls D to generate a remote call rpc3, C remote calls E to generate a remote call rpc4, after processing, A provides a response ReplyX to the user.

[0082] The intelligent agent TongAI can be deployed in this DAG to track data.

[0083] The embodiment of the present application is data tracking performed at the data packet level. A data packet is the basic unit for data transmission in a computer network. It is an encapsulation structure containing specific information and data payload (i.e., application layer data), which is usually transmitted in a standardized format in a network protocol.

[0084] continue Figure 2 The embodiment of the present application is a data packet level tracking. Therefore, each microservice processes a school bag. During streaming transmission, the data packet is a streaming data packet grpc stream.

[0085] Tracing refers to the process of recording and tracking requests and operations across different services and components. It is the life cycle of a complete object (such as a request, transaction, or data packet) from the start to the end in a distributed system. It helps developers and operation and maintenance personnel understand the operation of the system and identify performance bottlenecks, service failures, and other problems by capturing detailed information of each operation (such as start time, end time, status, dependencies, etc.).

[0086] A trace has a globally unique trace ID, which identifies a transaction, or represents the complete chain of a request from the time it is sent, flows through multiple subsystems, and finally receives a response. It is the data generated by the tracing process. All data packets or operations involved in the same request share the same traceID.

[0087] Span is the smallest unit in tracing, representing the life cycle of a single operation in the system. Use span to encapsulate or wrap the computational logic you want to focus on (a handler, an RPC, or several lines of business code), collect the time consumed by the business logic, and label and attribute the business logic.

[0088] Span has a globally unique span identifier spanID, which identifies a section of computing logic and is generated in each microservice.

[0089] Each span typically includes:

[0090] Operation name: The operation represented by this span (e.g., remote call, database query, etc.).

[0091] Start Time and End Time: The start and end time of the span, which helps calculate the duration of the operation.

[0092] Context information: records related information, such as request ID, user ID, geographic location, etc.

[0093] Status: Records whether the operation was successful, or additional error information if it failed.

[0094] A trace can usually include one or more spans. According to the parent-child relationship between timestamps and spans, the trace tree corresponding to the trace can be obtained. Each microservice generates a corresponding span when processing the corresponding data packet received. When tracing, spanID and traceID are used as meta information and passed to the downstream distributed system along with the request.

[0095] In the embodiment of the present application, the first microservice processes the first data packet, and the obtained second data packet generates a first span for the process of the first microservice processing the first data packet at the first microservice. The first span includes a corresponding span identifier, a start time (the time corresponding to the time when the first microservice starts processing the first data packet), an end time (the time corresponding to the time when the first microservice ends processing the first data packet), a status, etc.

[0096] Step 120: Inject the tracing context information including the span identifier of the first span and the tracing identifier of the target trace to which the first span belongs into the serialized Protobuf data of the second data packet to obtain a third data packet.

[0097] Trace context is an information structure used to track requests and operation flows across multiple services or components in a distributed system. Its purpose is to provide a way to track the flow of a request from the entry to the exit of each system component in a complex distributed system, and to record the processing of these requests in each system.

[0098] The trace context usually includes traceID and spanID. In the related art, the trace context is located in the header, which makes it impossible to trace the data packet.

[0099] In order to realize data packet tracing, the embodiment of the present application types the trace context into serialized Protobuf data (full name Protocol Buffers data, abbreviated as pb data).

[0100] Specifically, for the first microservice, the tracing context information including the span identifier spanID of the first span and the tracing identifier of the target trace to which the first span belongs can be injected into the second data packet to obtain a third data packet, thereby realizing the injection and propagation of the trace context.

[0101] Taking C++ language as an example, a Carrier can be implemented, and this Carrier can be used to inject tracecontext into TongOSEvent data (which belongs to Protobuf data).

[0102] Step 130: Instruct the first microservice to send a third data packet to the second microservice.

[0103] The second microservice in the embodiment of the present application is a microservice that processes the second data packet, such as a microservice that is remotely called by the first microservice. After obtaining the third data packet, the embodiment of the present application can instruct the first microservice to send the third data packet to the second microservice, thereby realizing the cross-microservice transmission of the trace context of the data packet.

[0104] After receiving the merged first data packet, the second microservice will parse the trace context from the third data packet. The trace context includes the spanID of the upstream microservice (the first microservice). The second microservice can build a parent-child relationship (also called a parent relationship) between the spanID of the upstream microservice in the third data packet and the spanID of the locally generated span, that is, the spanID of the upstream microservice is the parent span identifier ParentSpanID of the spanID of the local microservice. A tracing tree can be formed based on the parent-child relationship.

[0105] like Figure 5 As shown, an embodiment of the present application provides a schematic diagram of data tracing based on a data tracing method, including a client, microservice 1, microservice 2, etc. The client and microservice 1, as well as microservice 1 and microservice 2 are connected via http / 2. The Protobuf in the streaming data packet DataFrame sent by the client to microservice 1 has trace context1 injected into it, and the traceID in the trace context1 is trace1, and the spanID is spanA; the Protobuf in the streaming data packet DataFrame sent by microservice 1 to microservice 2 has trace context2 injected into it, and the traceID in the trace context2 is trace1, and the spanID is spanB.

[0106] Will Figure 3 and Figure 5 By comparison, we can find Figure 5 The illustrated embodiment implements tracing at the packet level.

[0107] The embodiment of the present application injects the tracing context information including the span identifier of the first span and the tracing identifier of the target trace to which the first span belongs into the serialized Protobuf data of the second data packet to obtain a third data packet, and instructs the first microservice to send the third data packet to the second microservice. In terms of tracing granularity, tracing at the data packet level is implemented, and tracing at the data packet level is conducive to quickly locating anomalies or faults.

[0108] In some embodiments, the serialized Protobuf data of the first data packet includes a span identifier of the second span generated by the third microservice; the first data packet is generated by the third microservice;

[0109] After obtaining the span identifier of the first span, the method further includes:

[0110] Creating a parent-child relationship between the span identifier of the first span and the span identifier of the second span; wherein the parent-child relationship indicates that the first span is a child span and the second span is a parent span;

[0111] Creating an inclusion relationship between the tracking identifier of the target tracking and the span identifier of the first span;

[0112] Based on the tracking identifier of the target tracking and the inclusion relationship between each span, as well as the parent-child relationship between each span, a tracking tree for target tracking is constructed.

[0113] In fact, the first data packet is sent by the upstream microservice of the first microservice, namely the third microservice. The serialized Protobuf data of the first data packet includes the span identifier of the second span generated by the third microservice. The first data packet is generated by the third microservice.

[0114] After receiving the first data packet, the first microservice in the embodiment of the present application will parse the span identifier of the second span generated by the third microservice from the Protobuf data of the first data packet, and the span identifier of the second span is the parent span of the first span generated locally by the first microservice. Therefore, after obtaining the span identifier of the first span, a parent-child relationship between the span identifier of the first span and the span identifier of the second span can be created; wherein the parent-child relationship characterizes that the first span is a child span and the second span is a parent span, that is, the span of the upstream microservice is the parent span of the span of the local microservice.

[0115] In addition, since a complete call will have a traceID throughout, and each span has its own trace, it is also necessary to create an inclusion relationship between the trace identifier traceID of the target trace to which the first span belongs and the span identifier spanID of the first span, thereby recording that the first span belongs to the target trace.

[0116] like Figure 6As shown, it exemplarily shows a schematic diagram of the parent-child relationship between spans generated in each microservice, the microservices include a first microservice, a second microservice, a third microservice, a fourth microservice and a fifth microservice, the spanID of the span generated in the first microservice is span1; span1 has no parent span, which is represented as "noParentSpanID" in the first microservice, the spanID of the span generated in the second microservice is span2; the parent span of span2 is span1, which is represented as "ParentSpanID: span1" in the second microservice; the span generated in the third microservice The spanID of the span generated in the fourth microservice is span3; the parent span of span3 is span1, which is represented as "ParentSpanID: span1" in the third microservice; the spanID of the span generated in the fourth microservice is span4; the parent span of span4 is span3, which is represented as "ParentSpanID: span3" in the fourth microservice; the spanID of the span generated in the fifth microservice is span5; the parent span of span5 is span3, which is represented as "ParentSpanID: span3" in the fifth microservice. The connection between microservices represents the direct relationship between the span and its parent span.

[0117] Span represents a processing logic that we care about, such as an RPC remote call, a function execution process, etc.

[0118] In this distributed calling process, the calling relationship is connected in series through span, and a complete call will have a traceID throughout, which is the core idea of ​​distributed tracing.

[0119] For each target tracking, the spans included in the target tracking can be determined based on the tracking identifier of the target tracking and the inclusion relationship between the spans. Based on the parent-child relationship between the spans, the spans included in the target tracking are linked, thereby constructing a tracking tree for the target tracking.

[0120] In some embodiments, a tracking tree for target tracking is constructed based on the tracking identifier of the target tracking and the inclusion relationship between the spans, and the parent-child relationship between the spans, including:

[0121] Based on the tracking identifier of the target tracking and the inclusion relationship between the spans, each span included in the target tracking is obtained;

[0122] Based on the parent-child relationship between each span, each span is linked to obtain the tracking tree for target tracking.

[0123] Specifically, Figure 7As shown, an embodiment of the present application provides a schematic diagram of the inclusion relationship between a target trace and each span, and the parent-child relationship between spans. The vertical axis represents trace, and the horizontal axis represents span. Trace1 has an inclusion relationship with spanA, spanB, spanC, spanD, and spanE, indicating that spanA, spanB, spanC, spanD, and spanE belong to a complete call, and spanA has a parent-child relationship with spanB and spanC, respectively, and spanB has a parent-child relationship with spanC and spanD, respectively.

[0124] like Figure 8 As shown, an embodiment of the present application provides a schematic diagram of a tracing tree, where the tracing tree includes the spans included in the target tracing and the parent-child relationship between the spans. Each node of the tracing tree represents a span, and the connection between the nodes represents the parent-child relationship between the spans. The parent-child relationship is actually generated based on the call relationship between the microservices to which the spans belong. Figure 7 The trace tree shown is based on Figure 8 The inclusion relationship and parent-child relationship are generated in , so I will not go into details here.

[0125] In some embodiments, before obtaining the span identifier of the first span, the method further includes:

[0126] Get the tracking ID of the target tracking to which the first span belongs based on the following method:

[0127] In the case where the second data packet is obtained based on the first data packet and other data packets, selecting a target data packet from the first data packet and other data packets; using the tracking identifier included in the Protobuf data of the target data packet as the tracking identifier of the target tracking to which the first span belongs; or

[0128] Generate a new tracking identifier, and use the new tracking identifier as the tracking identifier of the target tracking to which the first span belongs;

[0129] When the second data packet is obtained based on the first data packet and not based on other data packets, the tracking identifier contained in the Protobuf data of the first data packet is used as the tracking identifier of the target tracking belonging to the first span; the tracking identifiers contained in the Protobuf data of the first data packet and the other data packets are different.

[0130] It can be understood that the embodiment of the present application needs to determine the tracking identifier of the target tracking to which the first span belongs when generating the first span. When the second data packet is obtained based on the first data packet and not based on other data packets, it means that the second data packet has only one data source, the first data packet. This situation does not belong to multi-path aggregation, and the tracking identifier contained in the Protobuf data of the first data packet can be directly used as the tracking identifier of the target tracking to which the first span belongs.

[0131] However, in a distributed system, it is inevitable to encounter the problem of multi-channel aggregation of data packets. For example, the second data packet is obtained through the first data packet and other data packets, and the tracking identifiers contained in the Protobuf data of the first data packet and the other data packets are different. For example, the second data packet is a video data stream, the first data packet is an image data stream, and the other data packets are audio data streams. In this case, it belongs to multi-channel aggregation and there are multiple traceIDs. In this case, it is necessary to determine the traceID to which the first span generated by the first microservice belongs.

[0132] This embodiment of the application provides the following two methods to determine the traceID to which the first span belongs:

[0133] Method 1: Select a target data packet from the first data packet and other data packets; use the tracking identifier contained in the Protobuf data of the target data packet as the tracking identifier of the target tracking to which the first span belongs;

[0134] Method 2: Generate a new tracking identifier, and use the new tracking identifier as the tracking identifier of the target tracking to which the first span belongs.

[0135] In method one, the embodiment of the present application can select a target traceID from multiple traceIDs according to a certain selection rule. Specifically, a target data packet can be selected from the first data packet and the other data packets; the tracing identifier contained in the Protobuf data of the target data packet is used as the tracing identifier of the target trace belonging to the first span, so that a trace among multiple traces can be complete and will not be truncated due to the newly generated traceID.

[0136] For details, see Figure 9a The embodiment of the present application provides a schematic diagram of obtaining a tracking identifier for target tracking based on method one. Assuming that the traceIDs in the Protobuf data of multiple data packets are traceA, traceB, and traceC, method one is to select a traceID from traceA, traceB, and traceC as the tracking identifier for the target tracking belonging to the first span, for example, select traceA.

[0137] In the second method, the embodiment of the present application creates a new tracing identifier. After multi-path aggregation, it can be understood as generating a new trace. A new traceID is created to represent the new trace, and the new traceID of the new trace is used as the tracing identifier of the target trace belonging to the first span.

[0138] For details, see Figure 9b The embodiment of the present application provides a schematic diagram of obtaining a tracking identifier for target tracking based on method 2. Assuming that the traceIDs in the Protobuf data of multiple data packets are traceA, traceB, and traceC, method 2 generates a new traceID, and the new traceID is traceD, which serves as the tracking identifier for the target tracking to which the first span belongs.

[0139] In practical applications, when the Protobuf data of at least one of the first data packet and the other data packet includes traceID, the priority of method 1 is usually higher than that of method 2, that is, method 1 is used to directly select a target data packet, and the tracking identifier contained in the Protobuf data of the target data packet is used as the tracking identifier of the target tracking belonging to the first span, so that the target tracking is complete and will not be truncated due to the newly generated traceID.

[0140] When the Protobuf data in the first data packet and other data packets do not include traceID, for example, when the first microservice in the DAG processes the first data packet and other data packets, the above method 1 cannot be used, and the above method 2 is directly used.

[0141] In some embodiments, selecting a target data packet from the first data packet and the other data packets includes:

[0142] sorting the data packet names of the first data packet and the other data packets;

[0143] The data packet with the target bit sequence in the sorting result is taken as the target data packet.

[0144] The selection rule of the embodiment of the present application can be to sort the data packet name stream_name of the first data packet and other data packets. Specifically, they can be sorted according to the first letter of the data packet name (and so on. If the first letter is the same, compare the next letter). The data packet with the target sequence in the sorting result can be used as the target data packet. The target sequence can be any position, such as the first sequence, the second sequence, the last sequence, etc. Normally, the target sequence is the first sequence.

[0145] Of course, in some other embodiments, the target data packet may be determined according to other selection rules, which is not limited to this.

[0146] In some embodiments, after selecting a target data packet from the first data packet and the other data packets, the method further comprises:

[0147] When the target data packet is the first data packet, creating a span link between the tracking identifier included in the Protobuf data of the other data packets and the span identifier of the first span;

[0148] When the target data packet is not the first data packet, a span link is created between the tracking identifier included in the Protobuf data of the first data packet and the span identifier of the first span.

[0149] Span Links is a mechanism used in Distributed Tracing systems to represent associations or dependencies between different tracing contexts, especially in call chains across services or systems. Span Links are often used to represent the relationship between "multiple independent traces or spans", especially when some operations are not directly derived from the parent span.

[0150] In the embodiment of the present application, when the target data packet is the first data packet, the tracking trace corresponding to the tracking identifier contained in the Protobuf data of the first data packet is complete, while the tracking trace corresponding to the tracking identifier contained in the Protobuf data of other data packets is incomplete. In this case, it is necessary to create a span link between the tracking identifier contained in the Protobuf data of other data packets and the span identifier of the first span.

[0151] When the target data packet is not the first data packet, the tracking trace corresponding to the tracking identifier contained in the Protobuf data of other data packets is complete, while the tracking trace corresponding to the tracking identifier contained in the Protobuf data of the first data packet is incomplete. In this case, it is necessary to create a span link between the tracking identifier contained in the Protobuf data of the first data packet and the span identifier of the first span.

[0152] The above process can be understood in business terms as a main traceID (the tracing identifier contained in the Protobuf data in the target data packet) is attached with other stream traceIDs (the tracing identifiers contained in data packets other than the target data packet). This main traceID is selected as the trunk and then passed on. The remaining stream traceIDs are associated with the main traceID through span links.

[0153] In some embodiments, after generating a new tracking identifier, the method further comprises:

[0154] Create span links between the new tracking identifier and the tracking identifiers included in the Protobuf data of the first data packet and other data packets.

[0155] In the second method, the embodiment of the present application creates a new tracing identifier. After multi-path aggregation, it can be understood as generating a new trace, creating a new traceID to represent the new trace, and then the new traceID is linked to the upstream tracing identifier through Span Links, that is, creating a new tracing identifier and a span link between the tracing identifiers contained in the Protobuf data of the first data packet and other data packets, respectively, so that the new traceID keeps in touch with the original multi-path traceID to ensure the integrity of the target trace.

[0156] The above method 1 or method 2 can be set for the distributed tracing system. Usually, the above method 1 is set. Therefore, consistent results can be guaranteed on all microservices according to the stream_name sorting. In this way, the trace is complete as a whole and will not be truncated due to the newly generated traceID.

[0157] The data tracking method provided in the embodiment of the present application can be executed by a data tracking device. In the embodiment of the present application, the data tracking device provided in the embodiment of the present application is described by taking the data tracking method executed by the data tracking device as an example.

[0158] The embodiment of the present application also provides a data tracking device.

[0159] like Fig.10 As shown, the data tracking device includes: a first processing module 1010 , a second processing module 1020 and a third processing module 1030 .

[0160] The first processing module 1010 is used to obtain a span identifier of a first span; the first span is a span generated by a process of obtaining a second data packet by processing a first microservice to receive a first data packet;

[0161] The second processing module 1020 is used to inject the tracking context information including the span identifier of the first span and the tracking identifier of the target tracking to which the first span belongs into the serialized Protobuf data of the second data packet to obtain a third data packet;

[0162] The third processing module 1030 is used to instruct the first microservice to send a third data packet to the second microservice.

[0163] According to the data tracking device provided in the embodiment of the present application, by injecting the tracing context information including the span identifier of the first span and the tracing identifier of the target trace to which the first span belongs into the serialized Protobuf data of the second data packet, a third data packet is obtained, and the first microservice is instructed to send the third data packet to the second microservice. In terms of tracing granularity, tracing at the data packet level is achieved, and tracing at the data packet level is conducive to quickly locating anomalies or faults.

[0164] In some embodiments, the data tracking device further comprises:

[0165] The fourth processing module is used to obtain the tracking identifier of the target tracking to which the first span belongs based on the following method:

[0166] In the case where the second data packet is obtained based on the first data packet and other data packets, selecting a target data packet from the first data packet and other data packets; using the tracking identifier included in the Protobuf data of the target data packet as the tracking identifier of the target tracking to which the first span belongs; or

[0167] Generate a new tracking identifier, and use the new tracking identifier as the tracking identifier of the target tracking to which the first span belongs;

[0168] When the second data packet is obtained based on the first data packet and not based on other data packets, the tracking identifier contained in the Protobuf data of the first data packet is used as the tracking identifier of the target tracking belonging to the first span; the tracking identifiers contained in the Protobuf data of the first data packet and the other data packets are different.

[0169] In some embodiments, the fourth processing module is specifically configured to:

[0170] sorting the data packet names of the first data packet and the other data packets;

[0171] The data packet with the target bit sequence in the sorting result is taken as the target data packet.

[0172] In some embodiments, the data tracking device further comprises:

[0173] a fifth processing module, for creating a span link between a tracking identifier included in Protobuf data of other data packets and a span identifier of the first span when the target data packet is the first data packet;

[0174] When the target data packet is not the first data packet, a span link is created between the tracking identifier included in the Protobuf data of the first data packet and the span identifier of the first span.

[0175] In some embodiments, the fifth processing module is further configured to:

[0176] Create span links between the new tracking identifier and the tracking identifiers included in the Protobuf data of the first data packet and other data packets.

[0177] In some embodiments, the serialized Protobuf data of the first data packet includes a span identifier of the second span generated by the third microservice; the first data packet is generated by the third microservice;

[0178] The data tracking device also includes:

[0179] The sixth processing module is used for:

[0180] Creating a parent-child relationship between the span identifier of the first span and the span identifier of the second span; wherein the parent-child relationship indicates that the first span is a child span and the second span is a parent span;

[0181] Creating an inclusion relationship between the tracking identifier of the target tracking and the span identifier of the first span;

[0182] Based on the tracking identifier of the target tracking and the inclusion relationship between each span, as well as the parent-child relationship between each span, a tracking tree for target tracking is constructed.

[0183] In some embodiments, the sixth processing module is specifically configured to:

[0184] Based on the tracking identifier of the target tracking and the inclusion relationship between the spans, each span included in the target tracking is obtained;

[0185] Based on the parent-child relationship between each span, each span is linked to obtain the tracking tree for target tracking.

[0186] The data tracking device in the embodiment of the present application can be an electronic device or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0187] The data tracking device in the embodiment of the present application may be a device having an operating system. The operating system may be a Microsoft (Windows) operating system, an Android (Android) operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0188] The data tracking device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented in the method embodiment of FIG. 9 are not described again here.

[0189] In some embodiments, the embodiments of the present application provide an intelligent agent. In practical applications, the intelligent agent may be the aforementioned TongAI. The intelligent agent can implement each process implemented in the aforementioned data tracking method embodiment. To avoid repetition, it will not be repeated here.

[0190] In some embodiments, Fig.11 As shown, an embodiment of the present application also provides an electronic device 1100, including a processor 1101, a memory 1102, and a computer program stored in the memory 1102 and executable on the processor 1101. When the program is executed by the processor 1101, each process of the above-mentioned data tracking method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0191] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0192] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned data tracking method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0193] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0194] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned data tracking method when executed by a processor.

[0195] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0196] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned data tracking method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0197] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0198] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0199] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0200] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0201] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0202] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A data tracking method, characterized in that: Data tracking system for intelligent agents, including: Obtaining a span identifier of a first span; the first span is a span generated by a process in which the first microservice processes a received first data packet to obtain a second data packet; Injecting the tracing context information including the span identifier of the first span and the tracing identifier of the target trace to which the first span belongs into the serialized Protobuf data of the second data packet to obtain a third data packet; Instruct the first microservice to send the third data packet to the second microservice; the first data packet, the second data packet and the third data packet belong to a stream data packet Data Frame; Before obtaining the span identifier of the first span, the method further includes: In the case where the second data packet is obtained based on the first data packet and other data packets, and the Protobuf data of at least one of the first data packet and the other data packets includes traceID, select a target data packet from the first data packet and the other data packets; use the trace identifier included in the Protobuf data of the target data packet as the trace identifier of the target trace to which the first span belongs; The first data packet and the second data packet belong to different types of stream data packets DataFrame in multimodal data; the multimodal data includes at least two of image data stream, audio data stream and video data stream; the first microservice and the second microservice both have corresponding nodes in the DAG network of the intelligent agent.

2. The data tracking method according to claim 1, characterized in that: Before obtaining the span identifier of the first span, the method further includes: When the second data packet is obtained based on the first data packet and other data packets, and when the Protobuf data in the first data packet and other data packets do not include traceID, generate a new tracing identifier, and use the new tracing identifier as the tracing identifier of the target trace to which the first span belongs; When the second data packet is obtained based on the first data packet and not based on the other data packets, the tracking identifier contained in the Protobuf data of the first data packet is used as the tracking identifier for tracking the target belonging to the first span; the tracking identifiers contained in the Protobuf data of the first data packet and the other data packets are different.

3. The data tracking method according to claim 2, characterized in that: The selecting a target data packet from the first data packet and the other data packets comprises: sorting the data packet names of the first data packet and the other data packets; The data packet with the target bit sequence in the sorting result is taken as the target data packet.

4. The data tracking method according to claim 2 or 3, characterized in that: After selecting a target data packet from the first data packet and the other data packets, the method further includes: When the target data packet is the first data packet, creating a span link between the tracking identifier included in the Protobuf data of the other data packet and the span identifier of the first span; When the target data packet is not the first data packet, a span link is created between the tracking identifier included in the Protobuf data of the first data packet and the span identifier of the first span.

5. The data tracking method according to claim 2, characterized in that: After generating a new tracking identifier, the method further includes: Create span links between the new tracking identifier and the tracking identifiers included in the Protobuf data of the first data packet and the other data packets.

6. The data tracking method according to claim 1, characterized in that: The serialized Protobuf data of the first data packet includes a span identifier of the second span generated by the third microservice; The first data packet is generated by the third microservice; After obtaining the span identifier of the first span, the method further includes: Creating a parent-child relationship between the span identifier of the first span and the span identifier of the second span; wherein the parent-child relationship indicates that the first span is a child span and the second span is a parent span; Creating an inclusion relationship between the tracking identifier of the target tracking and the span identifier of the first span; Based on the tracking identifier of the target tracking and the inclusion relationship between the spans, and the parent-child relationship between the spans, a tracking tree for the target tracking is constructed.

7. The data tracking method according to claim 6, characterized in that: The tracking tree of the target tracking is constructed based on the tracking identifier of the target tracking and the inclusion relationship between the spans, and the parent-child relationship between the spans, including: Based on the inclusion relationship between the tracking identifier of the target tracking and each span, obtaining each span included in the target tracking; Based on the parent-child relationship between the spans, the spans are linked to obtain the tracking tree of the target tracking.

8. A data tracking device, characterized in that: Data tracking system for intelligent agents, including: A first processing module is used to obtain a span identifier of a first span; the first span is a span generated by a process of obtaining a second data packet by processing a first microservice to receive a first data packet; A second processing module is used to inject the tracking context information including the span identifier of the first span and the tracking identifier of the target tracking to which the first span belongs into the serialized Protobuf data of the second data packet to obtain a third data packet; A third processing module is used to instruct the first microservice to send the third data packet to the second microservice; the first data packet, the second data packet and the third data packet belong to a stream data packet Data Frame; a fourth processing module, configured to select a target data packet from the first data packet and the other data packets when the second data packet is obtained based on the first data packet and the other data packets, and when the Protobuf data of at least one of the first data packet and the other data packets includes traceID; and use the trace identifier included in the Protobuf data of the target data packet as the trace identifier of the target trace to which the first span belongs; The first data packet and the second data packet belong to different types of stream data packets DataFrame in multimodal data; the multimodal data includes at least two of image data stream, audio data stream and video data stream; the first microservice and the second microservice both have corresponding nodes in the DAG network of the intelligent agent.

9. An intelligent agent, characterized in that: The data tracking operation is performed based on the data tracking method according to any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the data tracking method according to any one of claims 1 to 7 is implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data tracking method according to any one of claims 1 to 7 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the data tracking method according to any one of claims 1 to 7 is implemented.

Images