Data processing method, system, device and data management center

Abstract

An embodiment of the present application provides a data processing method, system, device, and data management center, which relate to the field of computer technologies and can be applied to scenarios such as cloud technologies, artificial intelligence, intelligent transportation, and Internet of Vehicles. The method includes: sending a log query request to a data management center to instruct the data management center to return at least one debugging log and corresponding debugging sequence information; displaying the at least one received debugging log and the corresponding debugging sequence information; wherein, the debugging sequence information is generated according to the routing information of each data processing system participating in processing the debugging log, and the routing information includes the actual debugging sequence of each data processing system, the system identifier, and the log identifier of the debugging log. The embodiments of the present application can help debuggers accurately and quickly locate the links where problems occur during the debugging process and reduce the cost of manual problem troubleshooting.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a data processing method, system, apparatus, and data management center. Background Technology

[0002] Within a company, multiple systems typically collaborate on processing data. However, due to issues with the source data reporting format, network, or the processing subsystem itself, the data may not reach the destination system.

[0003] To address this issue, the industry typically offers solutions such as real-time integration testing and packet capture to verify data reporting connectivity. However, these verification methods exist in isolation within the data reporting and processing links and do not cover the entire system. This means that when there is no data at a certain stage, they cannot effectively help users troubleshoot upstream problems, nor can they help troubleshoot downstream problems. Troubleshooting data absence issues at each stage becomes an extremely difficult problem. Summary of the Invention

[0004] This application provides a data processing method, system, apparatus, and data management center that overcomes or at least partially solves the above-mentioned problems.

[0005] Firstly, a data processing method is provided, the method comprising:

[0006] Obtain messages, which include debug logs sent by the device or debug data and routing information from the previous data processing system;

[0007] Generate debugging data and routing information for the current data processing system based on the message;

[0008] The routing information of the current data processing system is sent to the data management center, and the debugging data and routing information of the current data processing system are packaged into a new message and sent to the next data processing system.

[0009] The routing information includes the actual debugging sequence of the corresponding data processing system, the system identifier, and the log identifier of the debugging log.

[0010] Secondly, a data processing method is provided, the method comprising:

[0011] Send a log query request to the data management center to instruct the data management center to return at least one debug log and the corresponding debug order information;

[0012] Display the received at least one debug log and the corresponding debug order information;

[0013] The debugging order information is generated based on the routing information of each data processing system involved in processing the debugging log. The routing information includes the actual debugging order of each data processing system, the system identifier, and the log identifier of the debugging log.

[0014] Thirdly, a data processing method is provided, which includes:

[0015] The receiving device sends a log query request, the log query request including the unique identifier of the device;

[0016] Find and return at least one debug log and debug sequence information corresponding to the unique identifier of the device;

[0017] The debugging order information is generated based on the routing information of each data processing system involved in processing the debugging log. The routing information includes the actual debugging order of each data processing system, the system identifier, and the log identifier of the debugging log.

[0018] Fourthly, an apparatus is provided, comprising:

[0019] The log query module is used to send a log query request to the data management center, instructing the data management center to return at least one debug log and the corresponding debug order information;

[0020] The log display module is used to display the received at least one debug log and the corresponding debug order information;

[0021] The debugging order information is generated based on the routing information of each data processing system involved in processing the debugging log. The routing information includes the actual debugging order of each data processing system, the system identifier, and the log identifier of the debugging log.

[0022] Fifthly, a data processing system is provided, comprising:

[0023] The message acquisition module is used to acquire messages, which include debugging logs sent by the device or debugging data and routing information from the previous data processing system;

[0024] The message processing module is used to generate debug data and routing information for the current data processing system based on the message.

[0025] The message sending module is used to send the routing information of the current data processing system to the data management center, and to package the debugging data and routing information of the current data processing system into a new message and send it to the next data processing system.

[0026] The routing information includes the actual debugging sequence of the corresponding data processing system, the system identifier, and the log identifier of the debugging log.

[0027] Sixthly, a data management center is provided, including:

[0028] A log query request receiving module is used to receive log query requests sent by the device, wherein the log query request includes the unique identifier of the device;

[0029] The log return module is used to find and return at least one debug log and debug sequence information corresponding to the unique identifier of the device;

[0030] The debugging order information is generated based on the routing information of each data processing system involved in processing the debugging log. The routing information includes the actual debugging order of each data processing system, the system identifier, and the log identifier of the debugging log.

[0031] In a seventh aspect, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the methods provided in the first, second, or third aspects.

[0032] Eighthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the methods provided in the first, second, or third aspects.

[0033] Ninthly, embodiments of this application provide a computer program including computer instructions stored in a computer-readable storage medium. When a processor of a computer device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, causing the computer device to perform steps implementing the methods provided in the first, second, or third aspects.

[0034] The data processing method, system, device, and data management center provided in this application embodiment, by sending a log query request to the data management center, instructs the data management center to return at least one debug log and the corresponding debug order information, and displays the received at least one debug log and the corresponding debug order information. This eliminates the need for debuggers to manually check each upstream link, and helps debuggers quickly obtain debug logs and debug order information, laying the foundation for subsequent users to quickly find abnormal data processing systems from debug order information. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0036] Figure 1 A schematic diagram of the system architecture of the application environment for the data processing method provided in the embodiments of this application;

[0037] Figure 2 A schematic flowchart of the data processing method provided in the embodiments of this application on the device side;

[0038] Figure 3 A schematic diagram of the interface for data debugging of the device provided in the embodiments of this application;

[0039] Figure 4 This is a schematic diagram of the topology diagram provided in one embodiment of this application;

[0040] Figure 5 This is a schematic diagram of the topology diagram provided in another embodiment of this application;

[0041] Figure 6 A flowchart illustrating the data processing method provided in this application embodiment on the data processing system side;

[0042] Figure 7 A schematic diagram illustrating data transmission through a data pipeline provided in an embodiment of this application;

[0043] Figure 8 This is a schematic diagram of the topology diagram provided in another embodiment of this application;

[0044] Figure 9 A flowchart illustrating the data processing system and data tracking SDK involved in processing debug logs, provided in an embodiment of this application;

[0045] Figure 10 A flowchart illustrating the data processing method on the data management center side provided in this application embodiment;

[0046] Figure 11 A schematic diagram of the system architecture for the application environment of a data processing method provided in another embodiment of this application;

[0047] Figure 12 This application provides an example of the interaction diagram between various services in a data management center.

[0048] Figure 13 A schematic diagram of the structure of an apparatus provided in an embodiment of this application;

[0049] Figure 14 This is a schematic diagram of the structure of a data processing system provided in an embodiment of this application;

[0050] Figure 15 This application provides a schematic diagram of the structure of a data management center according to an embodiment of the present application.

[0051] Figure 16 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0052] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0053] Those skilled in the art will understand that, unless explicitly stated otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in the specification of this application means the presence of features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless couplings. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0055] The data processing method, system, apparatus, and data management center provided in this application are intended to solve the above-mentioned technical problems of the prior art.

[0056] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0057] Please see Figure 1 The figure exemplifies a schematic diagram of the system architecture of the data processing method according to an embodiment of this application. As shown, the system architecture 100 may include device 101, one or more of data processing systems 201, 202, and 203, and data management center 301. It should be understood that... Figure 1The number of devices and data processing systems shown is merely illustrative. Depending on implementation needs, any number of devices and data processing systems can be included.

[0058] Device 101 can be various electronic devices with a display screen, including but not limited to desktop computers, portable computers, smartphones, tablets, smart voice interaction devices, smart home appliances, in-vehicle devices, etc. The device in this embodiment can generate debugging logs and send them to the data processing system when data scheduling is required. The data processing system then performs debugging based on the debugging logs. Furthermore, when it is necessary to view the debugging results, a log query request can be sent to the data management center 301 to instruct the data management center to return at least one debugging log and the corresponding debugging sequence information.

[0059] The data processing system and data management center in this application embodiment can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0060] A data processing system is a system used for data processing, such as a log server, a real-time ETL (Extract, Cleanse, Transform, Load) task system, a data warehouse, and a real-time online analysis system. Different data processing systems generally exist in isolation at different stages of data processing. A data processing system can obtain debugging data and routing information from the previous data processing system; it processes this debugging data and routing information to generate debugging data and routing information for the current data processing system. The data management center 301 then generates debugging sequence information based on the routing information from each data processing system.

[0061] Please see Figure 2 The figure exemplarily illustrates a flowchart of the data processing method of this application on the device side, as shown, including:

[0062] S101. Send a log query request to the data management center to instruct the data management center to return at least one debug log and the corresponding debug order information;

[0063] In this embodiment of the application, the debugger can send a log query request to the data management center through the device. The log query request may include the debugger's user identifier and / or the device identifier, etc., where the user identifier is the identifier information that represents the debugger's identity, and the device identifier is the identifier information that represents the device used by the debugger.

[0064] In this embodiment, the user identifier of the debugger can be registered in advance in the data management center. When the debugger sends a log query request to the data management center for the first time through the device, the data management center records the device identifier of the debugger's device when it determines that the user identifier is valid. Thus, debuggers who use the device subsequently can perform the data processing tasks of this embodiment, improving the convenience of data management.

[0065] The data management center verifies the user identifier and / or device identifier in the received log query request. If the identifier is valid, it will return the debugging logs and debugging sequence information of the debugger or the debugging device before or during the current debugging.

[0066] Understandably, during application execution, discrepancies between the results and expectations often arise. Developers need to investigate and fix vulnerabilities in the program to achieve the expected results. Generally, developers need to access the program's debug logs to locate the problems that need to be solved. Allowing debug logs requires collaboration among multiple data processing systems; therefore, different data processing systems may have different debugging orders during scheduling. In this embodiment, the data management center records the actual debugging order and system identifiers of each data processing system participating in the debug log processing, and returns this debugging order information along with the debug logs to the device for display. The system identifier can generally be implemented using the <type, instance ID> format. The type can refer to the system's functional type, such as whether it belongs to a log server, a data warehouse, or a data analysis system, etc., and the instance ID is the unique identifier of the instance.

[0067] In one embodiment, the data management center can return debug logs to the device in reverse chronological order, allowing the debugger to prioritize viewing the most recent debug logs. For example, if there are four debug logs dated January 1, 2, 3, and 4, 2021, the data management center can return the debug logs from January 4, 3, 2, and 1 to the device in that order. When the device displays the logs according to the received order, it can show them in the order of January 4, 3, 2, and 1, allowing the debugger to see the debug logs from January 4 first.

[0068] The debugging order information in this embodiment is generated based on the routing information of each data processing system participating in the processing of the debugging logs. Each data processing system, by participating in the processing of the debugging logs, sends the generated routing information to the data management center, which then generates the debugging order information based on the routing information of each debugging log. The routing information in this embodiment includes the actual debugging order of each data processing system, the system identifier, and the log identifier of the debugging logs.

[0069] The debugging sequence information may include the actual debugging order of each data processing system involved in processing the debugging log, as well as the system identifier. For example, the debugging sequence information may be (1)xy32123→(2)xDRF2199→(3)xITc2895, where (1), (2), and (3) represent the actual debugging order of each data processing system, and xy32123, xDRF2199, and xITc2895 represent the system identifiers of each data processing system. The debugger can clearly know which data processing systems participated in processing a particular debugging log and the debugging order of each system through this debugging sequence information.

[0070] S102. Display the received at least one debug log and the corresponding debug order information;

[0071] For example, debuggers can perform data debugging and query debug logs using preset applications; please refer to [link to relevant documentation]. Figure 3 The figure illustrates an exemplary interface diagram of the device for data debugging according to an embodiment of this application. As shown in the figure, the left side of the figure shows the function entry for data debugging and viewing debugging logs. Users can access the debugging log viewing page shown in the figure through the function entry for viewing debugging logs. The debugging log viewing page displays the debugging logs returned by the data management center and the corresponding debugging order information. As shown in the debugging log viewing page, each line of data includes the debugging time, the unique identifier of the device (device identifier), the log identifier of the debugging log, and the debugging order information.

[0072] It should be noted that the data processing method in this application embodiment, by sending a log query request to the data management center, instructs the data management center to return at least one debug log and the corresponding debug order information, and displays the received at least one debug log and the corresponding debug order information, can help debuggers quickly obtain debug logs and debug order information, laying the foundation for subsequent users to quickly find abnormal data processing systems from debug order information.

[0073] Based on the above embodiments, as an optional embodiment, at least one debug log and corresponding debug sequence information are displayed, followed by:

[0074] In response to the operation command on the debugging sequence information, the topology diagram corresponding to the debugging sequence information is obtained and displayed.

[0075] When a user interacts with the adjustment order information corresponding to a specific debug log, the device will further display the topology diagram corresponding to the debug order information. In this embodiment, the interaction can be clicking, long-pressing, swiping, etc. The topology diagram can be generated by the device based on the debug order information, or the device can generate a topology diagram retrieval request in response to an operation command. Optionally, the topology diagram retrieval request may include a unique identifier for the debug log. The data management center generates and returns the topology diagram based on the topology diagram retrieval request.

[0076] The topology graph of this application embodiment includes a first node, a first directed edge, and the routing information of the first node; wherein, the first node is used to represent a data processing system participating in processing the debug log, and the first directed edge is used to connect two nodes that have an adjacent relationship in the actual debugging sequence. A debug log, once generated, is sequentially passed to multiple data processing systems. For example, data processing system 1 generates debug data 1 based on the debug log, data processing system 2 processes debug data 1 to generate debug data 2, data processing system 3 processes debug data 2 to generate debug data 3, and so on, until debugging is stopped or completed. These data systems participating in processing the debug log are all displayed as nodes in the topology graph. Simultaneously, the topology graph also connects two nodes that have a vector relationship in the actual debugging sequence with directed edges.

[0077] For example, for a certain debug log, a total of 4 data processing systems participated in the processing of the debug log. Specifically, data processing system 1 processed the debug log to generate debug data 1, data processing system 2 processed debug data 1 to generate debug data 2, data processing system 3 processed debug data 2 to generate debug data 3, data processing system 4 processed debug data 3 to generate debug data 4, and the debugging ended.

[0078] Please see Figure 4 The example illustrates a schematic diagram of a topology graph according to an embodiment of this application. This topology graph involves four nodes, 1 to 4, which correspond to data processing systems 1 to 4, respectively. A directed edge 1 connects node 1 and node 2, a directed edge 2 connects node 2 and node 3, and a directed edge 3 connects node 3 and node 4. Clearly, this topology graph allows users to intuitively determine the data processing systems involved in the debugging log and the debugging sequence of each system, providing assurance for users to troubleshoot faulty data processing systems.

[0079] Based on the above embodiments, as an optional embodiment, the topology graph of this application embodiment further includes at least one of a second node and a second directed edge.

[0080] The second node is used to represent the data processing system that did not participate in the processing of the debug log as planned. The data processing system that was planned to participate in the processing of the debug log refers to the data processing system that was set in advance by the debugger or determined by the data management center based on historical debug records. The data processing system referred to by the second node includes both debug logs that appeared in the plan but were not actually processed and data processing systems that were not debugged according to the preset debug.

[0081] For example, a debug log was originally planned to involve four data processing systems. The planned debugging sequence was as follows: Data processing system 1 would process the debug log and generate debug data 1; data processing systems 2 and 3 would process debug data 1 respectively and generate debug data 2 and 3; data processing system 4 would process debug data 2 and 3 and generate data 4, ending the debugging process. However, during actual debugging, data processing system 3 did not generate debug data 3 based on debug data 1, and correspondingly, data processing system 4 also failed to correctly generate debug data 4. Therefore, since neither data processing system 3 nor 4 participated in processing the debug log as planned, they both belong to the second node.

[0082] The second directed edge is used to connect two adjacent nodes in the target debugging sequence; the target debugging sequence is a debugging sequence that is different from the preset debugging sequence.

[0083] Please see Figure 5 This example illustrates a schematic diagram of the topology graph of another embodiment of this application. A debugging log was originally planned to involve five data processing systems, corresponding to nodes 1-5 in the topology graph. The planned debugging sequence was as follows: Data processing system 1 processes the debugging log to generate debugging data 1; data processing systems 2 and 3 process debugging data 1 to generate debugging data 2 and 3 respectively; data processing system 4 processes data 2 and 3 to generate debugging data 4; and data processing system 5 generates data 5 based on debugging data 4, ending the debugging process. However, during actual debugging, data processing system 3 did not generate debugging data 3 based on data 1 and send it to data processing system 4. Correspondingly, data processing system 4 also did not correctly generate debugging data 4, and therefore did not send data to data processing system 5. Since data processing systems 3, 4, and 5 did not participate in processing the debugging log as planned, they all belong to the second node. A second directed edge connects data processing systems 3 and 4, and data processing systems 4 and 5 are also directly connected by a second directed edge. Figure 5It can be clearly found that there are obvious differences in the display of the first node and the second node, as well as the first directed edge and the second directed edge in the embodiments of the present application. It should be understood that the embodiments of the present application can also distinguish the first node and the second node, the first directed edge and the second directed edge in other display ways, such as from colors, shapes, fonts, etc. The embodiments of the present application do not make specific limitations.

[0084] By distinguishing the display methods of the first node and the second node, the first directed edge and the second directed edge in the embodiments of the present application, it helps the debugger intuitively see which data processing systems have problems at which steps. Figure 5 For example, since the display methods of node 3, node 4, and node 5 are the display methods of the second node, the debugger can intuitively see that the data processing systems 3-5 have an unexpected state during the debugging process. Further, since the directed edge between node 1 and node 3 is the first directed edge, and the directed edge between node 3 and node 4 is the second directed edge, it can be known that the data processing system 3 successfully received the debugging data of the data processing system 1, but did not generate debugging data 3 based on the debugging data 1 and send it to the data processing system 4. Similarly, the directed edge between node 2 and node 4 is the first directed edge, and the directed edge between node 4 and node 5 is the second directed edge. It can be known that the data processing system 4 successfully received the debugging data 2 of the data processing system 2, but since the data processing system 4 did not receive the debugging data 3, the data processing system 4 did not generate debugging data 4, and thus did not send the expected debugging data to the data processing system 5. It can be seen that the topology diagram of the embodiments of the present application can significantly improve the efficiency of the debugger to understand the debugging process, accurately and quickly locate the link where the problem occurs, and reduce the cost of manual troubleshooting.

[0085] The embodiments of the present application can pre-set the data processing systems planned to participate in the debugging log and the preset debugging order for the debugging log, so that after the debugging is aborted or ended, according to the data processing systems actually participating in processing the debugging log and the debugging order, the data processing systems that did not process according to the plan can be determined, thereby facilitating the user to more intuitively determine the abnormal situation of the debugging. Figure 5 For example, the debugger can intuitively determine that there are abnormalities in the data processing systems 3-5 based on the two second directed edges, and thus can quickly initiate a troubleshooting of the data processing systems 3-5.

[0086] Based on the above embodiments, as an optional embodiment, the routing information further includes the debugging start time and the debugging end time of the data processing system; the debugging start time is the time when the data processing system determines the actual debugging order; the debugging end time is the time when the data processing system generates debugging data.

[0087] Furthermore, embodiments of this application can also calculate the difference between the debugging start time and the debugging end time, that is, the debugging time of the data processing system. By recording the debugging start time, debugging end time and / or debugging time in the routing information, and marking the debugging start time, debugging end time and / or debugging time in the topology diagram accordingly, it can help the debugger to see the processing time of different data processing systems more intuitively, laying the foundation for discovering faulty data processing systems.

[0088] Furthermore, embodiments of this application can also record the difference between the debugging end time of the preceding node and the debugging start time of the following node in the routing information and topology diagram, which helps the debugger to intuitively understand the processing delay between the two data processing systems and lays the foundation for discovering other faults outside the data processing system.

[0089] Based on the above embodiments, as an optional embodiment, the topology diagram of this application embodiment also marks at least one of the debugging time of the first node and the transmission time of the first directed edge;

[0090] Wherein, the debugging duration of the first node is the time interval between the debugging start time and debugging end time of the corresponding data processing system; the transmission duration of the first directed edge is the time interval between the debugging end time of the data processing system corresponding to the head node of the directed edge and the debugging start time of the data processing system corresponding to the tail node of the directed edge.

[0091] For example, if the debugging start time of a data processing system (data processing system a) is 13:20 and the debugging end time is 13:24, then the debugging duration of the first node is 4 minutes. Another data processing system (data processing system b) receives the debugging data from data processing system a, so there is a first directed edge between data processing system a and data processing system b. If the debugging start time of data processing system b is 13:25 and the debugging end time is 13:26, then the transmission duration of this first directed edge is 1 minute (the time interval between 13:25 and 13:24).

[0092] This application embodiment records the debugging time of nodes and the transmission time of directed edges in the topology graph, which makes it convenient for the debugger to clearly understand both the debugging time of the data processing system and the transmission time of the debugging data.

[0093] Based on the above embodiments, the embodiments of this application can also pre-set preset index values ​​for the debugging time of each first node and the transmission time of each first directed edge, and prominently display nodes and directed edges whose duration exceeds or does not exceed the preset index values, providing data support for the whole process performance optimization.

[0094] Based on the above embodiments, as an optional embodiment, this application embodiment sends a log query request to the data management center, and further includes the following steps beforehand:

[0095] Send debugging logs to the data processing system. The debugging logs include the log identifier of the debugging logs and the device identifier of the device.

[0096] The user can activate debug mode on the device. The device's identifier will then be displayed on the device interface. The device reports this identifier to the data management center. When the device connects to the data management center, it displays a message indicating successful device identifier registration and prompts the user to begin debugging. If the device cannot connect to the data management center, it will prompt the user to check the network.

[0097] This application embodiment sends debug logs to the data processing system to instruct the data processing system to process the debug logs. It should be noted that the debug logs in this application embodiment also include the log identifier of the debug logs and the device identifier of the device, which lays the foundation for the subsequent data processing system to mark the data generated after processing the debug logs and to track the data processing process.

[0098] Please see Figure 6 The figure illustrates, by way of example, a flowchart of the data processing method of this application embodiment on the data processing system side, as shown in the figure, including:

[0099] S201. Obtain a message, which includes the debug log sent by the device or the debug data and routing information of the previous data processing system.

[0100] In this embodiment of the application, the device can send the debugging log to the data processing system in the form of a message through the data channel. Similarly, the data processing system can send the generated debugging data and routing information to the next data processing system in the form of a message through the data channel. The data pipeline is only used for data transmission and does not perform data transfer operations.

[0101] The routing information in this embodiment includes the actual debugging order of the corresponding data processing system, the system identifier, and the log identifier of the debugging log. By recording the above three pieces of information, the routing information in this embodiment can lay the foundation for updating the actual debugging order and generating debugging order information. Furthermore, the addition of the actual debugging order can identify different debugging orders of the same data processing node when loops exist in the topology graph.

[0102] This application embodiment can send the preset debugging order of the debugging log to the input data pipeline. Thus, after the data processing system generates scheduling data and routing information, it only needs to send the scheduling data and routing information to the data pipeline, and the data pipeline will perform data transmission. This allows the data processing system to focus only on data processing (including debugging).

[0103] It should be noted that the data communication formats between the data processing systems in this application embodiment are diverse. For example, the data reported from the device to the log service system may be based on the HTTP protocol, and the data from the message queue to the ETL process may be based on a special consumption protocol of the message queue. These protocols are required by technology and business needs, and therefore cannot be limited to define a universal protocol. However, nodes still need to know some information to perform special processing on the debugging data to construct routing information for reporting to the data management center. Optionally, this application embodiment uses the extended headers of these specific protocols to send routing data.

[0104] Please see Figure 7 The figure illustrates a schematic diagram of data transmission in a data pipeline according to an embodiment of this application. As shown in the figure, there are three data processing systems in this application scenario: data processing system A1, data processing system B2, and data processing system C3. The expected storage order of the debug log with log identifier rz321 is pre-stored in the data pipeline: data processing system A1 → data processing system C3.

[0105] When data processing system A1 generates debug data 1 and routing information 1 for the debug log, it sends debug data 1 and routing information 1 to the data pipeline. The routing information 1 records the actual debugging order 1 of data processing system A1, system identifier A1, and log identifier rz321 of the debug log. Based on the expected storage order and system identifier A1, the data pipeline determines that the debug data and routing information need to be further sent to data processing system C3.

[0106] For example, there are three data processing systems involved in processing a certain debug log, designated as data processing systems 1 through 3. Data processing system 1 processes the debug log to generate data 1; data processing system 2 processes debug data 1 to generate debug data 2; data processing system 3 processes debug data 2 to generate debug data 3; and data processing system 1 processes debug data 3 to generate debug data 4. The processing then ends. The corresponding topology diagram is shown below. Figure 8 As shown, the directed edges between nodes 1 to 3 corresponding to data processing system 1 to 3 form a ring. There are two actual debugging sequences for node 1, namely 1 and 4. The debugger can clearly see the data flow through the directed edges and nodes in the topology graph.

[0107] The data channel in this application embodiment is not a fixed path. Not only does its path change with the changes in the system topology, but it also routes to different data processing systems depending on the content of the data itself. This application embodiment can construct a data routing topology according to different situations.

[0108] S202. Generate debugging data and routing information for the current data processing system based on the message.

[0109] After receiving debugging data from the previous data processing system, the data processing system can process the debugging data according to a preset data processing method to generate debugging data for the current data processing system. It will also generate routing information for the current data processing system based on the routing information from the previous system. The actual debugging order of the current data processing system in the newly generated routing information is the actual debugging order of the previous system plus one. For example, if the routing information of the previous system is (1, xt01, rz123), and the system identifier of the current system is xt21, then the routing information generated by the current system will be (2, xt21, rz123). Here, 2 indicates that the actual debugging order of the current system is 2, and the system identifier is updated to the current system's own identifier. However, the log identifier in the debugging log remains unchanged, thus ensuring the continuity of debugging log tracking.

[0110] It is understood that the debugging data of the first data processing system in this application embodiment is generated based on the debugging log output by the device, and the debugging data received by subsequent data processing systems are all generated based on the debugging data of the previous data processing system.

[0111] S203. Send the routing information of the current data processing system to the data management center, and package the debugging data and routing information of the current data processing system into a new message and send it to the next data processing system.

[0112] After generating debugging data and routing information, the data processing system in this embodiment needs to send the routing information to the data management center for recording, and send the debugging data and routing information to the next data processing system through the data pipeline. This ensures that the debugging task can be continuously transferred and executed in different data processing systems, and also ensures that the execution process of the debugging task can be recorded, laying the foundation for subsequent querying of debugging records. More importantly, since the data management center does not receive the debugging data, and each system does not need to grant full data access permissions to the troubleshooters, data security is guaranteed.

[0113] Based on the above embodiments, as an optional embodiment, the routing information also includes the debugging start time and debugging end time of the current data processing system;

[0114] The debugging start time is the time when the current data processing system determines the actual debugging sequence; the debugging end time is the time when the current data processing system generates debugging data.

[0115] The data processing system in this application embodiment sends the time when the actual debugging sequence of the current data processing system is obtained as the debugging start time of the current data processing system, and the time when the debugging data of the current data processing system is obtained as the debugging end time of the current data processing system to the data management center.

[0116] This application embodiment records the debugging start time and debugging end time. The data management center records these two times in the debugging sequence information of each node in the topology diagram. This allows the debugger to intuitively see the time consumed by each data processing system during debugging, promptly identify data processing systems with excessively long processing times, and verify whether a fault has occurred.

[0117] Based on the above embodiments, as an optional embodiment, the data processing system can report the debugging start time, debugging end time, debugging data, and routing information to the data management center by calling a preset data tracking software development kit (SDK).

[0118] The data tracing SDK in this application embodiment handles the actual work required for data tracing, thereby reducing the cost of data nodes accessing the data tracing system. The data tracing SDK can provide the aforementioned functionality for different data pipeline types, such as HTTP, Kafka, and gRPC, and also allows users to extend the supported types for specific data pipelines.

[0119] This application embodiment generates debug data and routing information for the current data processing system based on the message, including:

[0120] S301. Call the preset SDK to generate the actual debugging sequence of the current data processing system and the debugging start time based on the debugging log in the message or the routing information of the previous data processing system.

[0121] S302. Process the debug log in the message or the debug data of the previous data processing system to obtain the debug data of the current data processing system.

[0122] S303. Call the SDK to generate the debugging end time, obtain the system identifier of the data processing system and the log identifier of the debugging log, and generate the routing information of the current data processing system.

[0123] Please see Figure 9 The example illustrates a flowchart of the data processing system and data tracking SDK involved in processing debug logs according to an embodiment of this application. Figure 9 As shown:

[0124] 1. The data pipeline sends a message to the data processing system, which includes debug data a and routing information b;

[0125] 2. The data processing system unpacks the message to obtain the debugging data a and routing information b. It can be understood that the debugging data a can be the debugging data after the previous data processing system has performed debugging, or it can be the debugging log sent by the device. The routing information includes the actual debugging order of the previous data processing system. When the routing information is empty, it indicates that the message was sent from the device and the current data processing system is the first data processing system to process the debugging log.

[0126] 3. The data processing system calls the data tracing SDK and, based on the actual debugging order of the previous data processing system recorded in the routing information b, generates the actual debugging order of the current data processing system and adds it to the routing information d of the current data processing system;

[0127] 4. Generate the debugging start time of the previous data processing system based on the time when the actual debugging sequence of the current data processing system was generated;

[0128] 5. The data tracing SDK returns debug data 'a' to the data processing system;

[0129] 6. The data processing system performs debugging-related data processing on debugging data a to obtain debugging data c;

[0130] 7. The data processing system packages the debug data c and sends it to the data tracing SDK;

[0131] 8. The data tracing SDK generates a debugging end time based on the time when the debugging data is received, records it in the routing information d, and packages the debugging data c and the routing information d into a new message and sends it to the data processing system.

[0132] 9. The data processing system sends the new message to the data pipeline;

[0133] 10. The data tracking SDK sends the debugging end time to the data management center to end the process.

[0134] As can be seen from the above embodiments, the SDK of this application has the following functions:

[0135] a. The logic for collecting debugging data and the actual time;

[0136] b. Extract the information transmitted from upstream and determine whether there is debug data in the message based on the logic of debug data;

[0137] c. Based on the actual time, construct the debugging start time and debugging end time, and report them to the data management center;

[0138] d. Construct the message to be output downstream.

[0139] It should be noted that the data processing system does not necessarily receive all messages containing debugging data from the data pipeline. Therefore, before step 1 above, it is necessary to determine whether the message contains debugging data. If so, proceed with the subsequent steps.

[0140] Please see Figure 10 The figure illustrates a flowchart of a data processing method on the data management center side according to an embodiment of this application, as shown in the figure, including:

[0141] S401. Receive a log query request sent by the receiving device, the log query request including the unique identifier of the device;

[0142] S402. Locate and return at least one debug log and debug sequence information corresponding to the unique identifier of the device;

[0143] The debugging order information is generated based on the routing information of each data processing system involved in processing the debugging log. The routing information includes the actual debugging order of each data processing system, the system identifier, and the log identifier of the debugging log.

[0144] The data management center in this application embodiment can store the unique identifier of the device that initiated the debugging, the log identifier of the debugging log, and the corresponding debugging sequence information for each debugging session. Thus, when a log query request is received, the center can find and return the log identifier of the debugging log and the corresponding debugging sequence information based on the unique identifier of the device, making it easier for the debugger to quickly find the abnormal data processing system.

[0145] Based on the above embodiments, as an optional embodiment, the log query request sent by the receiving device further includes:

[0146] Receive the routing information sent by the data processing system;

[0147] Update the debugging order information corresponding to the debugging log based on the routing information.

[0148] The data structure of routing information can be represented as follows:<MessageID,NodeID,LT> In this context, the MessageID field represents the log identifier of the debug log, the NodeID field represents the system identifier of the corresponding data processing system, and LT is an abbreviation for LogicTime, which in this embodiment refers to the actual debugging order of the corresponding data processing system. In an optional embodiment, the routing information may also include the system identifier of the previous data processing system, represented as PreNodeID.

[0149] Since the routing information records the actual debugging order and system identifiers of each data processing system related to the debugging logs, the debugging order information can be continuously updated based on this information. Specifically, the debugging order information in this embodiment may include the following three tables:

[0150] The message table, whose data structure can be represented as follows:<MessageID,DeviceID,Log> The DeviceID field represents the unique identifier of the device, and the Log field is used to record debug logs.

[0151] The data structure of the routing node table can be represented as follows:<MessageID,NodeID,LT> If the routing information further includes the debug start time and debug end time, it can be represented as:<MessageID,NodeID,LT,StartTime,EndTime> When some entries in the routing node table do not have an EndTime, there are three possibilities: the node may be the end node, an error has occurred, or the route has not ended.

[0152] The data structure of the routing edge table can be represented as follows:<MessageID,StartNodeID,EndNodeID,LT> Here, StartNodeID and EndNodeID represent two data processing systems that have a sequential relationship in the debugging order.

[0153] For example, Tables 1 to 3 are the message table, routing node table, and routing edge table of this application embodiment, respectively:

[0154] Rz031 827439 Log1 Rz412 811321 Log2

[0155] Table 1 Message Table of Embodiments of this Application

[0156]

[0157]

[0158] Table 2 Routing node table of this application embodiment

[0159] Rz031 Xt00031 Xt00012 1 Rz031 Xt00012 Xt00044 2 Rz031 Xt00044 Xt00031 3 Rz412 Xt00015 Xt00001 1

[0160] Table 3 Routing edge table of this application embodiment

[0161] The data processing center can generate two debugging sequence information based on the log identifiers of the two debugging logs recorded in Tables 1 to 3 and the actual debugging sequence of the corresponding data processing system. Debugging sequence information 1 records the debugging sequence information of debugging log Rz031: (1)Xt00031→(2)Xt00012→(3)Xt00044→(4)Xt00031; Debugging sequence information 2 records the debugging sequence information of debugging log Rz412: (1)Xt00015→(2)Xt00001.

[0162] When a new debug log is generated, a new row is created in the message table. This row records the log identifier, the unique identifier of the device that generated the debug log, and the specific content of the debug log. When a data processing system generates new debug data and routing information, the data management center updates the routing node table and routing edge table based on the routing information.

[0163] Based on the above embodiments, as an optional embodiment, the data management center of this application embodiment is also used for:

[0164] The device receives a topology map acquisition request generated in response to an operation instruction on the debugging sequence information, wherein the topology map acquisition request includes a unique identifier of the debugging log.

[0165] A topology graph is generated based on the debugging sequence information corresponding to the unique identifier of the debugging log. The topology graph includes a first node and a first directed edge.

[0166] Return the topology map to the device.

[0167] Specifically, since the debugging sequence information records the actual debugging order and system identifier of each data processing system, the data processing system participating in processing the debugging log can be taken as the first node, and a first directed edge can be connected between two nodes that are adjacent in the actual debugging order to obtain a topology graph. Furthermore, embodiments of this application can also generate a topology graph based on a routing node table or a routing edge table, that is, the nodes recorded in the routing node table are taken as the first nodes in the topology graph, and a first directed edge is added to the corresponding nodes in the topology graph based on the StartNodeID and EndNodeID recorded in the routing edge table. In addition, incoming or outgoing edges of nodes can be constructed according to the order of the debugging start time and debugging end time of the routing information. For example, if the debugging end time in routing information a is closest to the debugging start time in routing information b, then the outgoing edge of the node corresponding to the data processing system in routing information a points to the node corresponding to the data processing system in routing information b.

[0168] Taking Tables 1-3 above as examples, if the device wants to view the topology of the debug log with file identifier Rz031, the topology acquisition request includes the unique identifier Rz031 of the debug log. The data management center queries all rows of data with MessageID Rz031 in Table 2, and then generates the topology based on the NodeID and LT data in each row. Alternatively, it can query all rows of data with MessageID Rz031 in Table 3 and generate the topology based on StartNodeID and EndNodeID.

[0169] Based on the above embodiments, the embodiments of this application can also use OpenTracing related technologies to track the call chain triggered by data, and then use this to track the data and construct a topology graph. It is understood that this requires that all existing system participants have microservice access to OpenTracing technology.

[0170] Based on the above embodiments, as an optional embodiment, the topology graph also includes a second node and a second directed edge;

[0171] The second node represents a data processing system that did not participate in processing the debug logs as planned; the second directed edge connects two adjacent nodes in the target debug order; the target debug order is a debug order that is different from the preset debug order.

[0172] The data management center in this embodiment can also obtain the planned data processing systems participating in the debug logs and the preset debug order of each data processing system, and establish a predicted route node table and a predicted route edge table. As the names suggest, the predicted route node table records the system identifiers of the planned data processing systems participating in the debug logs and their preset debug order, while the predicted route edge table records the system identifiers of adjacent data processing systems in the preset debug order of the debug logs. By comparing whether the NodeID corresponding to the same LT in the predicted route node table and the route node table is the same, the existence of a second node can be quickly determined.

[0173] Please see Figure 11 The figure illustrates an exemplary system architecture diagram of another embodiment of this application. As shown, the system includes a device, a data management center front end, a data management center, a data pipeline, and a data processing system.

[0174] The data processing method starts from the device, reporting debug logs to the data pipeline. The data pipeline then sends the debug logs to the data processing system, and the data processing system sends the generated debug data and routing information back to the data pipeline for further transmission to downstream data processing systems. The data processing system also sends the routing information to the routing reporting service of the data management center. The routing reporting service in this embodiment can receive routing information from the data processing system. This service can be a standalone service or a log server; the key is the reliability of the service and data storage. Received logs are generally stored in the data pipeline.

[0175] The data management center records the routing information of each data processing system through the routing calculation service, and generates the debugging sequence information of the corresponding debugging logs, which is stored in the database. Depending on the amount of data, different databases can be selected in this embodiment, such as relational databases, graph databases, etc.

[0176] The data management center also provides a routing query service, offering detailed information about the topology of debug logs to the data management center's front-end or other external systems. It primarily provides two query interfaces:

[0177] 1. Query the debug log based on conditions such as device number and field values;

[0178] 2. Query the topology diagram based on the log identifiers in the debug logs;

[0179] The typical interaction pattern is as follows: the frontend provides the device ID and initiates a log query request to the API service. The API service reads information from the database and returns a list of logs to the frontend. Then, based on the user's selection of a log entry, the frontend continues to query the API service for the routing network of the selected log entry. The API service retrieves the vertex and edge information stored in the database based on the MessageID and returns it to the frontend. The frontend's directed graph component then renders the graph and presents it to the user.

[0180] Based on the above embodiments, as an optional embodiment, the management center front-end of this application embodiment requests a list of device identifiers from the data management center, sends the list of device identifiers to the device, and the device presents it to the debugger. The debugger selects a device identifier, or according to other conditions, such as certain field values ​​in the debug log, the device responds to the information selected by the debugger to construct a log query request and sends it to the data management center to request the corresponding debug log (which may be debug logs sent in the past or in real time by the device corresponding to the device identifier, or debug logs that match the field values. This application embodiment does not specifically limit the field values ​​that can be searched in the debug log, and can be set according to the actual application scenario).

[0181] The data management center returns a log list in reverse chronological order. In response to the debugger selecting a specific debug log, the device requests the debug order information and corresponding topology diagram of that debug log from the data management center's front-end. The topology diagram includes a first node and a first directed edge. The first node represents the data processing system involved in processing the debug log, and the first directed edge connects two nodes that are adjacent in the actual debug order, indicating the temporal relationship of data passing through the processing nodes. Routing information such as debug start time and debug end time can be marked on the first node.

[0182] Please see Figure 12 The figure illustrates an exemplary diagram of the interaction between various services in the data management center according to an embodiment of this application.

[0183] 1. The routing reporting service receives routing information (referred to as routing messages in the diagram) sent by the data processing system and sends the routing information to the routing calculation service;

[0184] 2. The routing calculation service updates the NodeID in the routing node table based on the system identifier of the data processing system recorded in the routing information;

[0185] 3. The routing calculation service updates the StartNodeID and EndNodeID in the routing edge table and records the updated routing node table and routing edge table in the database;

[0186] 4. When the query service receives a log query request or topology map retrieval request from the device, it retrieves relevant information from the database to generate debugging sequence information or a topology map and returns it to the device.

[0187] This application provides an apparatus, such as... Figure 13 As shown, the device may include: a log query module 201 and a log display module 202, specifically:

[0188] The log query module 201 is used to send a log query request to the data management center to instruct the data management center to return at least one debug log and the corresponding debug order information;

[0189] The log display module 202 is used to display the received at least one debug log and the corresponding debug order information;

[0190] The debugging order information is generated based on the routing information of each data processing system involved in processing the debugging log. The routing information includes the actual debugging order of each data processing system, the system identifier, and the log identifier of the debugging log.

[0191] The apparatus provided in this application specifically executes the process described in the above method embodiments. For details, please refer to the embodiments of the data processing method on the device side, which will not be repeated here. The apparatus provided in this application, by sending a log query request to the data management center, instructs the data management center to return at least one debug log and corresponding debug order information, and displays the received at least one debug log and corresponding debug order information. This helps debuggers quickly obtain debug logs and debug order information, laying the foundation for subsequent users to quickly find abnormal data processing systems from the debug order information.

[0192] This application provides a data processing system, such as... Figure 14 As shown, the data processing system may include: a message acquisition module 301, a message processing module 302, and a message sending module 303, specifically:

[0193] The message acquisition module 301 is used to acquire messages, which include debugging logs sent by the device or debugging data and routing information from the previous data processing system;

[0194] Message processing module 302 is used to generate debugging data and routing information of the current data processing system based on the message;

[0195] The message sending module 303 is used to send the routing information of the current data processing system to the data management center, and to package the debugging data and routing information of the current data processing system into a new message and send it to the next data processing system.

[0196] The routing information includes the actual debugging sequence of the corresponding data processing system, the system identifier, and the log identifier of the debugging log.

[0197] The data processing system provided in this application embodiment specifically executes the process described in the above method embodiment. For details, please refer to the embodiment of the data processing method on the data processing system side; further details will not be repeated here. The data processing system provided in this application embodiment, by receiving messages and generating new messages based on those messages to send downstream, ensures that debugging tasks can continuously flow and execute across different data processing systems. It also ensures that the execution process of debugging tasks can be recorded, laying the foundation for subsequent querying of debugging records. More importantly, since the data management center does not receive debugging data, data security is guaranteed.

[0198] This application provides a data management center, such as... Figure 15 As shown, the data management center may include: a log query request receiving module 401 and a log return module 402, specifically:

[0199] The log query request receiving module 401 is used to receive a log query request sent by the device, wherein the log query request includes the unique identifier of the device;

[0200] Log return module 402 is used to find and return at least one debug log and debug sequence information corresponding to the unique identifier of the device;

[0201] The debugging order information is generated based on the routing information of each data processing system involved in processing the debugging log. The routing information includes the actual debugging order of each data processing system, the system identifier, and the log identifier of the debugging log.

[0202] The data management center provided in this application embodiment specifically executes the process described in the above method embodiment. For details, please refer to the content of the above data processing method embodiment on the data management center side, which will not be repeated here. The data management center provided in this application embodiment stores the unique identifier of the device that initiated the debugging, the log identifier of the debugging log, and the corresponding debugging sequence information for each debugging session. Thus, when a log query request is received, the unique identifier of the device is used to find and return the log identifier of the debugging log and the corresponding debugging sequence information, making it convenient for the debugger to quickly find the abnormal data processing system.

[0203] This application provides an electronic device comprising: a memory and a processor; at least one program stored in the memory, which, when executed by the processor, can achieve the following compared to the prior art: by sending a log query request to a data management center, the data management center is instructed to return at least one debug log and corresponding debug order information, and the received at least one debug log and corresponding debug order information are displayed, which can help debuggers quickly obtain debug logs and debug order information, laying the foundation for subsequent users to quickly find abnormal data processing systems from debug order information.

[0204] In one alternative embodiment, an electronic device is provided, such as Figure 16 As shown, Figure 16 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of this electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0205] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0206] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 16 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0207] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0208] The memory 4003 stores application code that executes the solution of this application, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0209] This application provides a computer-readable storage medium storing a computer program that, when run on a computer, enables the computer to execute the corresponding content described in the aforementioned method embodiments. Compared to existing technologies, by sending a log query request to a data management center, instructing the data management center to return at least one debug log and corresponding debug order information, and displaying the received at least one debug log and corresponding debug order information, debuggers can quickly obtain debug logs and debug order information, laying the foundation for subsequent users to quickly find abnormal data processing systems from the debug order information.

[0210] This application provides a computer program including computer instructions stored in a computer-readable storage medium. When a processor of a computer device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, causing the computer device to perform the content shown in the foregoing method embodiments. Compared with the prior art, by sending a log query request to a data management center, instructing the data management center to return at least one debug log and corresponding debug order information, and displaying the received at least one debug log and corresponding debug order information, debuggers can quickly obtain debug logs and debug order information, laying the foundation for subsequent users to quickly find abnormal data processing systems from debug order information.

[0211] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0212] The above are only some embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data processing method, characterized by, The method, applied to one of a plurality of data processing systems, includes: Obtain messages, which include debug logs sent by the device or debug data and routing information from the previous data processing system; Generate debugging data and routing information for the current data processing system based on the message; The routing information of the current data processing system is sent to the data management center, and the debugging data and routing information of the current data processing system are packaged into a new message and sent to the next data processing system. The routing information includes the actual debugging sequence of the corresponding data processing system, the system identifier, and the log identifier of the debugging log.

2. The data processing method according to claim 1, characterized in that, The routing information also includes the debugging start time and debugging end time of the current data processing system; The debugging start time is the time when the current data processing system determines the actual debugging sequence; the debugging end time is the time when the current data processing system generates debugging data.

3. The data processing method of claim 1, wherein, The step of generating debug data and routing information for the current data processing system based on the message includes: The system invokes a preset software development kit and generates the actual debugging sequence and debugging start time of the current data processing system based on the debugging log in the message or the routing information of the previous data processing system. The debug logs in the message or the debug data of the previous data processing system are processed to obtain the debug data of the current data processing system. The software development kit is invoked to generate a debugging end time, and the system identifier of the data processing system and the log identifier of the debugging log are obtained to generate the routing information of the current data processing system.

4. A data processing method, characterized by, Applied to a device, the method includes: Send a log query request to the data management center to instruct the data management center to return at least one debug log and the corresponding debug order information; Display the received at least one debug log and the corresponding debug order information; The debugging sequence information is generated based on the routing information of each data processing system participating in the processing of the debugging log. The routing information is generated by the corresponding data processing system according to the method described in claim 1. The routing information includes the actual debugging sequence of each data processing system, the system identifier, and the log identifier of the debugging log.

5. The data processing method according to claim 4, characterized in that, The description includes displaying at least one debug log and the corresponding debug order information, followed by: In response to the operation command on the debugging sequence information, obtain and display the topology diagram corresponding to the debugging sequence information; The topology graph includes a first node, a first directed edge, and the routing information of the first node; the first node represents a data processing system that participates in processing the debug logs, and the first directed edge connects two nodes that are adjacent in the actual debugging sequence.

6. The data processing method according to claim 5, characterized in that, The topology graph also includes at least one of a second node and a second directed edge; The second node represents a data processing system that did not participate in processing the debug logs as planned. The second directed edge is used to connect two adjacent nodes in the target debugging sequence; the target debugging sequence is a debugging sequence that is different from the preset debugging sequence.

7. The data processing method according to claim 5, characterized in that, The routing information also includes the debugging start time and debugging end time of the data processing system; The debugging start time is the time when the data processing system determines the actual debugging sequence; the debugging end time is the time when the data processing system generates the debugging data.

8. The data processing method of claim 5, wherein, The topology graph also marks at least one of the debugging time of the first node and the transmission time of the first directed edge; Wherein, the debugging duration of the first node is the time interval between the debugging start time and debugging end time of the corresponding data processing system; the transmission duration of the first directed edge is the time interval between the debugging end time of the data processing system corresponding to the head node of the directed edge and the debugging start time of the data processing system corresponding to the tail node of the directed edge.

9. A data processing method, characterized by, Applied to data management centers, including: The receiving device sends a log query request, the log query request including the unique identifier of the device; Find and return at least one debug log and debug sequence information corresponding to the unique identifier of the device; The debugging sequence information is generated based on the routing information of each data processing system participating in the processing of the debugging log. The routing information is generated by the corresponding data processing system according to the method described in claim 1. The routing information includes the actual debugging sequence of each data processing system, the system identifier, and the log identifier of the debugging log.

10. The data processing method according to claim 9, characterized in that, The log query request sent by the receiving device also includes, prior to: Receive the routing information sent by the data processing system; The debugging order information corresponding to the debugging log is updated according to the routing information; the debugging order information includes a routing node table and a routing edge table; the routing node table includes the system identifier and the log identifier of the debugging log; the routing edge table includes the log identifier of the debugging log and the system identifiers of two data processing systems that are adjacent in the actual debugging order.

11. An apparatus, comprising: include: The log query module is used to send a log query request to the data management center, instructing the data management center to return at least one debug log and the corresponding debug order information; The log display module is used to display the received at least one debug log and the corresponding debug order information; The debugging sequence information is generated based on the routing information of each data processing system participating in the processing of the debugging log. The routing information is generated by the corresponding data processing system according to the method described in claim 1. The routing information includes the actual debugging sequence of each data processing system, the system identifier, and the log identifier of the debugging log.

12. A data processing system, characterized by include: The message acquisition module is used to acquire messages, which include debugging logs sent by the device or debugging data and routing information from the previous data processing system; The message processing module is used to generate debug data and routing information for the current data processing system based on the message. The message sending module is used to send the routing information of the current data processing system to the data management center, and to package the debugging data and routing information of the current data processing system into a new message and send it to the next data processing system. The routing information includes the actual debugging sequence of the corresponding data processing system, the system identifier, and the log identifier of the debugging log.

13. A data management center characterized by, include: A log query request receiving module is used to receive log query requests sent by the device, wherein the log query request includes the unique identifier of the device; The log return module is used to find and return at least one debug log and debug sequence information corresponding to the unique identifier of the device; The debugging sequence information is generated based on the routing information of each data processing system participating in the processing of the debugging log. The routing information is generated by the corresponding data processing system according to the method described in claim 1. The routing information includes the actual debugging sequence of each data processing system, the system identifier, and the log identifier of the debugging log.

14. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the data processing method as described in any one of claims 1 to 10.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause the computer to perform the steps of the data processing method as described in any one of claims 1 to 10.

Images