A data processing flow debugging method, system and device
By introducing breakpoint events and read and write events in the data exchange area, using event listeners and debug controllers to build a breakpoint queue, the user interface displays data and sets filtering conditions, the low debugging efficiency of data processing flow in the existing technology is solved. The problem of this achieves instant viewing and efficient debugging.
Patent Information
- Application Number
- CN202311727809.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-14
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2043-12-14
AI Technical Summary
The existing data processing process debugging methods are inefficient and require multiple executions of data processing processes to modify filter conditions, resulting in cumbersome debugging process.
By introducing breakpoint events and read and write events in the data exchange area, and using event listeners and debug controllers to build a breakpoint queue, the user interface can display data according to the breakpoint location and set filtering conditions to reduce the number of debugging times.
It realizes instant viewing of breakpoint position data without re-executing the data processing flow, improving debugging efficiency and reducing the number of debugging times.
Smart Images

Figure CN118312408B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing flow debugging, and in particular to a data processing flow debugging method, system and equipment. Background Art
[0002] In the data audit scenario, data audit rules can usually be converted into data processing flows. The data processing flows need to be debugged to check whether the parameter configurations of each data processing step in the data processing flow are correct and whether the results of data processing are consistent with the expected data audit results.
[0003] Data processing software, typically ETL software such as Pantaho Kettle, usually uses directed acyclic graphs to describe data processing flows. Verifying whether the data processing logic is correct and troubleshooting anomalies in the data processing process all require debugging capabilities for the data processing flow. The debugging function provided by Pantaho Kettle is implemented by adding data filtering conditions to the nodes of the directed acyclic graph. If data that meets the conditions appears, the execution of the data processing flow is interrupted. This debugging method of adding data filtering conditions is to add breakpoints to the data processing steps, set the filtering conditions in advance, and interrupt when the data filtering conditions are met. The filtering conditions cannot be modified. If the filtering conditions are unreasonable, the data processing flow can only be re-executed and re-debugged. This is done multiple times, resulting in low debugging efficiency. Summary of the invention
[0004] The purpose of the present invention is to provide a data processing flow debugging method, system and device to solve the problem of low debugging efficiency.
[0005] To achieve the above object, the present invention provides the following solutions:
[0006] A data processing flow debugging method, comprising:
[0007] Using a predecessor data processing thread to write data into a data exchange area, and using a successor data processing thread to read the data in the data exchange area;
[0008] When the data in the data exchange area changes, a breakpoint event and a read / write event are generated;
[0009] Using an event listener to listen to the breakpoint event and the read / write event, and sending a display read count and write count instruction to a user interface, and sending an add breakpoint instruction to a debug controller;
[0010] Using a debug controller to construct a breakpoint queue according to the breakpoint instruction and send an operation instruction to the data exchange area according to a control instruction sent by a user interface; the control instruction includes a continue instruction, a stop instruction and a preview instruction; the operation instruction includes an instruction to allow increase of a read count, an instruction to set a data exchange area status and an instruction to view data;
[0011] Using a breakpoint monitor to monitor the breakpoints in the breakpoint queue through the debug controller, and sending a display interrupt status instruction to a user interface;
[0012] The user interface is used to display the data at the breakpoint position according to the preview instruction, and the data filtering conditions are set to provide auxiliary means for debugging to check whether the data meets expectations.
[0013] Optionally, using a predecessor data processing thread to input data into a data exchange area, and using a subsequent data processing thread to read the data in the data exchange area, specifically includes:
[0014] Determining whether the data exchange area is in a discarded state;
[0015] If yes, terminate the operation of the data processing thread; the data processing thread includes a predecessor data processing thread and a successor data processing thread;
[0016] If not, the operation of the data processing thread is executed according to the predefined function.
[0017] Optionally, the operations in the data exchange area specifically include:
[0018] When the predecessor data processing thread writes data to the data exchange area, if the write count is equal to the read count, the predecessor data processing thread is instructed to write data to the data exchange area, and a read / write event and a breakpoint event are triggered;
[0019] When the subsequent data processing thread reads the data in the data exchange area, if the read count is less than the allowed read count, the subsequent data processing thread is instructed to read the data in the data exchange area, the read count is increased by 1, and a read-write event is triggered;
[0020] When the data exchange area receives the increase allowed read count instruction from the debug controller, the allowed read count of the data exchange area currently in the interrupt state is increased by 1;
[0021] When the data exchange area receives the instruction of setting the exchange area state from the debug controller, the data exchange area state is set to "abandoned" state, and the execution of all data processing threads is terminated;
[0022] When the data exchange area receives the data viewing instruction from the debug controller, the current data of the data exchange area is returned.
[0023] Optionally, the operations in the debugging controller specifically include:
[0024] When receiving the add breakpoint instruction of the event listener, generating a breakpoint based on the data exchange area identifier and inserting it to the top of the breakpoint queue;
[0025] When receiving a check breakpoint request from the breakpoint monitor, checking the breakpoint at the bottom of the breakpoint queue, and returning a data exchange area identifier; one data exchange area identifier corresponds to one data exchange area;
[0026] When receiving a continue instruction from the user interface, taking a breakpoint from the bottom of the breakpoint queue, and sending an increase allowed read count instruction to the corresponding data exchange area according to the data exchange area identifier;
[0027] When receiving a termination instruction from the user interface, sending a setting exchange area state instruction to all exchange areas in the data processing flow;
[0028] When receiving a preview instruction from the user interface, the breakpoint at the bottom of the breakpoint queue is checked, and a view data instruction is sent to the corresponding data exchange area according to the data exchange area identifier, and the data obtained from the data exchange area is returned to the user interface.
[0029] Optionally, the operations in the user interface specifically include:
[0030] receiving a display read count and write count instruction from the event listener, and displaying the current number of times data is written and read at the middle position of each edge on a directed acyclic graph representing a data processing flow;
[0031] Receive the interruption status display instruction of the breakpoint monitor, cancel the currently displayed interruption status, search for the directed edge in the directed acyclic graph according to the data exchange area identifier, and display a pause identifier in the middle position of the directed edge to identify the current interruption position.
[0032] Optionally, the operations in the event listener specifically include:
[0033] When a read or write event of the data exchange area is monitored, a display read count and write count instruction is sent to the user interface; the display read count and write count instruction includes a data exchange area identifier, a read count and a write count;
[0034] When a breakpoint event of the data exchange area is monitored, an add breakpoint instruction is sent to the debug controller; the add breakpoint instruction includes a data exchange area identifier.
[0035] Optionally, the breakpoint monitor sends a check breakpoint request to the debug controller at fixed time intervals, and sends a display interrupt status instruction to the user interface according to the returned data exchange area identifier; the display interrupt status instruction message includes the data exchange area identifier.
[0036] A data processing flow debugging system, comprising:
[0037] The predecessor data processing thread is used to write data into the data exchange area, and the subsequent data processing thread reads the data in the data exchange area;
[0038] A data exchange area, used to generate a breakpoint event and a read / write event when data in the data exchange area changes;
[0039] An event listener, used for monitoring the breakpoint event and the read / write event, and sending a display read count and write count instruction to the user interface, and sending an add breakpoint instruction to the debug controller;
[0040] A debug controller, configured to construct a breakpoint queue according to the breakpoint instruction and send an operation instruction to the data exchange area according to a control instruction sent by a user interface; the control instruction includes a continue instruction, a stop instruction, and a preview instruction; the operation instruction includes an instruction to allow increase of a read count, an instruction to set a data exchange area status, and an instruction to view data;
[0041] A breakpoint monitor, configured to monitor the breakpoints in the breakpoint queue through the debug controller and send a display interrupt status instruction to a user interface;
[0042] The user interface is used to display the data at the breakpoint position according to the preview instruction, set data filtering conditions, and provide auxiliary means for debugging to check whether the data meets expectations.
[0043] An electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned data processing flow debugging method.
[0044] A computer-readable storage medium stores a computer program, which implements the above-mentioned data processing flow debugging method when executed by a processor.
[0045] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: in the embodiments of the present invention, when the data in the data exchange area changes, a breakpoint event and a read-write event are generated, and the data at the breakpoint position is viewed through an event listener, a regulation controller, a breakpoint monitor, and a user interface, the data at the breakpoint position is displayed, and data filtering conditions are set to determine whether the data in the data exchange area is executed correctly, and auxiliary means debugging is provided to check whether the data meets expectations. The data at the breakpoint position can be known immediately without re-executing the data processing flow, thereby reducing the number of debugging times and improving debugging efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0047] Figure 1 This is a schematic diagram of the principle of the data processing flow debugging method provided by the present invention;
[0048] Figure 2 A schematic diagram of a directed acyclic graph of the data processing flow provided by the present invention;
[0049] Figure 3 Add a breakpoint schematic diagram to the data flow directed acyclic graph provided by the present invention;
[0050] Figure 4 This is a schematic diagram of the debugging state of the data flow directed acyclic graph provided by the present invention. DETAILED DESCRIPTION
[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0052] The purpose of the present invention is to provide a data processing flow debugging method, system and device, which can improve debugging efficiency.
[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0054] Embodiment 1
[0055] like Figure 1 As shown, the data processing process includes data processing threads (including data extraction, conversion, loading, etc.) and a data exchange area. Asynchronous data exchange is achieved between data processing threads through the data exchange area, that is, the predecessor data processing thread writes data to the data exchange area, and the subsequent data processing thread reads data from the data exchange area.
[0056] A directed acyclic graph can be used to represent the data processing flow, wherein the nodes in the directed acyclic graph correspond to data processing threads, predecessor nodes and successor nodes are distinguished according to the data exchange relationship between the data processing threads, directed edges from predecessor nodes to successor nodes correspond to data exchange areas, nodes and their corresponding data processing threads have the same globally unique identifier, and directed edges and their corresponding data exchange areas have the same globally unique identifier.
[0057] The debugging module includes a debugging controller, a user interface, an event listener and a breakpoint monitor, wherein the debugging controller maintains a breakpoint queue, which is a first-in-first-out (FIFO) queue, and implements debugging state control according to the debugging control instructions of the user interface; the user interface includes a debugging state panel and a debugging control panel, the debugging state panel displays the debugging state, and the debugging control panel controls the debugging state by sending control instructions to the debugging controller; the event listener monitors events in the data exchange area in the data processing flow and notifies the debugging controller and the user interface; the breakpoint monitor notifies the user interface to display the interruption state by intermittently querying the debug controller for breakpoints in the breakpoint queue.
[0058] The present invention provides a data processing flow debugging method, comprising:
[0059] The data is written into the data exchange area by the predecessor data processing thread, and the data in the data exchange area is read by the successor data processing thread.
[0060] When the data in the data exchange area changes, a breakpoint event and a read / write event are generated.
[0061] The event listener is used to listen to the breakpoint event and the read / write event, and a display read count and write count instruction is sent to the user interface, and an add breakpoint instruction is sent to the debug controller.
[0062] A debug controller is used to construct a breakpoint queue according to the breakpoint instruction and to send an operation instruction to the data exchange area according to the control instruction sent by the user interface; the control instruction includes a continue instruction, a termination instruction and a preview instruction; the operation instruction includes an instruction to allow increase of the read count, an instruction to set the data exchange area status and an instruction to view data.
[0063] The breakpoints in the breakpoint queue are monitored by the debug controller using a breakpoint monitor, and a display interrupt status instruction is sent to a user interface.
[0064] The user interface is used to display the data at the breakpoint position according to the preview instruction, and the data filtering conditions are set to provide auxiliary means for debugging to check whether the data meets expectations.
[0065] In practical applications, 1. The data processing flow includes data processing threads and data exchange areas.
[0066] The data processing threads are divided into predecessor data processing threads and successor data processing threads. One data processing thread can be both a predecessor data processing thread and a successor data processing thread.
[0067] (1) The predecessor data processing thread writes data to the data exchange area. If the data exchange area is in the "discarded" state, the data processing thread is terminated; if the data exchange area is not in the "discarded" state, an attempt is made to write data to the data exchange area until the writing is successful.
[0068] (2) The subsequent data processing thread reads data from the data exchange area. If the data exchange area is in the "discarded" state, the data processing thread is terminated; if the data exchange area is not in the "discarded" state, an attempt is made to read data from the data exchange area until the reading is successful.
[0069] (3) The data processing thread reads data from the input data exchange area, processes the data according to the predefined data processing logic, and writes the processing result data into the output data exchange area.
[0070] 2. The data processing flow includes data processing threads and data exchange areas.
[0071] The data exchange area maintains three variables: "write count", "read count" and "allowed read count", which are used to debug the data processing flow. The specific process includes:
[0072] (1) When the front data processing thread needs to write data to the swap area, if the "write count" is equal to the "read count", the data is written to the swap area, the "write count" is increased by 1, and the read / write event and breakpoint event are triggered. Otherwise, wait and try again.
[0073] (2) When the subsequent data processing thread needs to read data from the swap area, if the "read count" is less than the "allowed read count", the data is read, the "read count" is increased by 1, and a read / write event is triggered. Otherwise, wait and retry. At this time, the process branch is in an interrupted state and the data in the swap area can be viewed.
[0074] (3) When receiving the "increase allowed read count" instruction from the debug controller, add 1 to the "allowed read count" of the swap area that is currently in the interrupt state.
[0075] (4) When receiving the "set swap area status" instruction from the debug controller, the swap area is set to the "discarded" state and all data processing threads terminate execution.
[0076] (5) When receiving the "view data" command from the debug controller, return the current data.
[0077] 3. The debug controller maintains a breakpoint FIFO queue, and the breakpoint contains the data exchange area identifier.
[0078] (1) When the event listener "add breakpoint" instruction is received, a breakpoint is generated based on the data exchange area identifier and inserted at the top of the breakpoint queue.
[0079] (2) When a "check breakpoint" request is received from the breakpoint monitor, the breakpoint at the bottom of the breakpoint queue is checked and the data exchange area identifier is returned.
[0080] (3) When a "continue" instruction is received from the user interface, a breakpoint is taken from the lower end of the breakpoint queue, and an "increase allowed read count" instruction is sent to the corresponding data exchange area according to the data exchange area identifier.
[0081] (4) When receiving the "Terminate" command from the user interface, send a "Set Swap Area Status" command to all swap areas in the data flow.
[0082] (5) When receiving the "preview" command from the user interface, check the breakpoint at the bottom of the breakpoint queue, and send a "view data" command to the corresponding data exchange area according to the data exchange area identifier, and return the data obtained from the data exchange area to the user interface.
[0083] 4. The user interface includes a debug status panel.
[0084] (1) The end of the edge connection writing atomic operation indicates whether a breakpoint is set by a hollow circle or a solid circle, where a hollow circle indicates that a breakpoint is not set and a solid circle indicates that a breakpoint is set.
[0085] (2) After debugging starts, the "display read count, write count" command of the receiving event listener is displayed in the middle of each edge to display the current number of times data is written and read.
[0086] (3) After debugging starts, the "display interrupt status" instruction of the breakpoint monitor is received, the currently displayed interrupt status is canceled, the directed edge in the directed acyclic graph is searched according to the data exchange area identifier, and a pause identifier is displayed in the middle of the directed edge to identify the current interrupt position.
[0087] 5. The debugging control panel includes the following buttons.
[0088] (1) Click the “Add Breakpoint” button to add a breakpoint to the currently selected directed edge in the directed acyclic graph representing the data processing flow, so that a solid circle appears at the end of the edge connected to the atomic operation being written.
[0089] (2) The “Cancel Breakpoint” button cancels the breakpoint of the currently selected directed edge in the directed acyclic graph of the data processing flow, so that a hollow circle is displayed on the end connected to the atomic operation being written.
[0090] (3) Click the “Start” button to submit the data processing flow defined by the current directed acyclic graph for debugging and execution.
[0091] (4) "Continue" button sends a "Continue" command to the debug controller.
[0092] (5) The "Preview" button sends a "Preview" command to the debug controller and displays the returned data through the data preview window. The data preview window provides query functions to support exploratory analysis of the preview data.
[0093] (6) Click the “Terminate” button to send a “Terminate” command to the debugging controller, and the debugging ends.
[0094] 6. The operations in the event listener include:
[0095] (1) When a data exchange area read or write data event is monitored, a "display read count, write count" message is sent to the user interface, which includes information such as the data exchange area identifier, read count, and write count.
[0096] (2) When a breakpoint event in the data exchange area is monitored, an "add breakpoint" message is sent to the debug controller, and the message includes the data exchange area identification information.
[0097] 7. The breakpoint monitor sends a "check breakpoint" request to the debug controller at fixed time intervals, and sends a "display interrupt status" message to the user interface based on the returned data exchange area identifier, which includes the data exchange area identifier.
[0098] like Figure 2 The data processing flow directed acyclic graph shown in the figure includes three nodes: "Database Extraction", "Dummy Operation" and "CSV Data Loading", which respectively represent querying data from the database, forwarding data, and writing data into a CSV file. The directed edges between "Database Extraction" and "Dummy Operation" and between "Dummy Operation" and "CSV Data Loading" respectively represent that "Database Extraction" queries data from the database and hands the data to "Dummy Operation", and "Dummy Operation" forwards the data to "CSV Data Loading".
[0099] Add a breakpoint on the directed edge between "Database Extraction" and "Dummy Operation", such as Figure 3 shown.
[0100] When the data processing flow is submitted for debugging and execution, three data processing threads will be created according to the node definition, and two data exchange areas will be created according to the definition of the directed edge. The number of data exchanges is displayed in real time on the directed edge. Whenever the "database extraction" data processing thread writes a number to the exchange area, a breakpoint will be triggered, thereby interrupting the execution of the data processing flow and displaying the interruption status on the directed acyclic graph, such as Figure 4 As shown, you can click the "Preview" button to preview the data in the exchange area.
[0101] The present invention adds breakpoints on the edges of a directed acyclic graph. Whenever new data is generated, the execution of the data processing flow is interrupted to generate breakpoints, and adaptive data filtering conditions are set as required at the breakpoint positions to determine whether the data in the data exchange area is executed correctly, thereby reducing the number of debugging times. The present invention supports previewing and exploratory queries of newly generated data.
[0102] In addition, the present invention adopts a breakpoint method commonly used in software development, so the debugging information is richer, the debugging logic is clearer, and it is more user-friendly.
[0103] Embodiment 2
[0104] In order to execute the method corresponding to the above-mentioned embodiment 1 to achieve corresponding functions and technical effects, a data processing flow debugging system is provided below.
[0105] A data processing flow debugging system, comprising:
[0106] The predecessor data processing thread is used to write data into the data exchange area, and the subsequent data processing thread reads the data in the data exchange area.
[0107] The data exchange area is used to generate a breakpoint event and a read / write event when the data in the data exchange area changes.
[0108] The event listener is used to listen to the breakpoint event and the read / write event, and send a display read count and write count instruction to the user interface, and send an add breakpoint instruction to the debug controller.
[0109] The debugging controller is used to build a breakpoint queue according to the breakpoint instruction and send an operation instruction to the data exchange area according to the control instruction sent by the user interface; the control instruction includes a continue instruction, a termination instruction and a preview instruction; the operation instruction includes an instruction to allow the read count to be increased, an instruction to set the data exchange area status and an instruction to view data.
[0110] The breakpoint monitor is used to monitor the breakpoints in the breakpoint queue through the debug controller and send a display interrupt status instruction to the user interface.
[0111] The user interface is used to display the data at the breakpoint position according to the preview instruction, set data filtering conditions, and provide auxiliary means for debugging to check whether the data meets expectations.
[0112] Embodiment 3
[0113] An electronic device includes a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned data processing flow debugging method.
[0114] A computer-readable storage medium stores a computer program, which implements the above-mentioned data processing flow debugging method when executed by a processor.
[0115] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0116] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of the present invention. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A data processing flow debugging method, characterized in that: include: Using a predecessor data processing thread to write data into a data exchange area, and using a successor data processing thread to read the data in the data exchange area; When the data in the data exchange area changes, a breakpoint event and a read / write event are generated; Using an event listener to listen to the breakpoint event and the read / write event, and sending a display read count and write count instruction to a user interface, and sending an add breakpoint instruction to a debug controller; Using the debug controller to construct a breakpoint queue according to the breakpoint instruction and send an operation instruction to the data exchange area according to the control instruction sent by the user interface; the control instruction includes a continue instruction, a termination instruction and a preview instruction; the operation instruction includes an increase allowed read count instruction, a data exchange area status instruction and a data viewing instruction; The operations in the data exchange area specifically include: When the predecessor data processing thread writes data to the data exchange area, if the write count is equal to the read count, the predecessor data processing thread is instructed to write data to the data exchange area, and a read / write event and a breakpoint event are triggered; When the subsequent data processing thread reads the data in the data exchange area, if the read count is less than the allowed read count, the subsequent data processing thread is instructed to read the data in the data exchange area, the read count is increased by 1, and a read-write event is triggered; When the data exchange area receives the increase allowed read count instruction from the debug controller, the allowed read count of the data exchange area currently in the interrupt state is increased by 1; When the data exchange area receives the instruction of setting the exchange area state from the debug controller, the data exchange area state is set to "abandoned" state, and the execution of all data processing threads is terminated; When the data exchange area receives the data viewing instruction from the debug controller, the current data of the data exchange area is returned; The operations in the debugging controller specifically include: When receiving the add breakpoint instruction of the event listener, generating a breakpoint based on the data exchange area identifier and inserting it to the top of the breakpoint queue; When receiving a check breakpoint request from a breakpoint monitor, checking the breakpoint at the bottom of the breakpoint queue, and returning a data exchange area identifier; one data exchange area identifier corresponds to one data exchange area; When receiving a continue instruction from the user interface, taking a breakpoint from the bottom of the breakpoint queue, and sending an increase allowed read count instruction to the corresponding data exchange area according to the data exchange area identifier; When receiving a termination instruction from the user interface, sending a setting exchange area state instruction to all exchange areas in the data processing flow; When receiving a preview instruction from the user interface, checking the breakpoint at the bottom of the breakpoint queue, sending a viewing data instruction to the corresponding data exchange area according to the data exchange area identifier, and returning the data obtained from the data exchange area to the user interface; Using a breakpoint monitor to monitor breakpoints in the breakpoint queue through the debug controller, and sending a display interrupt status instruction to a user interface; causing the breakpoint monitor to send a check breakpoint request to the debug controller at a fixed time interval, and sending a display interrupt status instruction to the user interface according to the returned data exchange area identifier; the display interrupt status instruction message includes a data exchange area identifier; The user interface is used to display the data at the breakpoint position according to the preview instruction, and the data filtering conditions are set to provide auxiliary means for debugging to check whether the data meets expectations.
2. The data processing flow debugging method according to claim 1, characterized in that: Using a predecessor data processing thread to input data into a data exchange area, and using a successor data processing thread to read the data in the data exchange area, specifically includes: Determining whether the data exchange area is in a discarded state; If yes, terminate the operation of the data processing thread; the data processing thread includes a predecessor data processing thread and a successor data processing thread; If not, the operation of the data processing thread is executed according to the predefined function.
3. The data processing flow debugging method according to claim 1, characterized in that: The operations in the user interface specifically include: receiving a display read count and write count instruction from the event listener, and displaying the current number of times data is written and read at the middle position of each edge on a directed acyclic graph representing a data processing flow; Receive the interruption status display instruction of the breakpoint monitor, cancel the currently displayed interruption status, search for the directed edge in the directed acyclic graph according to the data exchange area identifier, and display a pause identifier in the middle position of the directed edge to identify the current interruption position.
4. The data processing flow debugging method according to claim 1, characterized in that: The operations in the event listener specifically include: When a read or write event of the data exchange area is monitored, a display read count and write count instruction is sent to the user interface; the display read count and write count instruction includes a data exchange area identifier, a read count and a write count; When a breakpoint event of the data exchange area is monitored, an add breakpoint instruction is sent to the debug controller; the add breakpoint instruction includes a data exchange area identifier.
5. A data processing flow debugging system, characterized in that: include: The predecessor data processing thread is used to write data into the data exchange area, and the subsequent data processing thread reads the data in the data exchange area; A data exchange area, used to generate a breakpoint event and a read / write event when data in the data exchange area changes; An event listener, used for monitoring the breakpoint event and the read / write event, and sending a display read count and write count instruction to the user interface, and sending an add breakpoint instruction to the debug controller; A debugging controller, configured to construct a breakpoint queue according to the breakpoint instruction and send an operation instruction to the data exchange area according to a control instruction sent by a user interface; the control instruction includes a continue instruction, a stop instruction, and a preview instruction; the operation instruction includes an increase allowed read count instruction, a data exchange area status instruction, and a data viewing instruction; The operations in the data exchange area specifically include: When the predecessor data processing thread writes data to the data exchange area, if the write count is equal to the read count, the predecessor data processing thread is instructed to write data to the data exchange area, and a read / write event and a breakpoint event are triggered; When the subsequent data processing thread reads the data in the data exchange area, if the read count is less than the allowed read count, the subsequent data processing thread is instructed to read the data in the data exchange area, the read count is increased by 1, and a read-write event is triggered; When the data exchange area receives the increase allowed read count instruction from the debug controller, the allowed read count of the data exchange area currently in the interrupt state is increased by 1; When the data exchange area receives the instruction of setting the exchange area state from the debug controller, the data exchange area state is set to "abandoned" state, and the execution of all data processing threads is terminated; When the data exchange area receives the data viewing instruction from the debug controller, the current data of the data exchange area is returned; The operations in the debugging controller specifically include: When receiving the add breakpoint instruction of the event listener, generating a breakpoint based on the data exchange area identifier and inserting it to the top of the breakpoint queue; When receiving a check breakpoint request from a breakpoint monitor, checking the breakpoint at the bottom of the breakpoint queue, and returning a data exchange area identifier; one data exchange area identifier corresponds to one data exchange area; When receiving a continue instruction from the user interface, taking a breakpoint from the bottom of the breakpoint queue, and sending an increase allowed read count instruction to the corresponding data exchange area according to the data exchange area identifier; When receiving a termination instruction from the user interface, sending a setting exchange area state instruction to all exchange areas in the data processing flow; When receiving a preview instruction from the user interface, checking the breakpoint at the bottom of the breakpoint queue, sending a viewing data instruction to the corresponding data exchange area according to the data exchange area identifier, and returning the data obtained from the data exchange area to the user interface; A breakpoint monitor, used to monitor the breakpoints in the breakpoint queue through the debug controller, and send a display interrupt status instruction to the user interface; the breakpoint monitor sends a check breakpoint request to the debug controller at a fixed time interval, and sends a display interrupt status instruction to the user interface according to the returned data exchange area identifier; the display interrupt status instruction message includes the data exchange area identifier; The user interface is used to display the data at the breakpoint position according to the preview instruction, set data filtering conditions, and provide auxiliary means for debugging to check whether the data meets expectations.
6. An electronic device, characterized in that: It comprises a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the data processing flow debugging method as described in any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the data processing flow debugging method as described in any one of claims 1 to 4.
