An intelligent monitoring system and method for data task processing progress state

Abstract

The present application relates to the technical field of task progress monitoring, in particular to a data task processing progress state intelligent monitoring system and method, which comprises obtaining task records and archiving task numbers in batches, arranging execution jump sections and node name generation stage sections, checking node behavior to mark idle state, tracking state field change frequency to extract jump stage, and classifying abnormal stage to generate progress deviation results. The present application builds a batch tracking structure of task number and attribution, completes task item division and group collection, extracts execution jump sections and frequent sections to form jump distribution set, extracts operation traces combined with node behavior and stage section and marks non-behavior nodes, generates change frequency and jump sequence according to stage classification state field, cross-compare state and behavior records to output fluctuation trajectory mark, and supports task jump structure analysis and state hierarchical marking.

Description

Technical Field

[0001] This invention relates to the field of task progress monitoring technology, and in particular to an intelligent monitoring system and method for monitoring the progress status of data task processing. Background Technology

[0002] The task progress monitoring technology field encompasses data processing and management technologies related to real-time status tracking, stage division, and progress feedback during task execution. Core components include time recording at each stage of task execution, segmented progress identification, extraction and classification of abnormal data, and dynamic evaluation of overall task completion. In specific applications, such as IoT card management systems and other business scenarios involving large-scale data processing, task progress monitoring technology needs to handle large-scale batch operations such as card resource transfer and expiration reminders. During the multiple execution stages, including data import, logical processing, and anomaly analysis, it needs to provide clear timeline markers, visualized progress output, abnormal data item location, and prediction of remaining task time. Currently, this technology field still faces challenges such as insufficient real-time feedback capabilities, low transparency of execution status information, and difficulty in troubleshooting task failure data, limiting the efficient operation of the system and the user experience.

[0003] The intelligent monitoring system for data task processing progress refers to a system architecture used to monitor the entire process of batch data task execution in the IoT card management system. It targets data task types including batch card resource transfer tasks, card expiration reminder generation tasks, and card status update batch processing tasks. By parsing user-uploaded task files, the system extracts task data entries and their field structures, marks the status changes of each data entry in the processing flow, records processing time nodes, and establishes a detailed processing status table for each data entry. Tasks are categorized and tracked based on processing time intervals, execution success rates, and anomaly types. Simultaneously, the remaining time for the current task is estimated based on historical task execution time models. Regarding anomaly handling, the system directly correlates anomaly results with the original data entries, marking specific fields and error types for subsequent targeted correction and re-import. The system achieves full-process monitoring of task execution status through refined data tracking, rule-driven time estimation, and error backtracking mechanisms.

[0004] Existing technologies lack structured labeling methods for task data attribution identification. Task batches and source information have not been archived in a unified system. Status records mainly rely on fixed result data, ignoring the distribution of time jump segments and the progression of stages in the execution sequence. There is no direct mapping mechanism between node execution records and task processing segments, making it difficult to accurately determine the actual behavior of nodes in each stage. Changes in status fields are not frequency-identified or stage-based. There is no stable classification and discrimination logic for status jump behavior, making it difficult to locate potential abnormal states in a timely manner and to clearly label task status deviations, thus affecting the efficiency of subsequent task optimization and anomaly correction. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and to propose an intelligent monitoring system and method for monitoring the progress status of data task processing.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: an intelligent monitoring system for the progress status of data task processing, the system comprising:

[0007] The task access monitoring module obtains the submitted task records, identifies the task number, data source, and submission order fields, groups all records and marks them with batch numbers, archives the task group tags and attribution information for each batch of tasks, and generates a batch tracking number set for task processing.

[0008] The execution interval segmentation module reads each task batch from the task processing batch tracking number set, marks the jump status of the task processing order, records the start and end times of the frequently jump fields, extracts the execution node names associated in the task and completes the binding, and generates a task time advancement stage segment set.

[0009] The node behavior verification module reads the phase boundary time and node mapping data of the task time advancement phase segment set, checks the interval between the processing time of the execution node and the phase time, determines whether there is operation coverage, marks the consecutive missing nodes as idle state, and generates a node phase activity comparison structure.

[0010] The state fluctuation identification module reads the active node number in the node stage active comparison structure, performs sequential tracking of state changes within a stage, marks recurring state entries, extracts the stage number with frequent jumps, and generates a stage state offset segment list.

[0011] As a further embodiment of the present invention, the task processing batch tracking number set includes task batch identification code, group label information, and archived task index; the task time advancement stage segment set specifically includes stage number sequence, time segment label, and node association mapping; the node stage activity comparison structure includes node activity status identifier, stage validity label, and idle node list; and the stage status offset segment list specifically includes status jump mark, stage number record, and frequency anomaly label.

[0012] As a further aspect of the present invention, the task access monitoring module includes:

[0013] The task data recognition submodule obtains submitted task records, identifies the task number field, data source field, and submission order field, establishes field matching rules according to field format standards, matches three types of field tags for each record, removes records with incomplete fields, establishes a field structured mapping set for the remaining records, and generates a field structured mapping matrix.

[0014] The submission order segmentation submodule, based on the field structured mapping matrix, calls the corresponding values ​​of the submission order field, constructs a record time sequence queue according to the timestamp interval, task number prefix change and data source switching flag, and segments the queue using the task submission continuity judgment condition to generate a task submission order segmentation mark sequence.

[0015] The task batch registration submodule segments and marks the sequence of tasks according to the submission order. It calls the task number field value and the data source field value to perform clustering and grouping processing on records under the same segment mark. It sets the group label coding rules and generates the label number of each group according to the mark sequence order. It establishes the mapping relationship between task number and label number and generates a batch tracking number set for task processing.

[0016] As a further aspect of the present invention, the execution interval segmentation module includes:

[0017] The timing organization submodule obtains each task batch from the task processing batch tracking number set, extracts the execution time point field value from the task record, constructs a time series list according to the order of the task number sequence, calculates the time difference between adjacent records and generates a time difference sequence, calls the continuous non-outlier segment in the time difference sequence, establishes a set of continuous execution time point segments, and generates a task execution time point segment group.

[0018] The jump segment identification submodule identifies the interruption point of the time difference sequence between adjacent segments based on the task execution time point segment group, extracts the task number before and after the jump, sets the jump interval judgment threshold to twice the average task processing interval value, aggregates and classifies the numbering status within the jump segment interval, counts the frequency of each type of jump segment and extracts the start and end times of each segment, and generates a jump segment time range set.

[0019] The task node binding submodule extracts the associated processing node name field value from the task entries based on the jump segment time range set, filters out the processing node name set corresponding to the records within the time range, establishes a node name and corresponding task number binding mapping table, constructs a set of task number, time range and processing node triplet, and generates a task time advancement stage segment set.

[0020] As a further aspect of the present invention, the node behavior verification module includes:

[0021] The node stage coverage submodule obtains the stage boundary time and processing node name in the task time advancement stage segment set, filters the processing time field and node name field in the execution node operation record, performs data association by comparing the node name, and filters out records with no corresponding relationship, integrates the associated processing time set by node dimension, and generates a node processing time list set.

[0022] The active marker determination submodule, based on the node processing time list set, calls the start and end times in the task time advancement stage segment set, performs a logical judgment on whether the processing time of each node and the time interval of each stage are covered, records the coverage flag, establishes a node marker mapping matrix according to the stage, and generates a node stage coverage identifier matrix.

[0023] The idle node classification submodule counts the number of times each node covers all stages based on the node stage coverage identifier matrix, sets the activity identification threshold as the upper limit of the stage coverage count being zero, classifies nodes below the threshold as unrecorded nodes, constructs a comparison structure between node name and stage status, and generates a node stage activity comparison structure.

[0024] As a further aspect of the present invention, the state fluctuation identification module includes:

[0025] The status classification submodule reads the active node number from the node stage active comparison structure, obtains the status field value and timestamp information from the task data record, finds the stage to which the task belongs based on the task number corresponding to the active node number, groups and classifies each status record according to the stage number, establishes the corresponding mapping relationship between status records and stage numbers, and generates a stage status value mapping table.

[0026] The continuous repetition labeling submodule extracts the state value sequence within each stage based on the stage state value mapping table, calls the state value results of adjacent stages, counts the number of times the state value appears in consecutive stages, determines whether the state value appears twice or more consecutively in adjacent stages, marks the repetition identification information of the state value in the corresponding stage, and generates a state cross-stage repetition labeling matrix.

[0027] The transition frequency extraction submodule filters out state value change records that are not marked as duplicates based on the state cross-stage repetition marking matrix, accumulates the number of transitions by stage number, sets the threshold for the number of transitions to twice the average state change of that stage, collects stage numbers that exceed the threshold, establishes a set of associated entries between stage number and number of transitions, and generates a list of stage state offset segments.

[0028] As a further aspect of the present invention, the system further includes:

[0029] The stage status monitoring output module reads the offset stage number from the stage status offset segment list, compares the stage number with the node record behavior, identifies abnormal status stages, integrates the performance status and annotation information of each stage, and generates task progress status deviation monitoring results.

[0030] The task progress status deviation monitoring results include stage status interruption categories, status change feature classifications, and node status comparison annotations.

[0031] As a further aspect of the present invention, the stage status monitoring output module includes:

[0032] The stage information extraction submodule reads the offset stage number from the stage status offset segment list, extracts the stage name field and boundary time field value of the corresponding stage in the task time advancement stage segment set, performs start and end time verification processing on the boundary time field value, constructs a data structure corresponding to the stage number, its name and boundary time, and generates an offset stage information mapping table.

[0033] The state behavior comparison submodule extracts the state field from the task record and the operation time sequence data from the node behavior record based on the offset stage information mapping table. It partitions and extracts the two types of data according to the stage number, calls the cross-distribution of the marked state value sequence and the node behavior record in the same time period, judges whether there are state interruptions, repetitions or concentrated mutations, and generates a stage state behavior abnormality marking matrix.

[0034] The state deviation summary submodule extracts the stage number and corresponding state feature item with abnormal markers based on the stage state behavior anomaly marker matrix. It archives the three types of features, namely state interruption, repetition and concentrated mutation, in the form of tags. It calls the stage name and boundary time field value in the offset stage information mapping table to combine and generate a four-tuple record set of stage number, name, boundary time and state offset tag, and generates the task progress state deviation monitoring result.

[0035] A method for intelligently monitoring the progress status of data task processing, wherein the method is executed based on the aforementioned intelligent monitoring system for the progress status of data task processing, and includes the following steps:

[0036] S1: Obtain the task number, data source and submission order fields from the task record, identify the task submission order, group the tasks according to the source field, establish a number mapping relationship for each group of tasks, mark the corresponding group label and belonging information, complete the task record archiving and organization, and generate a batch tracking number set for task processing.

[0037] S2: Based on the task processing batch tracking number set, organize the execution order information in the task record, identify the jump position in the execution order, mark the task number to which the jump segment belongs, collect the frequently jumped numbers as processing segments, organize the start and end times of the jump segments, extract the processing node name in the task entry and bind it with the task content, map the processing time period of each node in the task, divide the continuous advancement stage in the task processing process, and generate a task time advancement stage segment set;

[0038] S3: Based on the stage boundary information and node mapping data of the task time advancement stage segment set, read the processing time in the node operation record, compare the relationship between the behavior of each node and the stage time range, mark the node number that has behavior in each stage, classify and identify the nodes that have not participated in any stage, establish the activity status structure of the node corresponding to the stage, and generate the node stage activity comparison structure.

[0039] S4: Based on the active node number in the node stage active comparison structure, divide the status field content in the task data according to the stage, organize the change order of the status value in each stage, mark the stage where the status value fluctuates repeatedly and frequently, summarize the stage numbers of the abnormal jump concentration as the offset segment, and generate a stage status offset segment list.

[0040] S5: Based on the offset stage number in the stage state offset segment list, extract the name and start and end time of the corresponding stage, read the node behavior and status record information, compare the recorded status and node behavior within the stage, identify the interruption, repetition or abnormal fluctuation in the stage, complete the annotation and summary of various status behaviors, and generate the task progress status deviation monitoring result.

[0041] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0042] In this invention, by establishing a batch tracking structure with task numbers and data attribution, the item division and grouping of batch tasks are completed. Combined with the segment identification and frequent segment classification in the execution order, a set of progress rhythm and jump distribution in the task processing process is formed. Multi-stage operation traces are extracted from the node behavior records and the corresponding stage segments. The behavior coverage within the stage is defined and nodes without behavior are marked. The status field forms a change frequency distribution and jump sequence during the stage progression. A set of stage numbers with repeated state fluctuations is established. The behavior records and status labels are cross-compared at each stage, and the status trajectory identifiers such as concentrated fluctuations and sudden changes are output. This supports the sorting of jump structure, node behavior screening, and hierarchical labeling of status fluctuations in the task processing chain. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating the overall system flow of the present invention.

[0044] Figure 2 This is a flowchart of the system modules of the present invention;

[0045] Figure 3 This is a flowchart of the method of the present invention. Detailed Implementation

[0046] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0047] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0048] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0049] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0050] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0051] Please see Figure 1 This invention provides a technical solution: an intelligent monitoring system for the progress status of data task processing, the system comprising:

[0052] The task access monitoring module obtains the submitted task records, identifies the task number, data source, and submission order fields, divides all records sequentially and groups them in batches, archives and registers the grouped tasks with their corresponding numbers, marks each batch of tasks with grouping tags and attribution information, and generates a batch tracking number set for task processing.

[0053] The execution interval segmentation module reads each task batch from the task processing batch tracking number set, sorts the execution time points of the task records in order, marks the jump situation of the continuous execution order, classifies the execution number of the processing segment with frequent jumps, records the start and end time of each jump segment, extracts the processing node name associated with the task entry and binds it to the corresponding task, and generates a set of task time advancement stage segments.

[0054] The node behavior verification module reads the phase boundary time and node mapping data of the task time advancement phase segment set, performs a coverage check on the processing time and phase start and end time in the execution node operation record, marks whether each node has a valid record in the corresponding phase, and if there is no record in multiple phases, it is classified as an idle node, and generates a node phase activity comparison structure.

[0055] The status fluctuation identification module reads the active node number in the node stage active comparison structure, classifies the status field in the task data record by stage, performs sequential tracking of the frequency of status value changes within a stage, marks the repeated occurrence of status values ​​in consecutive stages, collects the stage number where the number of jumps exceeds the set standard, and generates a stage status offset segment list.

[0056] The phase status monitoring output module reads the offset phase number from the phase status offset segment list, extracts the name and boundary time of the corresponding phase, performs cross-comparison of the marked status and node behavior records within the phase, performs classification and labeling on phases with status interruptions, repetitions or concentrated changes, summarizes the performance status and record annotations of all phases, and generates the task progress status deviation monitoring results.

[0057] The task processing batch tracking number set includes task batch identification code, group label information, and archived task index. The task time advancement stage segment set specifically includes stage number sequence, time segment label, and node association mapping. The node stage activity comparison structure includes node activity status identifier, stage validity label, and idle node list. The stage status offset segment list specifically includes status jump mark, stage number record, and frequency anomaly label. The task progress status deviation monitoring results include stage status interruption category, status change feature classification, and node status comparison label.

[0058] Please see Figure 2 The task access monitoring module includes:

[0059] The task data recognition submodule obtains submitted task records, identifies the task number field, data source field, and submission order field, establishes field matching rules according to field format standards, matches three types of field tags for each record, removes records with incomplete fields, establishes a field structured mapping set for the remaining records, and generates a field structured mapping matrix.

[0060] First, each submitted data entry is read sequentially according to the text or table format of the task record. Each entry extracts candidate fields that may contain task number, data source, and submission order. By iterating through all fields in each record, the field content is compared and matched against preset format standards. For example, the task number field is usually a combination of letters and numbers, such as "TSK-20231101"; the data source field may be a specified system name or source code, such as "DR1" or "EXT_SYS_2"; and the submission order field is an auto-incrementing integer sequence, such as "1,2,3...". In practical applications, such as when monitoring the daily data of an information processing platform... In the task record table, string pattern detection is performed column by column for each record when extracting fields. For example, the matching rule "TSK-[0-9]+" is used to identify the task number field, the matching rule "EXT_SYS_[0-9]+" is used to identify the data source field, and the matching rule "^\d+$" is used to identify the submission order field. After matching all records, a preliminary field mapping is formed for each record. During the field matching process, when a field may have multiple formats, the confidence score of the field content is compared with the matching confidence score of each rule in turn. The confidence score is assigned a value of 1, 0.5, or 0 depending on whether the field is a complete match, a partial match, or a format error. For example, if a record contains the field "TSK-20251028", then this field completely matches the task number rule, and its confidence score is 1. If the data source field in a record is empty or contains garbled characters such as "####", then this field cannot match any rule, and its matching confidence score is 0. By summarizing the field matching of each record, records that all three fields (task number, data source, and submission order) are present and have a confidence score greater than or equal to 0.5 are filtered out, and the remaining records are removed. In the specific implementation, the field missing threshold is set to 0.5, that is, if any two of the three types of fields have a matching score of 0, then that record is removed. Records are not retained. For example, record R1 contains the task number field "TSK-12345" (score 1), the data source field is empty (score 0), and the submission order field "3" (score 1). The total matching score of this record is 2, which is less than the number of fields 3. Therefore, it is considered to have incomplete fields and is discarded. For the retained records, a structured mapping set is constructed according to the field order. That is, the task number, data source, and submission order fields are combined into a key-value mapping table, such as {number:TSK-12345, source:EXT_SYS_2, order:3}. This process is repeated until all records are processed, and finally a field structured mapping matrix is ​​formed.

[0061] The submission order segmentation submodule, based on the field structured mapping matrix, calls the corresponding values ​​of the submission order field, constructs a record time sequence queue according to the timestamp interval, task number prefix change and data source switching flag, and uses the task submission continuity judgment condition to segment and mark the queue, generating a task submission order segmentation mark sequence;

[0062] The process reads each record in the structured mapping, extracting the submission order field, task number field, data source field, and timestamp information from each record. For the submission order field, all records are sorted in ascending order by value to construct an initial record sequence queue. Then, adjacent record pairs are traversed sequentially. For each pair, the timestamp field is called to calculate the time difference. If the time difference is greater than a set threshold, the current record is marked as the starting point of a new segment. The time difference threshold is set based on the actual task submission frequency. For example, if the regular task interval is set to 20 seconds, the time interval threshold is set to 30 seconds. If the current record time is 15:02:00 and the next record time is 15:02:45, the time difference is 45 seconds, which is greater than the 30-second threshold, thus considered the starting point of a new segment. The record number field is then analyzed, taking the prefix of the task number between the current and previous records (the field before the "-" in the number). If the prefixes are inconsistent, the process is considered complete. The task group switch is defined, and the current record is marked as the starting point of the new segment. For example, the prefixes of task numbers "TSK20251001" and "PRC20251002" are "TSK2025" and "PRC2025" respectively. Since the prefixes are different, they are considered to be across task groups. In addition, the data source field is compared. If the record source field changes, that is, the source of the previous record is "DR1" and the source of the current record is "EXT_SYS2", then the current record is marked as the starting point of the new segment. If any one of the three conditions is true, the current record is marked as the starting point of the new segment, forming a set of logical segmentation points. After marking all segmentation points in all records in turn, the complete queue is divided into multiple continuous sub-segments according to these starting points. Records in each sub-segment are assigned a unified segment number as a segment identifier, such as "Segment_1", "Segment_2", etc. The final output task submission order segmentation marking sequence is the set of segment number sequences for each record.

[0063] The task batch registration submodule segments and marks the sequence according to the task submission order. It calls the task number field value and the data source field value to perform clustering and grouping processing on records under the same segment mark. It sets the group label coding rules and generates the label number of each group according to the mark sequence order. It establishes the mapping relationship between task number and label number and generates a task processing batch tracking number set.

[0064] The segmentation marker value of each record is read and combined with the task number field and data source field. Records with the same segmentation marker value are grouped into the same group to form a preliminary clustering result. In the actual processing, for the record set with segmentation marker Segment_1, the task number field value (e.g., "TSK1001", "TSK1002", "TSK1003") and the corresponding data source field value (e.g., "EXT_SYS1", "EXT_SYS2") of each record are extracted. The record set within the current segment is then sub-clustered based on the data source. If there are multiple data sources within a segment, records from different sources are divided into subgroups. For example, Segment_1 forms subgroup A and subgroup B, where subgroup A corresponds to the data source "EXT_SYS1" and includes task numbers "TSK1001" and "TSK1002", and subgroup B corresponds to the source "EXT_SYS1". S2, containing the number "TSK1003", then sets the group label encoding rules. Label numbers are generated for each subgroup after clustering according to the order of its segment and the index within the group. The label number structure adopts the format "B + segment number + group number". For example, subgroup A corresponds to label "B1_1", and subgroup B corresponds to "B1_2". The record sets of Segment_2 and Segment_3 are processed to generate labels such as "B2_1" and "B2_2". After completing the label encoding, a one-to-one correspondence is established between the task number value in each record and its corresponding label number. For example, the record task number "TSK1001" corresponds to label "B1_1", and the record "TSK1003" corresponds to "B1_2". A complete mapping set is constructed sequentially, and this label mapping action is performed on all records. Finally, a batch tracking number set for task processing is generated. Each record in the task processing flow can be traced back to its segment and data source cluster identity through its label number.

[0065] The execution interval segmentation module includes:

[0066] The timing organization submodule obtains the batches of tasks in the task processing batch tracking number set, extracts the execution time point field value from the task record, constructs a time series list according to the order of the task number sequence, calculates the time difference between adjacent records and generates a time difference sequence, calls the continuous non-outlier segment in the time difference sequence, establishes a set of continuous execution time point segments, and generates a task execution time point segment group.

[0067] All task records are categorized and processed in batches according to the tracking number set. For each batch of records, the execution time field value is extracted sequentially. This field value is represented in standard hour, minute, and second format, such as "14:05:21" and "14:06:10". Then, for each record within the batch, its task number field value is retrieved and sorted in ascending order according to the number sequence rules to construct a time series list corresponding to that batch. Each item in the list is the execution time value after being sorted by number sequence. Next, the time series list is processed pairwise, calculating the time difference between adjacent records. The execution time of the next item is subtracted from the execution time of the previous item using a progressively decreasing method. If the previous item is "14:05:21" and the next item is "14:06:10", the difference is 49 seconds. A set of time difference values, i.e., a time difference sequence, is constructed accordingly. Further judgment operations are performed on each item in the time difference sequence. Segments with consecutive values ​​in the normal range are extracted. The judgment criterion is that the time difference falls within a set threshold range, with a lower limit of 5 seconds for the time difference. The upper limit is 90 seconds. Time differences below 5 seconds or above 90 seconds are considered outliers. If a continuous sequence of time differences is "10 seconds, 12 seconds, 14 seconds, 110 seconds, 11 seconds", then the subsequence "10, 12, 14" formed by the first three time differences is extracted as a continuous non-outlier segment, and the 110-second interval is considered an outlier. The judgment process involves comparing each time difference with the set interval boundary. For example, when the time difference is 14 seconds, the set upper limit of 90 seconds is compared with it. If the result is less than 90 seconds, it is considered a normal value. Next, check if it is greater than the lower limit of 5 seconds. If it is true, the difference is within the normal range. Repeat the operation until all continuous normal time difference segments are located. In each sub-segment with normal time difference, construct a continuous execution segment record by calling the execution time point values ​​of its start and end records. For example, if the start point is "14:05:21" and the end point is "14:06:10", it corresponds to a set of continuous execution time point segments. There may be multiple such segments in this batch. Finally, summarize the continuous time period set of all batches and output the task execution time point segment group.

[0068] The jump segment identification submodule identifies the interruption point of the time difference sequence between adjacent segments based on the task execution time segment group, extracts the task number before and after the jump, sets the jump interval judgment threshold to twice the average task processing interval value, aggregates and classifies the numbering status within the jump segment interval, counts the frequency of each type of jump segment and extracts the start and end times of each segment, and generates a set of jump segment time ranges.

[0069] The process reads the start and end times of each segment, arranges them in chronological order to form a segment list, and then iterates through the time interval between any two adjacent segments. The time difference between segments is obtained by subtracting the start time of the current segment from the end time of the previous segment. For example, if the end time of the previous segment is 15:12:30 and the start time of the current segment is 15:16:00, the calculated time difference is 210 seconds. All inter-segment differences are recorded to form an interrupt time difference sequence. A jump interval threshold is then set, calculated by multiplying the average processing time of adjacent tasks in all task records by a coefficient of 2. For example, if the average processing time is 80 seconds, the jump interval threshold is 160 seconds. The process continues to evaluate each value in the time difference sequence, checking if the time difference is greater than 160 seconds. If the evaluation is successful, the current segment is marked as a jump segment, and the last task number of the segment before the jump and the first task number of the current segment are recorded. For example, if the task number before the jump is "TSK123", the jump interval threshold is... 4”, after the jump, the number is “TSK2001”. All segments that meet the jump conditions are numbered and recorded. Then, the task number attribution of these jump segments is aggregated and classified. That is, the task number field of all records in the jump segment is called in turn, and the segments are grouped and classified according to the number prefix or number segment number. For example, the numbers “TSK2001” and “TSK2002” are classified into the same category, and the numbers “PRC5001” and “PRC5002” are classified into another category. The frequency of each type of jump segment is counted by counting operation. If the jump segment with the number prefix “TSK” appears 5 times and the jump segment with “PRC” appears 3 times, the recorded frequencies are 5 and 3 respectively. The time information of each type of jump segment is extracted, and its start time and end time are called and the interval identifier is constructed. For example, the time range of the jump segment number “TSK2001TSK2004” is “15:16:00-15:18:30”. Finally, the time range of all jump segments is summarized and output as a set of jump segment time ranges.

[0070] The task node binding submodule extracts the associated processing node name field value from the task entries based on the jump segment time range set, filters out the processing node name set corresponding to the records within the time range, establishes a mapping table between node name and corresponding task number, constructs a set of triplet sets of task number, time range and processing node, and generates a set of task time advancement stage segments.

[0071] Retrieve the start and end times of all jump segments and construct an index list of jump segment time ranges. Then, iterate through the task record dataset. For each record, extract its execution time field value. Compare the record's execution time with the start and end boundaries of each jump segment's time range. If a record's execution time is greater than the jump segment's start time but less than or equal to its end time, it is considered part of the current jump segment and added to the pending set. Continue extracting the processing node name field value from this record. This field is typically a process node code or task execution stage name, such as "NODE_A", "Node_Audit", or "Process Node 3". Extract and collect the node names corresponding to all records within the current jump segment, constructing a processing node set for the current jump segment. Since duplicate values ​​may exist in the processing node set, deduplication is performed on the node name set. Unique items are retained by comparing the string set for identical values. This process creates a set of independent nodes corresponding to the current jump segment. For example, the node names corresponding to the records in the jump segment "15:16:00-15:18:30" are "Node Receive", "Node Processing", and "Node Receive". After deduplication, only the names "Node Receive" and "Node Processing" are retained. Next, a binding mapping relationship between task numbers and node names is established. The jump segment records are traversed, and the task number field value and the node name field value in each record are called in turn to construct a one-to-one mapping record. For example, the number "TSK2001" corresponds to "Node Receive", and "TSK2002" corresponds to "Node Processing". All such mappings in all jump segments are summarized. Further, a set of triples of task number, time range and processing node is constructed. Each record is identified by its number, the time range of the jump segment to which it belongs, and the name of the processing node. Finally, all triples are summarized and sorted by time range before being output to form a complete set of task time advancement stage segments.

[0072] The node behavior verification module includes:

[0073] The node stage coverage submodule obtains the stage boundary time and processing node name in the task time advancement stage segment set, filters the processing time field and node name field in the execution node operation record, performs data association by comparing the node name, and filters out records with no corresponding relationship, integrates the associated processing time set by node dimension, and generates a node processing time list set.

[0074] The task number, start time, end time, and processing node name fields are read one by one from the triplet set. For each record, the start and end time values ​​are extracted and combined with the node name field to form a node time boundary index. Then, the execution node operation record dataset is read, and the processing time and node name field values ​​are extracted one by one. The processing time field is recorded in standard hours, minutes, and seconds format, such as "09:42:00" and "09:43:15". The node name field is represented by strings such as "node approval" and "node dispatch". The node name field extracted from the execution node record is then compared one by one with the processing node name field extracted from the task stage segment set. If the two string values ​​are completely identical, it is considered a name match, and the record is retained. If they are inconsistent, it is considered a mismatch and is removed. The removal rule is based on character length and character content. The process involves a bit-by-bit comparison to create a set of node operation records after filtering out records with no corresponding relationship. Then, for the remaining matching records, a classification operation is performed based on the node name field as the grouping key. All records with the same node name are grouped into the same group. For example, "Node Review" corresponds to 5 records, and "Node Dispatch" corresponds to 3 records. The processing time field value of the records under each group is extracted to construct a processing time set. If the time corresponding to "Node Review" is "09:42:00", "09:42:45", "09:43:30", "09:44:00", and "09:44:30", then the processing time set for this node is constructed using the above 5 time values. This operation is performed on each node dimension in turn to form a complete mapping structure between node names and corresponding processing time sets. Finally, a list of node processing times is output.

[0075] The active marking determination submodule, based on the node processing time list set, calls the start and end times in the task time advancement stage segment set, performs logical judgment on whether the processing time of each node and the time interval of each stage are covered, records the coverage flag, establishes a node marking mapping matrix by stage, and generates a node stage coverage identifier matrix.

[0076] Read the processing time set corresponding to each processing node, and extract the start and end times of all stages from the task time advancement stage segment set. Standardize all time fields in both sets and use them as judgment parameters for item-by-item comparison. Then, iterate through each record in the stage segment set, extracting the start time Tstart and end time Tend for that stage. Next, process each node in the node processing time list set, performing a range judgment on each processing time Ti under the node. This judgment is achieved by comparing whether Ti is greater than or equal to Tstart and less than or equal to Tend. If the condition is true, the processing time is considered to be within the current stage range, and the node is recorded as having an operation in that stage, with an overwrite flag set to "1". If the condition is false, it is marked as "0". For example, if a node "Node Review"... The processing time set for the "core" is "10:01:10", "10:03:00", and "10:05:30". The current stage segment time is "10:00:00-10:04:00". The first two times satisfy the interval judgment condition and are marked as "1". The last one exceeds the end time and does not meet the condition. Therefore, the corresponding stage is still a valid coverage. Continue to use the node name as the column index and the stage number as the row index. The result of each coverage judgment is assigned to the matrix position according to the node dimension. For example, under stage 1, "node review" is 1 and "node dispatch" is 0. Under stage 2, "node review" is 1 and "node dispatch" is 1. In this way, a matrix structure composed of Boolean values ​​is constructed. Each element of the matrix represents whether a node has a processing behavior in a certain stage. Finally, after completing the coverage judgment of all nodes and all stages, the output forms a complete node stage coverage identifier matrix.

[0077] The idle node classification submodule counts the number of times each node covers all stages based on the node stage coverage identifier matrix, sets the activity identification threshold as the upper limit of zero stage coverage, classifies nodes below the threshold as unrecorded nodes, constructs a comparison structure between node name and stage status, and generates a node stage activity comparison structure.

[0078] Perform a row-by-row statistical operation on each column of the matrix to obtain the coverage count of each node across all stages. Specifically, sum the values ​​of cells with a value of 1 in each column. If "Node A" appears 3 times in 5 stages, its coverage count is recorded as 3. After completing the statistics for all nodes, a list of node names and their corresponding total stage coverage is generated. Next, an activity identification threshold is set, which is defined as the upper limit for nodes with zero coverage counts. This value can be set by checking if the coverage count of all nodes is 0. For example, if any node has a coverage count of 0, the threshold is set to 1; otherwise, it is set to 0. Subsequently, the coverage count of each node in the node list is compared with the threshold. If a node's coverage count is zero... If the number of coverages is less than a set threshold, the node is classified as an unrecorded node and its status is marked as "unrecorded". If the number of coverages is greater than or equal to the threshold, its status is marked as "active". For example, if node "Node B" has 0 coverages, which is less than the threshold of 1, it is classified as an unrecorded node. If node "Node C" has 2 coverages, it is classified as an active node. Then, the node name and the corresponding status mark are combined and mapped to form a mapping table structure between node name and stage active status. This structure summarizes the activity status of each node by node dimension and is represented in tabular form, such as "Node A - Active", "Node B - Unrecorded", "Node C - Active". After all nodes are processed in sequence, a complete node stage active comparison structure is generated.

[0079] The state fluctuation identification module includes:

[0080] The status classification submodule reads the active node number from the node stage active comparison structure, obtains the status field value and timestamp information from the task data record, finds the stage to which the task belongs based on the task number corresponding to the active node number, groups and classifies each status record according to the stage number, establishes the corresponding mapping relationship between status records and stage numbers, and generates a stage status value mapping table.

[0081] The node dimension is filtered, retaining only the names of nodes marked as "active" and their associated task ID sets. Then, each record in the task data record set is read, and the status field and timestamp field values ​​are extracted sequentially. The status field indicates the task's processing status at a specific point in time, such as "waiting," "in execution," "completed," or "abnormal." The timestamp field indicates the specific time the status occurred, such as "09:45:30" or "09:47:00." The task IDs contained within the active node IDs are then matched one by one, searching the task data record set for all records where the task ID matches the active node set. The status field and timestamp field of the corresponding record are extracted. For the task ID field in each status record, the task time progression stage segment set is called, and the time range to which the task ID belongs in the triplet set is compared. This is done by... The timestamp of a status record is compared with the start and end times of the stage to determine its stage number. For example, if the status time "09:46:00" is between the stage times "09:45:00-09:48:00", then the status record is determined to belong to that stage number. A one-to-one mapping relationship is established between the status record and the corresponding stage number. If task number "TSK2001" has three status records, and their status times correspond to stage 1, stage 2 and stage 2 respectively, then the status record of this task will be classified in two stages. After processing all status records and completing the pairing of status and stage number, all records are grouped by the stage number as the index dimension. Each group contains all the status value records belonging to that stage, forming a mapping table structure with stage as the key and the list of status values ​​as the value. Finally, the stage status value mapping table is completed.

[0082] The continuous repetition labeling submodule extracts the state value sequence within each stage based on the stage state value mapping table, calls the state value results of adjacent stages, counts the number of times the state value appears in consecutive stages, determines whether the state value appears twice or more consecutively in adjacent stages, marks the repetition identification information of the state value in the corresponding stage, and generates a state cross-stage repetition labeling matrix.

[0083] For each stage number, an extraction operation is performed on the list of state values. After sorting the state values ​​in ascending order by stage number, a set of stage state sequences is constructed. Then, starting from the first stage, the state value sequence Sn of the current stage and the state value sequence Sn+1 of the next stage are sequentially obtained. Element comparisons are performed between the two stage state value sets to determine if the state value exists in both Sn and Sn+1. If a duplicate state value exists, its cross-stage occurrence count is recorded as 1. The comparison continues in the next stage between Sn+1 and Sn+2. If the state value still exists, its occurrence count is recorded as 2, and so on, until the state value no longer appears in a certain stage. For example, if "state value A" appears in stages 1, 2, and 3, but not in stage 4, then state value A has appeared consecutively across stages 3 times. Then, the consecutive occurrence counts are... The system performs a judgment operation based on the number of occurrences. If a state value is identified as appearing at least twice in two or more consecutive stages, it is marked as a "continuously repeated state". The system then marks the position of the state value in all stages in which it appears. A combination key is created between the current state value and the stage number, and a Boolean flag value of "1" is assigned to it, indicating that the state value is continuously repeated in this stage. If the state value appears independently in a certain stage, it is marked as "0". After all state values ​​and corresponding stages are combined, a Boolean identifier matrix of state values ​​and stage numbers is constructed. The rows of the matrix represent stage numbers, and the columns represent all independent state values. A value of "1" in the matrix indicates that the state value appears continuously in at least one of the stages before and after it, and "0" indicates that the state value exists independently in the current stage. Finally, after all stages have been traversed, a state cross-stage repetition marker matrix is ​​generated.

[0084] The jump frequency extraction submodule filters out state value change records that are not marked as duplicates based on the state cross-stage repetition marking matrix, accumulates the jump count by stage number, sets the jump count threshold to twice the average state change of the stage at that node, collects stage numbers that exceed the threshold, establishes a set of related entries between stage number and jump count, and generates a list of stage state offset segments.

[0085] The process reads the state flag sequence corresponding to each stage in the matrix row by row. State values ​​marked "0" are considered as not being identified as consecutively repeated state values, thus representing independent state change records for the current stage. A set of transition states is constructed based on the stage number. Then, starting from stage number 1, each stage is processed sequentially. For each stage, the items with a value of "0" in the state flag sequence are cumulatively counted to obtain the number of state transitions within the current stage. For example, if stage 1 has states "Completed," "Abnormal," and "Suspended," all of which are non-repeating flags, the transition count is 3. After completing all stages, a one-to-one correspondence table between stage numbers and transition counts is constructed. The transition counts for all stages are then summed and divided by the total number of stages to obtain the average transition count for all stages. If the transition count for all stages is... If the number is {2,3,1,4,2}, then the average value is 2.4. The jump threshold is then set to twice this average value, i.e., the threshold is 4.8, rounded up to 5. The judgment operation continues, comparing the number of jumps in each stage with the set threshold. If the number of jumps in a stage is greater than or equal to 5, it is judged as a frequently jumping stage, and the stage number is recorded in the jump stage set. After this process, a set of frequently jumping stage numbers is formed. Simultaneously, each stage number identified as frequently jumping is paired with its actual number of jumps to construct a mapping item between stage number and jump count. For example, if stage 6 has 6 jumps, then the entry (6,6) is generated. All frequently jumping stage numbers and statistical values ​​are summarized in this way, and finally, a list of stage state offset segments is output.

[0086] The stage status monitoring output module includes:

[0087] The phase information extraction submodule reads the offset phase number from the phase status offset segment list, extracts the phase name field and boundary time field value of the corresponding phase in the task time advancement phase segment set, performs start and end time verification processing on the boundary time field value, constructs a data structure corresponding to the phase number, its name and boundary time, and generates an offset phase information mapping table.

[0088] For each number in the list, a stage index set is constructed by extracting each entry. Then, the task time advancement stage segment set is entered. The stage number field, stage name field, and boundary time field in the triplet structure are synchronously called. The matching method for the corresponding numbers uses an equality check, i.e., whether the current offset stage number is equal to the stage number field of any record in the task advancement stage set. If a match is successful, the stage name field value, its start time field value Tstart, and its end time field value Tend are extracted from that record. Boundary time verification is then performed, i.e., verifying the logical correctness of Tstart being less than Tend. If a start time is later than an end time, it is marked as a time anomaly, and the current stage is marked as "invalid." If the verification is successful, the original time value is retained and the status is recorded as "valid". For example, if the offset stage number is 5, the name field found in the advancement stage set is "stage_manual verification", the start time is "10:22:00" and the end time is "10:25:00". Since the start and end relationship is correct, the time verification of this stage is passed. After completing the time boundary checks of all offset stages, the stage number, stage name, start time, end time and verification status are uniformly encapsulated into a structured record. The complete offset stage data structure set is constructed by sorting the stage numbers in ascending order. Each item in the structure contains three items: number field, name field and boundary time field, and is marked with a status flag indicating whether the time relationship is compliant. Finally, this set is output to form an offset stage information mapping table.

[0089] The State-Behavior Comparison Submodule extracts the state field from the task record and the operation timing data from the node behavior record based on the offset stage information mapping table. It partitions and extracts the two types of data according to the stage number, calls the cross-distribution of the marked state value sequence and the node behavior record in the same time period, judges whether there are state interruptions, repetitions or concentrated mutations, and generates a stage state-behavior anomaly marking matrix.

[0090] Read the stage number, stage name, start time, and end time field values ​​from each record, and use this time interval as a filter window. Then, extract all status field values ​​and their corresponding timestamp information from the task record set. Simultaneously, read the time sequence information field of each operation behavior from the node behavior record set. This field indicates the specific time point when the behavior action occurred, such as the time points corresponding to "submit," "review," and "process," which form the behavior sequence. Perform time range filtering operations on the above two types of data based on the offset stage number. For each stage number, extract all status records and operation behavior records between the stage start time Tstart and end time Tend, constructing the current stage's status time sequence and node behavior time sequence. Then, perform cross-distribution analysis on the two types of sequences within the stage. First, arrange the status values ​​in chronological order to form a status change trajectory sequence, and then arrange the operation behavior records in chronological order to form a node behavior trajectory sequence. Determine whether there are abnormal change patterns in the status trajectory, specifically including three situations: one is status interruption, that is, a continuous period of time without status records in the status trajectory. The judgment method is as follows: First, if there is an operation behavior within the time period, it is judged by calculating the time difference between two adjacent state records. If the time interval is greater than the preset interruption interval threshold and there is an operation record within the interval, it is marked as an interruption anomaly. Second, if the state is repeated, that is, the same state value appears repeatedly at multiple time points and is not associated with any new type of node behavior, it is judged by two or more consecutive records with the same state value in the state trajectory, while the behavior trajectory does not change. Third, if the state mutation is concentrated, that is, the state value changes continuously in a short period of time, it is judged by three consecutive state records with different values ​​and the time interval is less than the set mutation time threshold. For example, if the interval between each record is less than 10 seconds and the state value changes in the form of "A→B→C", it is considered a mutation. After performing the above three judgment operations for each stage, a mark bit is established for each type of anomaly in the corresponding stage. "1" indicates the existence of this type of anomaly and "0" indicates the absence of this type of anomaly. A three-dimensional Boolean matrix is ​​constructed according to the stage number as the row index and the anomaly type as the column index. Each row records whether the three types of anomalies occur in a certain stage. Finally, after processing all offset stages, the state behavior anomaly mark matrix is ​​output.

[0091] The State Deviation Summary Submodule extracts the stage number and corresponding state feature item with abnormal markings based on the stage state behavior anomaly marking matrix. It archives the three types of features, namely state interruption, repetition, and concentrated mutation, in the form of tags. It calls the stage name and boundary time field values ​​in the offset stage information mapping table to combine and generate a four-tuple record set of stage number, name, boundary time and state offset tag, and generates the task progress state deviation monitoring result.

[0092] The process iterates through all records in the matrix, reading each stage number row by row and checking for a "1" marker. If at least one anomaly marker exists, the current stage number is extracted and its corresponding anomaly type is recorded. Anomaly types are categorized into three types: state interruption, state repetition, and concentrated state mutation, corresponding to the three marker fields in the matrix. A marker value of "1" indicates the anomaly type is valid. All valid anomaly types are summarized in a list and converted into tag strings. For example, if a stage experiences both interruption and repetition, the tag "interruption, repetition" is generated. After mapping all stage numbers to anomaly types, the offset stage information mapping table is called to query the records corresponding to the aforementioned stage numbers, extracting the stage name field value and boundary time field value. The boundary time includes the start time field. The Tstart and Tend fields, after performing format validation on the time values ​​to ensure they are all in valid time formats, are used to construct a four-tuple structured record set consisting of the stage number, stage name, start time, end time, and status offset label. For example, if the stage number is 7, the corresponding stage name is "Stage_Delivery Review", the start time is "11:03:00", the end time is "11:06:20", and the anomaly label is "Recurrence, Mutation", then the constructed record would be (7, Stage_Delivery Review, 11:03:00-11:06:20, Recurrence, Mutation). This process is repeated for all abnormal stages to construct the four-tuple. All records are sorted in ascending order by stage number and uniformly packaged into a structured output set, ultimately generating the task progress status deviation monitoring result.

[0093] Please see Figure 3 A method for intelligently monitoring the progress status of data task processing includes the following steps:

[0094] S1: Obtain the task number, data source and submission order fields from the task record, identify the task submission order, group the tasks according to the source field, establish a number mapping relationship for each group of tasks, mark the corresponding group label and belonging information, complete the task record archiving and organization, and generate a batch tracking number set for task processing.

[0095] S2: Based on the batch tracking number set of task processing, organize the execution order information in the task record, identify the jump position in the execution order, mark the task number to which the jump segment belongs, collect the frequently jumped numbers as processing segments, organize the start and end times of the jump segments, extract the processing node name in the task entry and bind it with the task content, map the processing time period of each node in the task, divide the continuous advancement stage in the task processing process, and generate a set of task time advancement stage segments;

[0096] S3: Based on the stage boundary information and node mapping data of the task time advancement stage segment set, read the processing time in the node operation record, compare the relationship between the behavior of each node and the stage time range, mark the node number that has behavior in each stage, classify and mark the nodes that have not participated in any stage, establish the activity status structure of the node corresponding to the stage, and generate the node stage activity comparison structure.

[0097] S4: Based on the active node number in the node phase active comparison structure, divide the status field content in the task data according to the phase, organize the change order of the status value in each phase, mark the phases where the status value fluctuates repeatedly and frequently, summarize the phase numbers of the abnormal jump concentration as the offset segment, and generate a phase status offset segment list.

[0098] S5: Based on the offset stage number in the stage status offset segment list, extract the name and start and end time of the corresponding stage, read the node behavior and status record information, compare the recorded status and node behavior within the stage, identify the interruption, repetition or abnormal fluctuations that occur in the stage, complete the annotation and summary of various status behaviors, and generate the task progress status deviation monitoring results.

[0099] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A data task processing progress status intelligent monitoring system, characterized in that, The system includes: The task access monitoring module obtains the submitted task records, identifies the task number, data source, and submission order fields, groups all records and marks them with batch numbers, archives the task group tags and attribution information for each batch of tasks, and generates a batch tracking number set for task processing. The task access monitoring module includes: The task data recognition submodule obtains submitted task records, identifies the task number field, data source field, and submission order field, establishes field matching rules according to field format standards, matches three types of field tags for each record, removes records with incomplete fields, establishes a field structured mapping set for the remaining records, and generates a field structured mapping matrix. The submission order segmentation submodule, based on the field structured mapping matrix, calls the corresponding values ​​of the submission order field, constructs a record time sequence queue according to the timestamp interval, task number prefix change and data source switching flag, and segments the queue using the task submission continuity judgment condition to generate a task submission order segmentation mark sequence. The task batch registration submodule segments and marks the sequence according to the task submission order, calls the task number field value and the data source field value, performs clustering and grouping processing on records under the same segment mark, sets grouping label coding rules and generates label numbers for each group according to the mark sequence order, establishes a mapping relationship between task number and label number, and generates a task processing batch tracking number set. The execution interval segmentation module reads each task batch from the task processing batch tracking number set, marks the jump status of the task processing order, records the start and end times of the frequently jump fields, extracts the execution node names associated in the task and completes the binding, and generates a task time advancement stage segment set. The execution interval segmentation module includes: The timing organization submodule obtains each task batch from the task processing batch tracking number set, extracts the execution time point field value from the task record, constructs a time series list according to the order of the task number sequence, calculates the time difference between adjacent records and generates a time difference sequence, calls the continuous non-outlier segment in the time difference sequence, establishes a set of continuous execution time point segments, and generates a task execution time point segment group. The jump segment identification submodule identifies the interruption point of the time difference sequence between adjacent segments based on the task execution time point segment group, extracts the task number before and after the jump, sets the jump interval judgment threshold to twice the average task processing interval value, aggregates and classifies the numbering status within the jump segment interval, counts the frequency of each type of jump segment and extracts the start and end times of each segment, and generates a jump segment time range set. The task node binding submodule extracts the associated processing node name field value from the task entry based on the jump segment time range set, filters out the processing node name set corresponding to the record within the time range, establishes a node name and corresponding task number binding mapping table, constructs a set of task number, time range and processing node triplet, and generates a task time advancement stage segment set. The node behavior verification module reads the phase boundary time and node mapping data of the task time advancement phase segment set, checks the interval between the processing time of the execution node and the phase time, determines whether there is operation coverage, marks the consecutive missing nodes as idle state, and generates a node phase activity comparison structure. The node behavior verification module includes: The node stage coverage submodule obtains the stage boundary time and processing node name in the task time advancement stage segment set, filters the processing time field and node name field in the execution node operation record, performs data association by comparing the node name, and filters out records with no corresponding relationship, integrates the associated processing time set by node dimension, and generates a node processing time list set. The active marker determination submodule, based on the node processing time list set, calls the start and end times in the task time advancement stage segment set, performs a logical judgment on whether the processing time of each node and the time interval of each stage are covered, records the coverage flag, establishes a node marker mapping matrix according to the stage, and generates a node stage coverage identifier matrix. The idle node classification submodule counts the number of times each node covers all stages according to the node stage coverage identifier matrix, sets the activity identification threshold as the upper limit of the stage coverage count being zero, classifies nodes below the threshold as unrecorded nodes, constructs a comparison structure between node name and stage status, and generates a node stage activity comparison structure. The state fluctuation identification module reads the active node number in the node stage active comparison structure, performs sequential tracking of state changes within the stage, marks recurring state entries, extracts the stage number with frequent jumps, and generates a stage state offset segment list. The state fluctuation identification module includes: The status classification submodule reads the active node number from the node stage active comparison structure, obtains the status field value and timestamp information from the task data record, finds the stage to which the task belongs based on the task number corresponding to the active node number, groups and classifies each status record according to the stage number, establishes the corresponding mapping relationship between status records and stage numbers, and generates a stage status value mapping table. The continuous repetition labeling submodule extracts the state value sequence within each stage based on the stage state value mapping table, calls the state value results of adjacent stages, counts the number of times the state value appears in consecutive stages, determines whether the state value appears twice or more consecutively in adjacent stages, marks the repetition identification information of the state value in the corresponding stage, and generates a state cross-stage repetition labeling matrix. The transition frequency extraction submodule filters out state value change records that are not marked as duplicates based on the state cross-stage repetition marking matrix, accumulates the number of transitions by stage number, sets the threshold for the number of transitions to twice the average state change of that stage, collects stage numbers that exceed the threshold, establishes a set of associated entries between stage number and number of transitions, and generates a list of stage state offset segments.

2. The intelligent monitoring system for data task processing progress status according to claim 1, characterized in that: The task processing batch tracking number set includes task batch identification code, group label information, and archived task index. The task time advancement stage segment set specifically includes stage number sequence, time segment label, and node association mapping. The node stage activity comparison structure includes node activity status identifier, stage validity label, and idle node list. The stage status offset segment list specifically includes status jump mark, stage number record, and frequency anomaly label.

3. The intelligent monitoring system for data task processing progress status according to claim 1, characterized in that, The system also includes: The stage status monitoring output module reads the offset stage number from the stage status offset segment list, compares the stage number with the node record behavior, identifies abnormal status stages, integrates the performance status and annotation information of each stage, and generates task progress status deviation monitoring results. The task progress status deviation monitoring results include stage status interruption categories, status change feature classifications, and node status comparison annotations.

4. The intelligent monitoring system for data task processing progress status according to claim 3, characterized in that, The stage status monitoring output module includes: The stage information extraction submodule reads the offset stage number from the stage status offset segment list, extracts the stage name field and boundary time field value of the corresponding stage in the task time advancement stage segment set, performs start and end time verification processing on the boundary time field value, constructs a data structure corresponding to the stage number, its name and boundary time, and generates an offset stage information mapping table. The state behavior comparison submodule extracts the state field from the task record and the operation time sequence data from the node behavior record based on the offset stage information mapping table. It partitions and extracts the two types of data according to the stage number, calls the cross-distribution of the marked state value sequence and the node behavior record in the same time period, judges whether there are state interruptions, repetitions or concentrated mutations, and generates a stage state behavior abnormality marking matrix. The state deviation summary submodule extracts the stage number and corresponding state feature item with abnormal markers based on the stage state behavior anomaly marker matrix. It archives the three types of features, namely state interruption, repetition and concentrated mutation, in the form of tags. It calls the stage name and boundary time field value in the offset stage information mapping table to combine and generate a four-tuple record set of stage number, name, boundary time and state offset tag, and generates the task progress state deviation monitoring result.

5. A method for intelligent monitoring of data task processing progress status, characterized in that, The monitoring method is used in the intelligent monitoring system for data task processing progress status according to any one of claims 1-4, and includes the following steps: S1: Obtain the task number, data source and submission order fields from the task record, identify the task submission order, group the tasks according to the source field, establish a number mapping relationship for each group of tasks, mark the corresponding group label and belonging information, complete the task record archiving and organization, and generate a batch tracking number set for task processing. S2: Based on the task processing batch tracking number set, organize the execution order information in the task record, identify the jump position in the execution order, mark the task number to which the jump segment belongs, collect the frequently jumped numbers as processing segments, organize the start and end times of the jump segments, extract the processing node name in the task entry and bind it with the task content, map the processing time period of each node in the task, divide the continuous advancement stage in the task processing process, and generate a task time advancement stage segment set; S3: Based on the stage boundary information and node mapping data of the task time advancement stage segment set, read the processing time in the node operation record, compare the relationship between the behavior of each node and the stage time range, mark the node number that has behavior in each stage, classify and identify the nodes that have not participated in any stage, establish the activity status structure of the node corresponding to the stage, and generate the node stage activity comparison structure. S4: Based on the active node number in the node stage active comparison structure, divide the status field content in the task data according to the stage, organize the change order of the status value in each stage, mark the stage where the status value fluctuates repeatedly and frequently, summarize the stage numbers of the abnormal jump concentration as the offset segment, and generate a stage status offset segment list. S5: Based on the offset stage number in the stage state offset segment list, extract the name and start and end time of the corresponding stage, read the node behavior and status record information, compare the recorded status and node behavior within the stage, identify the interruption, repetition or abnormal fluctuation in the stage, complete the annotation and summary of various status behaviors, and generate the task progress status deviation monitoring result.

Images