Big data-based concentrator data collection and analysis method and system

Abstract

The application relates to the technical field of electric power data analysis, and discloses a concentrator collection data analysis method and system based on big data, which comprises the following steps: real-time capturing of high-priority tasks, generation of a concentrator collection task flow sequence arranged according to an initial scheduling order; cross-task dependence chain logical correlation tracking of the concentrator collection task flow sequence, construction of a multi-dimensional scheduling characteristic vector reflecting a task importance gradient; extraction of the evolution trajectory of the occupation of bandwidth and collection time slots by high-priority tasks based on the multi-dimensional scheduling characteristic vector, detection of potential scheduling out-of-order candidate tasks by using a hierarchical priority aggregation algorithm; priority classification and demarcation of the scheduling out-of-order candidate tasks and the associated low-priority task intervals, construction of a local task logical atlas with a priority weight gradient; and dynamic intervention judgment of the scheduling evolution path based on the local task logical atlas. The application has the advantage of improving the overall scheduling efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of power data analysis, and in particular to a method and system for analyzing concentrator collected data based on big data. Background Art

[0002] Currently, electricity users generally use a combination of smart meters and concentrators to collect and manage electricity usage information. As the core aggregation node of the power collection system, the concentrator is responsible for periodically collecting, caching, preprocessing, and uploading data from a large number of smart meters within its jurisdiction. However, with the widespread adoption of time-of-use (TOU) electricity pricing, user electricity usage behavior exhibits significant differences during peak, off-peak, and off-peak periods. This makes it no longer feasible to rely solely on fixed-period meter reading. Instead, it requires a combination of TOU pricing and load fluctuations to achieve differentiated collection and dynamic priority scheduling. In scenarios where high-priority tasks burst during peak periods, the bandwidth and collection window resources of a single concentrator are easily saturated. Regular periodic tasks compete with critical real-time tasks for resources, impacting the timeliness and integrity of high-priority data uploads. To alleviate the resource bottleneck of a single node, practical deployments often involve two concentrators working together. However, there is the challenge of dynamically alternating the sharing of data between the two concentrators based on the timeliness sensitivity of the collected data, task priority, and update frequency. Therefore, it is necessary to design a big data-based concentrator data analysis method and system to improve overall scheduling efficiency. Summary of the Invention

[0003] In response to the shortcomings of the existing technology, the present invention provides a concentrator data collection and analysis method and system based on big data, which has the advantage of improving overall scheduling efficiency and solves the problems in the above-mentioned background technology.

[0004] To achieve the above-mentioned purpose of improving overall scheduling efficiency, the present invention provides the following technical solution: a method for analyzing data collected by a concentrator based on big data, comprising the following steps:

[0005] Capture high-priority tasks in real time, extract urgency level labels, task timeliness boundaries, and bandwidth sensitivity parameters based on task semantic parsing algorithms, and generate a concentrator collection task flow sequence arranged in the initial scheduling order;

[0006] The concentrator collects task flow sequences and tracks the logical association of cross-task dependency chains. Combining historical high-priority task interference trajectories, low-priority task deferrable windows, and link conflict frequencies, a multi-dimensional scheduling feature vector reflecting the task importance gradient is constructed.

[0007] Based on the multi-dimensional scheduling feature vector, a hierarchical priority aggregation algorithm is used to extract the preemption evolution trajectory of high-priority tasks on bandwidth and acquisition time slots. By identifying the degree of deviation between the trajectory distribution and the initial scheduling order, potential scheduling misorder candidate tasks are detected.

[0008] Prioritize the scheduling out-of-order candidate tasks and the associated low-priority task intervals, and integrate the high-urgency nodes, deferrable nodes, and bandwidth scheduling nodes within the interval to construct a local task logic graph with a priority weight gradient.

[0009] Based on the local task logic graph, combined with the resource gap of high-priority imbalanced nodes and the task conflict oscillation trend under real-time bandwidth fluctuations, dynamic intervention judgment is made on the scheduling evolution path.

[0010] Preferably, the process of generating the concentrator collection task flow sequence is:

[0011] The data collection time of high-priority tasks is calibrated within the concentrator through a unified time synchronization protocol;

[0012] The task semantic parsing algorithm is used to pre-process the description information of high-priority tasks in real time, extract the urgency level label, task time limit boundary and bandwidth sensitivity parameters, construct a priority comprehensive scoring function, score and sort the high-priority tasks, and form a continuous concentrator acquisition task flow sequence.

[0013] Preferably, the process of tracking the cross-task dependency chain logic association of the concentrator collection task flow sequence is as follows:

[0014] Based on the data transmission path and triggering order of each task in the task flow sequence collected by the concentrator, the timing dependency and resource coupling relationship between tasks are identified, and a dependency graph reflecting the order of task scheduling and resource association is constructed;

[0015] Combining historical task priority change trajectories, link transmission delays, and conflict frequencies, we calculate the response frequency and weight strength of each task node in the dependency graph within different scheduling cycles, and obtain a weighted dependency graph that reflects the importance of tasks and the degree of dependency.

[0016] Based on the weighted dependency graph, the task nodes are arranged in chronological order, and a dynamic adjustment model based on time series analysis is used to track the weight fluctuations between adjacent nodes in the dependency graph. By identifying the priority transmission, response delay changes and resource competition trends between nodes, the logical association trajectory of the cross-task dependency chain where the dependency relationship evolves over time is output.

[0017] Preferably, the process of constructing a multi-dimensional scheduling feature vector reflecting the task importance gradient is:

[0018] The task nodes in the logical association trajectory are mapped to a multi-dimensional feature space by a cross-task dependency chain logic, and the emergency level label, task time limit boundary and bandwidth sensitivity parameter are fused;

[0019] The scheduling response frequency, task interaction mode and historical priority change trend in the task execution process are extracted to form a multi-dimensional statistical parameter set;

[0020] The multi-dimensional statistical parameter set is processed by normalization and dimension reduction technology to obtain a multi-dimensional scheduling feature vector reflecting the priority and importance gradient of the task.

[0021] Preferably, the evolution trajectory process of high-priority tasks occupying bandwidth and acquisition time slots is extracted by a hierarchical priority aggregation algorithm as follows:

[0022] Based on the constructed multi-dimensional scheduling feature vector, the scheduling performance of each high-priority task in the historical scheduling record is mapped to the feature space, and a hierarchical priority aggregation algorithm is used to cluster analyze the mapped task feature data, so as to identify the distribution pattern of the task occupying behavior of bandwidth and acquisition time slots under different priority levels;

[0023] The time series prediction method is introduced to dynamically track the priority ranking evolution process of high-priority tasks in continuous scheduling periods, extract the change trend of task scores in each period, and construct a task score fluctuation trajectory;

[0024] The resource occupation strength index of the task in the scheduling period is calculated by combining the task ranking deviation and the change of concentrator bandwidth occupancy rate in each time period;

[0025] The resource contention path of the task in the dynamic scheduling process is reconstructed by fusing the task score fluctuation trajectory and the resource occupation strength index, and an evolution trajectory of occupation is formed.

[0026] Preferably, the process of detecting potential scheduling out-of-order candidate tasks is as follows:

[0027] The evolution trajectory of occupation extracted by the hierarchical priority aggregation algorithm is compared with the reference trajectory in the pre-established normal scheduling trajectory library in terms of structural similarity;

[0028] The graph structure comparison algorithm is used to locate the key task area with structural deviation, node disorder or connection anomaly in the task nodes and dependency relationship in the evolution trajectory of occupation;

[0029] Combined with the preset deviation threshold judgment mechanism, the abnormal task sequence segment in the key task area is identified, and the identified abnormal task sequence segment is marked as a potential scheduling out-of-order candidate task.

[0030] Preferably, the process of preferentially classifying and delimiting the scheduling out-of-order candidate task and the associated low-priority task interval is as follows:

[0031] determine a core time period in which the scheduling out-of-order candidate task occurs based on a key task node in the scheduling out-of-order candidate task;

[0032] In the core time period, according to the logical dependency relationship, resource sharing relationship and scheduling conduction relationship between the key task node and the surrounding task node, the low-priority tasks in the influence range are extracted and mapped to form a candidate task interval;

[0033] In combination with the resource occupation strategy of each candidate node, the priority level of each task node in the candidate task interval is divided, and the priority classification is completed.

[0034] Preferably, the process of constructing a local task logical graph with priority weight gradient is:

[0035] Extract the key task nodes in the candidate task interval, including high urgency nodes, delayable nodes and bandwidth scheduling nodes, to form a multi-attribute heterogeneous node set;

[0036] According to the historical scheduling record, task dependency relationship and real-time scheduling feedback, a directed edge set between nodes is constructed to form a multi-modal task graph reflecting the actual scheduling execution logic;

[0037] Use graph neural network to mine key topological patterns in the multi-modal task graph, identify priority conduction abnormalities and bandwidth preemption hotspots, locate the key task nodes that have the greatest impact on the scheduling order, and construct a local task logical graph.

[0038] Preferably, the process of dynamically intervening in the scheduling evolution path is:

[0039] Based on the constructed local task logical graph, the key task nodes that have the greatest impact on the scheduling order are extracted, and the dependency relationship and resource occupation intensity of the key task nodes and upstream and downstream tasks are combined to locate the resource bottleneck nodes and key evolution paths in the scheduling chain;

[0040] Access real-time task scheduling data stream, continuously monitor the conflict frequency, preemption event and bandwidth shock degree between tasks on the key path, and comprehensively evaluate the stability and risk propagation of the scheduling evolution path;

[0041] When the scheduling evolution path shows signs of instability, a dynamic scheduling intervention strategy is designed according to the node priority weight and resource demand, the task execution order is adjusted, the resource allocation ratio is optimized, and the dependency relationship between tasks is coordinated.

[0042] The concentrator data analysis system based on big data includes:

[0043] Task identification module: real-time capture of high-priority tasks, analysis of task semantic labels, time limit boundaries and bandwidth sensitivity, and generation of preliminary collection task flow sequence;

[0044] Feature construction module: This module tracks the dependency chain logic between tasks and combines historical task interference information and link conflict frequency to construct a multi-dimensional scheduling feature vector.

[0045] Out-of-order detection module: This module analyzes the preemption trajectory of high-priority tasks based on scheduling feature vectors, identifies scheduling order deviations, and screens candidate tasks for scheduling out-of-order tasks.

[0046] Graph generation module: This module integrates the node information of out-of-order candidates and low-priority task intervals to construct a local task logic graph with weight gradients.

[0047] Path intervention module: Combines resource gaps and conflict trends to dynamically determine scheduling path adjustment plans.

[0048] Compared with the prior art, the present invention provides a method and system for analyzing data collected by a concentrator based on big data, which has the following beneficial effects:

[0049] By introducing task semantic parsing, priority scoring model and multi-dimensional scheduling feature vector construction mechanism, the present invention can fully perceive the differences in the sensitivity of different tasks to delay, bandwidth and resources, and realize real-time identification and refined sorting of high-priority tasks; using the hierarchical aggregation algorithm to extract the task's evolutionary behavior of bandwidth and acquisition window preemption, and combining historical interference trajectory and conflict frequency to identify scheduling risks, it can predict scheduling misorder nodes and resource bottlenecks in advance. By constructing local task logic maps and scheduling evolution paths, accurate positioning of key task nodes and priority-level intervention are achieved, enabling the system to adaptively adjust acquisition window allocation, bandwidth tilt ratio and scheduling control strategy. It can significantly improve the scheduling response speed and resource guarantee capability of high-priority tasks in peak load scenarios, reduce data upload delay and loss probability, and improve bandwidth utilization efficiency and the overall acquisition and scheduling performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 Schematic diagram of the method of the present invention;

[0051] Figure 2 Schematic diagram of the system of the present invention. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] Example 1: Please refer to Figure 1As shown, the data collection and analysis method based on big data concentrator according to the embodiment of the present invention includes the following steps:

[0054] S1: Capture high-priority tasks in real time, and extract urgency level labels, task timeliness boundaries, and bandwidth sensitivity parameters based on the task semantic parsing algorithm to generate a concentrator acquisition task flow sequence arranged in the initial scheduling order.

[0055] The process of generating the concentrator collection task flow sequence in S1 is as follows:

[0056] The data collection time of high-priority tasks is calibrated within the concentrator using a unified time synchronization protocol. The concentrator serves as the master clock node, periodically broadcasting synchronization signals to all intelligent collection terminals within its jurisdiction, and using IEEE 1588 or SNTP time synchronization protocols to calibrate the terminal system clocks. For uploaded high-priority task collection data, the original timestamp field is extracted, and combined with the synchronized local clock, the time drift is calculated and write-back calibration is performed. When the drift exceeds the preset threshold, the time resynchronization mechanism is triggered to ensure the accuracy and consistency of the collection time, thereby ensuring a unified timing reference benchmark for subsequent priority calculation and sorting of different high-priority tasks.

[0057] A task semantic parsing algorithm is used to pre-process the description information of high-priority tasks in real time, extract the emergency level label, task time limit and bandwidth sensitivity parameter, construct a priority comprehensive scoring function, score and sort the high-priority tasks, and form a continuous concentrator collection task flow sequence; the business description information of high-priority tasks, including alarm type, event level, meter category and task triggering method, is subjected to text parsing and semantic label recognition, and the pre-trained semantic classification is used to determine the emergency category of the task and map it to the emergency level label; then, the remaining tolerable delay is calculated based on the latest completion time allowed by the task minus the current system time and quantified as the task time limit; further, the sensitivity of the task to the uplink bandwidth, i.e., the bandwidth sensitivity parameter, is evaluated based on indicators such as historical data upload rate, data packet size and link occupancy rate; the above three types of features are assigned weights respectively to construct a priority comprehensive scoring function. Emergency Level Label Task time limit Bandwidth sensitivity parameter, where 、 、 All high-priority tasks are scored and ranked based on the weight of the emergency level label, the weight of the task time boundary, and the weight of the bandwidth sensitivity parameter. When multiple tasks have the same or similar scores, they are fine-tuned with reference to the calibrated timestamps, and the early arriving tasks are executed first, ultimately forming a concentrator acquisition task flow sequence with a clear priority arrangement relationship.

[0058] S2: The concentrator collects task flow sequences and tracks the logical associations of the cross-task dependency chains. Combining the historical interference trajectories of high-priority tasks, the deferrable windows of low-priority tasks, and the link conflict frequency, a multi-dimensional scheduling feature vector reflecting the task importance gradient is constructed.

[0059] The process of tracking the cross-task dependency chain logic association of the concentrator collection task flow sequence in S2 is as follows:

[0060] Based on the data transmission path and triggering order of each task in the concentrator acquisition task flow sequence, identify the timing dependency and resource coupling relationship between tasks, and construct a dependency graph that reflects the order of task scheduling and resource association; for each task node in the concentrator acquisition task flow sequence, record its data source node (such as smart meter number, concentrator interface), upload path, and task trigger source (such as timed trigger, event trigger); by comparing task trigger time and shared link status, analyze whether there is a "first execution-later execution", "shared bandwidth", or "cache occupancy" dependency relationship between tasks. If there is a dependency, establish a directed edge from upstream to downstream in the dependency graph. Nodes represent task entities, and edges represent resource and timing coupling between tasks, thereby constructing an initial dependency graph that reflects the task scheduling order and resource association relationship;

[0061] Combined with the historical task priority change trajectory, link transmission delay and conflict frequency, the response frequency and weight strength of each task node in the dependency graph in different scheduling cycles are calculated, and a weighted dependency graph reflecting the importance of the task and the degree of dependency is obtained; the recent historical scheduling records of the concentrator are called to extract the actual number of responses of each task node in the dependency graph in multiple scheduling cycles and calculate its scheduling frequency; at the same time, the average transmission delay of the link bandwidth occupied by each node and the number of conflict retries with other tasks are counted, and these indicators are combined to calculate the weight strength of the node. The response frequency and weight strength are used as attribute values ​​to measure the importance of the task and the degree of dependency, and are attached to the nodes and edges in the dependency graph, thereby forming an information-enhanced weighted dependency graph with node weights and edge weights;

[0062] Based on the weighted dependency graph, the task nodes are arranged in chronological order, and a dynamic adjustment model based on time series analysis is used to track the weight fluctuations between adjacent nodes in the dependency graph. By identifying the priority transmission, response delay changes and resource competition trends between nodes, the logical association trajectory of the cross-task dependency chain in which the dependency relationship evolves over time is output; the weighted dependency graph is expanded according to the task occurrence time to form dependency graph snapshots under different time windows; a time series modeling algorithm is used to monitor the weight changes of adjacent nodes in the graph snapshots. By analyzing the rising and falling trends of node weights, the fluctuations of edge response delays and the changes in the number of conflict retries, it is determined whether there is a priority transmission phenomenon or a resource competition amplification trend between tasks; finally, the node weight and edge weight trajectories of the graph structure that change over time are output as the logical association trajectory of the cross-task dependency chain.

[0063] The process of constructing the multi-dimensional scheduling feature vector reflecting the task importance gradient in S2 is:

[0064] The task nodes in the cross-task dependency chain logical association trajectory are mapped to a multidimensional feature space, integrating the urgency level label, task time limit, and bandwidth sensitivity parameter. The output cross-task dependency chain logical association trajectory is parsed, and the task nodes involved in the trajectory are extracted as mapping objects. For each task node, the urgency level label (such as level 1 alarm, level 2 exception), the time limit determined by the task completion time limit (such as ≤30s, ≤5min), and the bandwidth sensitivity parameter (calculated based on the task data volume, upload cycle, and link traffic model) are read from the task description information and real-time monitoring data. The three task attributes are defined as mapping feature dimensions. By establishing a multidimensional feature space, each task node is mapped to a feature coordinate point in the space.

[0065] Extract the scheduling response frequency, task interaction pattern and historical priority change trend during task execution to form a multi-dimensional statistical parameter set; retrieve the execution records of each task in multiple scheduling cycles from the concentrator scheduling system, and count the number of times the task is triggered by scheduling per unit time to obtain the scheduling response frequency; based on the dependency chain topology between tasks, analyze the interactive behavior of tasks during the scheduling process (such as whether they frequently share links with other tasks, whether they are triggered by a certain type of task), and use graph structure analysis and co-occurrence statistical models to extract task interaction patterns; use historical priority sorting records to calculate the magnitude of changes and trend indicators of task priorities in multiple schedulings, reflecting the fluctuations in their importance in historical scheduling, thereby forming a multi-dimensional statistical parameter set such as scheduling response frequency, task interaction pattern and historical priority change trend;

[0066] By processing the multidimensional statistical parameter set through normalization and dimensionality reduction techniques, a multidimensional scheduling feature vector reflecting the task priority and importance gradient is obtained.

[0067] S3: Based on the multi-dimensional scheduling feature vector, a hierarchical priority aggregation algorithm is used to extract the evolution trajectory of high-priority tasks' preemption of bandwidth and acquisition time slots. By identifying the degree of deviation between the trajectory distribution and the initial scheduling order, potential scheduling misorder candidate tasks are detected.

[0068] The hierarchical priority aggregation algorithm is used in S3 to extract the preemptive evolution trajectory of bandwidth and acquisition time slots of high-priority tasks:

[0069] Based on the constructed multi-dimensional scheduling feature vector, the scheduling performance of each high-priority task in the historical scheduling record is mapped to the feature space, and the hierarchical priority aggregation algorithm is used to perform cluster analysis on the mapped task feature data, so as to identify the distribution pattern of the bandwidth and acquisition time slot preemption behavior of tasks at different priority levels; the generated multi-dimensional scheduling feature vector is called as the input feature set, the feature vector is matched with the historical scheduling record (including scheduling results, resource usage, and scheduling timestamps), and a hierarchical priority aggregation algorithm (such as the priority hierarchical aggregation method based on density clustering) is used to cluster tasks with similar distribution of different feature characteristics. Multiple priority aggregation levels are divided according to the urgency of the tasks in the cluster, the amount of resources occupied, and the differences in scheduling results. The higher the level, the stronger the ability to preempt bandwidth and acquisition time slots, forming a clustering result representing the scheduling performance of task groups at different priority levels;

[0070] A time series prediction method is introduced to dynamically track the evolution of the priority ranking of high-priority tasks within continuous scheduling cycles, extract the changing trends of task scores within each cycle, and construct a task score fluctuation trajectory. For nodes identified as high-priority tasks, their comprehensive score values ​​and ranking position information are counted in different scheduling cycles. A time series modeling method based on a sliding time window is used to predict and trend-fit the score sequence and ranking sequence, obtaining a trend curve reflecting the change of task scores over time. The predicted results are compared with the actual scheduling results to obtain the score offset between different cycles, thereby forming a task score fluctuation trajectory to reflect the dynamic law of its priority change.

[0071] The resource preemption intensity index of the task within the scheduling cycle is calculated by combining the task ranking deviation and the change in the concentrator bandwidth occupancy rate in each time period. The change in the score ranking between adjacent scheduling cycles in the priority fluctuation trajectory is used as the ranking deviation index. The uplink bandwidth occupancy rate (such as Mbps utilization rate and connection time slot occupancy rate) of the corresponding task in each scheduling cycle is monitored on the concentrator. The predefined resource preemption intensity calculation formula is:

[0072]

[0073] Where F is the resource preemption intensity index, is the magnitude of the change in ranking, is the change in bandwidth occupancy, 、 are the weight coefficients of the magnitude of the change in ranking and the weight coefficient of the change in bandwidth occupancy, respectively, to quantify the intensity of task preemption behavior and obtain a sequence of resource preemption intensity indicators;

[0074] By integrating task score fluctuation trajectories and resource preemption intensity indicators, we reconstruct the resource contention paths of tasks during dynamic scheduling, forming a preemption evolution trajectory. Taking the scheduling period as the horizontal axis and the task score fluctuation trajectory as the vertical dominant characteristic line, combined with the resource preemption intensity indicator as the representation of resource competition, we fuse the two to generate a continuous sequence of "time-score-resource occupancy" triplets. Based on this sequence, we annotate the start time, duration, and variation of task bandwidth / collection time slot preemption, constructing a temporal path that reflects the progressive resource preemption characteristics in task scheduling. This ultimately forms a traceable trajectory of bandwidth and collection time slot preemption evolution for high-priority tasks.

[0075] The process of detecting potential mis-scheduling candidate tasks in S3 is as follows:

[0076] The preemption evolution trajectory extracted by the hierarchical priority aggregation algorithm is compared with the reference trajectory in the pre-established normal scheduling trajectory library for structural similarity. The obtained high-priority task preemption evolution trajectory is represented as time-series structured trajectory data, including features such as the scheduling order of task nodes, resource usage changes, and bandwidth preemption behavior. The reference trajectory recorded during the historical stable operation of scheduling is retrieved from the normal scheduling trajectory library, and the reference trajectory is standardized to construct a scheduling trajectory reference model. The structural similarity algorithm is used to match the trajectory sequence to be tested with the reference trajectory moment by moment, outputting a similarity matrix and local offset index. The overall structural deviation is used to preliminarily screen out time periods where scheduling anomalies may occur.

[0077] A graph structure comparison algorithm is used to identify critical task areas with structural offsets, node dislocations, or connection anomalies based on the task nodes and dependency relationships in the preemption evolution trajectory. The preemption evolution trajectory within each scheduling period is converted into a task dependency graph structure, where tasks are nodes and the scheduling order and resource preemption sequence between nodes are directed edges. A graph structure matching algorithm is used to compare the trajectory graph to be tested with the reference trajectory graph node by node, marking task nodes with changes in scheduling order as structural offset nodes. The graph is further analyzed for newly added or disappeared edges (connection anomalies) and node position misalignment (node ​​dislocation) to identify the task areas with the most significant offsets in the scheduling trajectory as critical task areas.

[0078] Combined with the preset deviation threshold judgment mechanism, abnormal task sequence segments in the critical task area are identified, and the identified abnormal task sequence segments are marked as potential scheduling misorder candidate tasks; if any deviation indicator in the critical task area is greater than the preset threshold, the task sequence segment in the critical task area is marked as an abnormal task sequence segment; if any deviation indicator in the critical task area is less than or equal to the preset threshold, the task sequence segment in the critical task area is marked as a non-abnormal task sequence segment; these abnormal task sequence segments are output as potential scheduling misorder candidate tasks.

[0079] S4: Prioritize the scheduling out-of-order candidate tasks and the associated low-priority task intervals, and integrate the high-urgency nodes, deferrable nodes, and bandwidth scheduling nodes in the interval to construct a local task logic graph with a priority weight gradient.

[0080] The process of prioritizing the scheduling out-of-order candidate tasks and the associated low-priority task intervals in S4 is as follows:

[0081] Based on the key task nodes in the potential scheduling misorder candidate tasks, determine the core time period when the scheduling misorder candidate tasks occur; read the abnormal task sequence segment marked as the potential scheduling misorder candidate task, count the actual start time, running duration and resource usage peak of each task node in the sequence in the scheduling trajectory, and select the task node with the largest sorting offset or the highest resource contention intensity as the key task node; based on the scheduling trigger time of the key task node, backtrack N scheduling cycles (such as 2 to 3 scheduling cycles before and after) to establish a time window, and use this time window as the core time period when the misorder event occurs;

[0082] During the core time period, based on the logical dependency, resource sharing, and scheduling transmission relationships between the key task node and surrounding task nodes, low-priority tasks within the impact range are extracted and mapped to form candidate task intervals. During the determined core time period, the dependency chain topology containing the out-of-order candidate tasks is analyzed, expanding from the key task node upstream and downstream along the logical dependency direction, identifying low-priority tasks with triggering, resource sharing, and timing dependency relationships, and obtaining the set of logical nodes associated with the key node. Based on the position of different nodes in the dependency chain and the dependency distance from the core node, they are divided into a high-urgency layer (dependency distance of 1), a secondary-urgency layer (distance of 2), and a deferrable layer (distance ≥ 3), forming a hierarchical candidate task interval structure.

[0083] Combined with the resource occupation strategy of each candidate node, the priority level of each task node in the candidate task interval is divided to complete the priority classification; the internal scheduling strategy model of the concentrator is called to quantitatively evaluate the impact of different layered tasks on resource occupation based on the actual allocation weights of tasks in the candidate interval in the acquisition window, cache space and upload link; based on the resource conflict probability, resource sensitivity and acquisition delay tolerance, the priority weight of each node is adjusted, and the high-urgency layer tasks are marked as priority protection objects, the sub-urgency layer tasks are marked as adjustable objects, and the delayable layer tasks are marked as delayed execution objects to form a clear priority classification strategy; at the same time, the scheduling boundaries of each task on the resource allocation table are defined.

[0084] The process of constructing a local task logic graph with priority weight gradient in S4 is as follows:

[0085] Extract key task nodes within the candidate task interval, including high-urgency nodes, deferrable nodes, and bandwidth-schedulable nodes, to form a multi-attribute heterogeneous node set. Read the task set within the prioritized candidate task interval and, based on the priority classification strategy, categorize the tasks into high-urgency nodes (priority-guaranteed tasks), deferrable nodes (tasks that can be postponed), and bandwidth-schedulable nodes (tasks that occupy or adjust link resources). For each task node, extract its task attribute parameters (such as task type, time constraint, bandwidth sensitivity) and required acquisition resource information (acquisition window length, cache space requirements, bandwidth peak occupancy, etc.), representing the node as a heterogeneous element with multi-attribute labels.

[0086] Based on historical scheduling records, task dependencies, and real-time scheduling feedback, a set of directed edges between nodes is constructed to form a multimodal task graph that reflects the actual scheduling execution logic. Based on the historical scheduling sequence of the concentrator, the triggering relationship, resource sharing relationship, and conflict preemption relationship between task nodes are counted, and directed edges between nodes are constructed in the logical manner of "first execution → last execution", "resource preemption → preempted", and "trigger → response". Different edge attributes are assigned to edges of different relationship types (such as triggering edges, preemption edges, and causal dependency edges), and the edge weights are updated by integrating real-time scheduling feedback data so that the edge weights reflect the actual scheduling priority, resource tilt, and conflict intensity of the task in real time. In this way, a multimodal task graph containing multiple types of edges and multiple attribute nodes is established to ensure that the real scheduling behavior connection between tasks in the candidate interval can be fully represented.

[0087] Graph neural networks are used to mine key topological patterns in multimodal task graphs, identify priority transmission anomalies and bandwidth preemption hotspots, locate key task nodes that have the greatest impact on scheduling order, and construct a local task logic graph. A graph neural network model is used to input the multimodal task graph, and high-dimensional feature representations of each task node in the graph structure are obtained through node embedding learning. The attention mechanism is used to perform weight analysis on the input and output edges of the nodes and their locations, and nodes with sudden weight changes in the priority transmission path are screened to identify priority transmission anomalies. At the same time, based on the node embedding features, areas with high bandwidth conflict frequency and concentrated preemption edge weights are detected as bandwidth preemption hotspots, and the nodes that most significantly affect the scheduling logic are identified as key task nodes. Different weight gradients are assigned according to the intensity of their impact on scheduling behavior, and finally a local task logic graph with priority weight gradients is formed.

[0088] S5: Based on the local task logic graph, combined with the resource gap of high-priority imbalanced nodes and the task conflict oscillation trend under real-time bandwidth fluctuations, dynamic intervention judgment is made on the scheduling evolution path.

[0089] The dynamic intervention judgment process for the scheduling evolution path in S5 is as follows:

[0090] Based on the constructed local task logic graph, the key task nodes with the greatest impact on the scheduling order are extracted. Combined with the dependency relationship between the key task nodes and upstream and downstream tasks and the resource occupancy intensity, the resource bottleneck nodes and key evolution paths in the scheduling chain are located. The generated local task logic graph is called to read the key task nodes marked as high-weight in the graph, and query their predecessor (upstream trigger) task nodes and successor (downstream dependency) task nodes in the scheduling dependency chain. The input and output edge attributes of the key task nodes are analyzed to identify their critical path position and resource occupancy behavior in the scheduling process. Resource consumption indicators such as the bandwidth required by the key nodes during peak hours, the acquisition window duration, and the cache space are further calculated, and compared with the currently allocable resources of the concentrator. When the resources required by the key node exceed the threshold or there is insufficient competitive resources, the node is marked as a resource bottleneck node.

[0091] Access real-time task scheduling data streams, continuously monitor conflict frequencies, preemption events, and bandwidth oscillations between tasks on critical paths, and comprehensively evaluate the stability and risk propagation of scheduling evolution paths. Access the concentrator in real time to collect task scheduling status data streams, including task congestion rates, conflict retry counts, preemption frequencies, and bandwidth occupancy fluctuations. Continuously monitor the scheduling evolution paths where critical task nodes are located, and use a sliding time window mechanism to calculate conflict oscillation trends. If the number of consecutive bandwidth conflicts on a certain path exceeds a preset threshold or the conflict interval is less than a set time interval, the path is determined to be unstable. Simultaneously, the node chain on the path is tracked to evaluate the possible transmission direction and propagation speed of the conflict in the dependency chain, determine whether the risk is spreading to downstream key nodes, and thus determine whether the scheduling evolution path has instability, deviation, or diffusion risks.

[0092] When the scheduling evolution path shows signs of instability, a dynamic scheduling intervention strategy is designed based on the node priority weight and resource demand, by adjusting the task execution order, optimizing the resource allocation ratio, and coordinating the dependencies between tasks; after determining that a certain scheduling evolution path has a resource bottleneck or conflict spread risk, different levels of scheduling intervention measures are selected according to the intervention strategy library, including: (1) Sequence adjustment strategy: re-arrange the execution order of key nodes and their downstream dependent nodes in the local task logic map, so that high-urgency tasks take priority in grabbing resources; (2) Resource optimization strategy: temporarily increase the available bandwidth, acquisition window or cache resources of key nodes, or migrate some delayable node tasks to the backup link for execution to achieve resource reallocation; (3) Dependency coordination strategy: modify the trigger threshold or dependency chain configuration between tasks to reduce the chain conflict caused by strong dependencies between tasks. According to the adjusted strategy, the local logic map and scheduling timing configuration are updated to achieve dynamic correction and real-time intervention of the scheduling evolution path, ensuring that high-priority tasks can obtain stable scheduling guarantees and preventing the spread of chain misordering events.

[0093] Example 2: Figure 2 As shown, the concentrator data collection and analysis system based on big data includes:

[0094] Task identification module: Captures high-priority tasks in real time, analyzes task semantic labels, timeliness boundaries, and bandwidth sensitivity, and generates preliminary acquisition task flow sequences;

[0095] Feature construction module: This module tracks the dependency chain logic between tasks and combines historical task interference information and link conflict frequency to construct a multi-dimensional scheduling feature vector.

[0096] Out-of-order detection module: This module analyzes the preemption trajectory of high-priority tasks based on scheduling feature vectors, identifies scheduling order deviations, and screens candidate tasks for scheduling out-of-order tasks.

[0097] Graph generation module: This module integrates the node information of out-of-order candidates and low-priority task intervals to construct a local task logic graph with weight gradients.

[0098] Path intervention module: Combines resource gaps and conflict trends to dynamically determine scheduling path adjustment plans.

[0099] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0100] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for analyzing data collected by a concentrator based on big data, characterized in that: The following steps are involved: High-priority tasks are captured in real time, and based on the task semantic parsing algorithm, emergency level labels, task timeliness boundaries, and bandwidth sensitivity parameters are extracted. A priority comprehensive scoring function is constructed to sort high-priority tasks and generate a concentrator acquisition task flow sequence. The process of generating the concentrator collection task flow sequence is as follows: The data collection time of high-priority tasks is calibrated within the concentrator through a unified time synchronization protocol; A task semantic parsing algorithm is used to pre-process the description information of high-priority tasks in real time, extracting emergency level labels, task timeliness boundaries, and bandwidth sensitivity parameters. A priority comprehensive scoring function is constructed to score and rank high-priority tasks, forming a continuous concentrator acquisition task flow sequence. The concentrator collects task flow sequences and tracks the logical association of cross-task dependency chains. Combining historical high-priority task interference trajectories, low-priority task deferrable windows, and link conflict frequencies, a multi-dimensional scheduling feature vector reflecting the task importance gradient is constructed. The process of tracking the cross-task dependency chain logic of the concentrator acquisition task flow sequence is as follows: Based on the data transmission path and triggering order of each task in the task flow sequence collected by the concentrator, the timing dependency and resource coupling relationship between tasks are identified, and a dependency graph reflecting the order of task scheduling and resource association is constructed; Combining historical task priority change trajectories, link transmission delays, and conflict frequencies, we calculate the response frequency and weight strength of each task node in the dependency graph within different scheduling cycles, and obtain a weighted dependency graph that reflects the importance of tasks and the degree of dependency. Based on the weighted dependency graph, task nodes are arranged in chronological order. A dynamic adjustment model based on time series analysis is used to track the fluctuations in weights between adjacent nodes in the dependency graph. By identifying priority transmission, response latency changes, and resource contention trends between nodes, the system outputs the logical association trajectory of the cross-task dependency chain that evolves over time. The process of constructing a multi-dimensional scheduling feature vector that reflects the task importance gradient is as follows: Mapping the task nodes in the cross-task dependency chain logical association trajectory to a multi-dimensional feature space, integrating the urgency level label, task time limit and bandwidth sensitivity parameters; Extract the scheduling response frequency, task interaction mode and historical priority change trend during task execution to form a multi-dimensional statistical parameter set; By processing the multidimensional statistical parameter set through normalization and dimensionality reduction techniques, a multidimensional scheduling feature vector reflecting the task priority and importance gradient is obtained; Based on the multi-dimensional scheduling feature vector, a hierarchical priority aggregation algorithm is used to extract the preemption evolution trajectory of high-priority tasks on bandwidth and acquisition time slots. By identifying the degree of deviation between the trajectory distribution and the initial scheduling order, potential scheduling misorder candidate tasks are detected. The hierarchical priority aggregation algorithm is used to extract the evolution trajectory of bandwidth and acquisition time slot preemption of high-priority tasks: Based on the constructed multi-dimensional scheduling feature vector, the scheduling performance of each high-priority task in the historical scheduling record is mapped into the feature space. A hierarchical priority aggregation algorithm is then used to perform cluster analysis on the mapped task feature data, thereby identifying the distribution patterns of bandwidth and acquisition time slot preemption behaviors of tasks at different priority levels. A time series prediction method is introduced to dynamically track the evolution of the priority ranking of high-priority tasks in continuous scheduling cycles, extract the changing trend of task scores in each cycle, and construct the task score fluctuation trajectory; Combining the task sorting deviation with the change in concentrator bandwidth occupancy in each time period, the resource preemption intensity index of the task within the scheduling cycle is calculated; By integrating task score fluctuation trajectories and resource preemption intensity indicators, we can reconstruct the resource contention path of tasks during dynamic scheduling and form a preemption evolution trajectory. Prioritize candidate tasks for out-of-order scheduling, integrate high-urgency nodes, deferrable nodes, and bandwidth scheduling nodes within the interval, and construct a local task logic graph with a priority weight gradient. Based on the local task logic graph, high-priority nodes with unbalanced resource allocation within a specific scheduling cycle are identified, and the scheduling evolution path is generated by combining the task conflict oscillation trend under real-time bandwidth fluctuations.

2. The method for analyzing data collected by a concentrator based on big data according to claim 1, characterized in that: The process of detecting potential mis-scheduling candidate tasks is as follows: The preemption evolution trajectory extracted by the hierarchical priority aggregation algorithm is compared with the reference trajectory in the pre-established normal scheduling trajectory library for structural similarity; Using a graph structure comparison algorithm, we can identify key task areas with structural deviations, node disarray, or abnormal connections based on the task nodes and dependencies in the preemption evolution trajectory. Combined with the preset deviation threshold judgment mechanism, abnormal task sequence segments in the critical task area are identified, and the identified abnormal task sequence segments are marked as potential scheduling misorder candidate tasks.

3. The method for analyzing data collected by a concentrator based on big data according to claim 2, characterized in that: The process of prioritizing the scheduling out-of-order candidate tasks and the associated low-priority task intervals is as follows: Based on the key task nodes in the potential mis-scheduling candidate tasks, determine the core time period in which the mis-scheduling candidate tasks occur; During the core time period, based on the logical dependency, resource sharing, and scheduling transmission relationships between key task nodes and surrounding task nodes, low-priority tasks within the impact range are extracted and mapped to form candidate task intervals. Combined with the resource occupation strategy of each candidate node, the priority level of each task node in the candidate task interval is divided to complete the priority classification.

4. The method for analyzing data collected by a concentrator based on big data according to claim 3 is characterized in that: The process of constructing a local task logic graph with priority weight gradient is as follows: Extract key task nodes within the candidate task interval, including high-urgency nodes, deferrable nodes, and bandwidth scheduling nodes, to form a multi-attribute heterogeneous node set; Based on historical scheduling records, task dependencies, and real-time scheduling feedback, a directed edge set between nodes is constructed to form a multimodal task graph that reflects the actual scheduling execution logic. Graph neural networks are used to mine key topological patterns in multimodal task graphs, identify priority transmission anomalies and bandwidth preemption hotspots, locate key task nodes that have the greatest impact on scheduling order, and construct local task logic graphs.

5. The method for analyzing data collected by a concentrator based on big data according to claim 4 is characterized in that: The dynamic intervention judgment process for the scheduling evolution path is as follows: Based on the constructed local task logic graph, the key task nodes that have the greatest impact on the scheduling order are extracted. Combined with the dependency relationship between the key task nodes and upstream and downstream tasks and the resource occupancy intensity, the resource bottleneck nodes and key evolution paths in the scheduling chain are located. Access real-time task scheduling data streams to continuously monitor conflict frequency, preemption events, and bandwidth oscillation between tasks on critical paths, and comprehensively evaluate the stability and risk propagation of scheduling evolution paths. When the scheduling evolution path shows signs of instability, a dynamic scheduling intervention strategy is designed based on the node priority weights and resource requirements to adjust the task execution order, optimize the resource allocation ratio, and coordinate the dependencies between tasks.

6. A concentrator data collection and analysis system based on big data, applied to the method according to any one of claims 1 to 5, characterized in that: include: Task identification module: Captures high-priority tasks in real time, analyzes task semantic labels, timeliness boundaries, and bandwidth sensitivity, and generates preliminary acquisition task flow sequences; Feature construction module: This module tracks the dependency chain logic between tasks and combines historical task interference information and link conflict frequency to construct a multi-dimensional scheduling feature vector. Out-of-order detection module: This module analyzes the preemption trajectory of high-priority tasks based on scheduling feature vectors, identifies scheduling order deviations, and screens candidate tasks for scheduling out-of-order tasks. Graph generation module: This module integrates the node information of out-of-order candidates and low-priority task intervals to construct a local task logic graph with weight gradients. Path intervention module: Combines resource gaps and conflict trends to dynamically determine scheduling path adjustment plans.

Images