A road traffic flow task state perception scheduling method and system

Abstract

The embodiment of the application provides a road traffic flow type task state perception scheduling method and system, and belongs to the technical field of task scheduling. The method comprises the following steps: acquiring a to-be-executed road traffic flow and an abnormal congestion analysis flow type job, constructing a flow type job execution graph, acquiring device state collection data of each computing node, determining node state parameters of each computing node and link state parameters of each data transmission edge in the flow type job execution graph; determining a target task node mapping scheme; executing the flow type job; and checking the target task node mapping scheme based on updated device state collection data in the execution process. According to the scheme, the node state parameters and the link state parameters are introduced to participate in the flow type task scheduling decision, and dynamic checking and rescheduling are realized in the execution process, so that the flow type job has higher execution stability and continuity in a weak network environment.

Description

Technical Field

[0001] This invention relates to the field of task scheduling technology, specifically to a road traffic flow-based task status perception scheduling method and system. Background Technology

[0002] With the continuous advancement of edge computing and the Internet of Things (IoT) technologies, data processing is gradually shifting from centralized architectures to distributed, near-source architectures. In scenarios such as industrial control, intelligent transportation, telemedicine, and satellite remote sensing, large amounts of continuously generated data require real-time analysis and response through streaming processing. Streaming engines are widely used to address these needs, with Apache Flink (an open-source stream processing framework) serving as a prime example. Providing high-throughput, low-latency data processing capabilities, it has become one of the mainstream technology choices. In road traffic flow and congestion analysis scenarios, roadside cameras, millimeter-wave radar, geomagnetic detectors, loop detectors, roadside units, and floating car data sources continuously generate traffic video streams, vehicle detection data, vehicle trajectory data, and road segment status data. This data typically requires streaming processing stages, including traffic perception data access, data cleaning and time synchronization, vehicle target detection, vehicle trajectory extraction, traffic flow statistics, road segment status aggregation, abnormal congestion identification, and alarm output.

[0003] In actual deployment, the aforementioned streaming processing systems often operate in edge nodes or heterogeneous distributed environments. In road traffic scenarios, this manifests as operation in roadside cabinets, intersection edge nodes, roadside base stations, regional traffic edge centers, or traffic management center nodes. These environments commonly suffer from unstable network conditions. For example, in scenarios such as roadside edge networks, intersection wireless backhaul links, regional traffic edge networks, and traffic management center access links, network latency fluctuates significantly, available bandwidth is limited, and packet loss rates exhibit considerable uncertainty. In such weak network environments, the data transmission and task execution processes within the streaming operation are highly dependent on network conditions. When peak hours, sudden traffic accidents, road construction, severe weather, or temporary traffic control lead to a rapid increase in traffic perception data at certain intersections or road sections, a decrease in bandwidth, increased latency, or a rise in packet loss rates in the corresponding links can easily lead to a decrease in task execution efficiency, and even the loss of traffic video frames, vehicle detection results, trajectory point sequences, or road segment statistical data. In existing technologies, the scheduling of streaming tasks primarily relies on resource-based decisions. Taking Apache Flink as an example, its native scheduler allocates tasks based on slot resources, focusing mainly on static resource metrics such as CPU and memory, lacking modeling of network conditions. Similarly, the Flink on Kubernetes solution manages resources and deploys clusters through the Kubernetes container orchestration system, but the Kubernetes scheduler also primarily makes scheduling decisions based on computing resources, lacking awareness of dynamic network metrics such as latency, bandwidth changes, and packet loss. For road traffic flow and congestion analysis, the deployment location of task nodes such as vehicle target detection, vehicle trajectory extraction, traffic flow statistics, road segment status aggregation, and congestion identification directly affects the transmission latency of traffic video data, vehicle target bounding box data, trajectory point data, and road segment statistical results between different computing nodes. Existing schedulers struggle to adapt task node deployment relationships based on the aforementioned traffic data transmission status.

[0004] Existing scheduling mechanisms primarily focus on resource availability, lacking consideration for constraints on data transmission processes. In streaming operations, such as road traffic flow and congestion analysis, traffic perception data access nodes, vehicle target detection nodes, vehicle trajectory extraction nodes, traffic flow statistics nodes, road segment status aggregation nodes, congestion identification nodes, and alarm output nodes are connected via data streams. If the data transmission path is restricted, it directly impacts overall processing performance, causing delays in traffic flow statistics, queue length estimation, road segment status aggregation, or congestion alarm results. Existing scheduling methods do not model this aspect. When introducing custom scheduling strategies, modifications to Flink's internal scheduling logic are typically required, involving source code layer modifications. This results in complex engineering implementation, high maintenance costs, and hinders rapid adaptation to different intersection, road segment, or regional traffic analysis scenarios.

[0005] In the existing architecture, task allocation is basically determined during the job startup phase, making it difficult to dynamically adjust according to environmental changes during operation. In particular, when network conditions fluctuate, it is impossible to migrate affected tasks accordingly. Although some systems can obtain node resource usage information, there is a lack of continuous monitoring and unified modeling mechanisms for key network indicators in weak network environments, such as network latency, bandwidth, and packet loss rate, making it difficult to support dynamic scheduling based on device status.

[0006] There is a lack of effective collaboration between the streaming computing engine and the container orchestration platform. Scheduling decisions and resource deployment processes are disconnected, failing to form a complete link from state awareness to scheduling decisions to execution, making it difficult to accurately apply scheduling results to specific node deployments. In road traffic edge computing platforms, this problem further manifests as follows: the scheduling results of traffic analysis tasks are difficult to accurately apply to roadside computing nodes, intersection computing nodes, regional edge computing nodes, or traffic management center computing nodes, and the actual deployment location of traffic analysis tasks cannot be adjusted in a timely manner based on changes in the status of equipment and links along the road.

[0007] Based on the above, it is evident that existing technologies have not yet solved the problem of how to incorporate device network status into the streaming task scheduling process and dynamically adjust it in conjunction with task execution relationships in a weak network edge computing environment for road traffic flow and abnormal congestion analysis. In particular, during the execution of streaming operations for road traffic flow and abnormal congestion analysis, the data transmission path between task nodes is also affected by network status. Existing methods lack modeling and scheduling mechanisms for data transmission links, making it difficult to guarantee overall execution stability. Therefore, how to simultaneously consider the data transmission link status of roadside, intersection, area edge, or traffic management center computing nodes and traffic analysis tasks during the execution of streaming operations for road traffic flow and abnormal congestion analysis, and construct a scheduling method capable of dynamically adjusting the mapping relationship between task nodes, thereby improving the stability and continuity of streaming tasks for road traffic flow statistics and abnormal congestion identification, and reducing the delay in abnormal congestion identification and alarm output, has become an urgent problem to be solved in this field. Summary of the Invention

[0008] The purpose of this invention is to provide a road traffic flow task status awareness scheduling method and system, so as to at least solve the problem that the prior art does not include the device network status and inter-task data transmission links in the scheduling decision, resulting in insufficient stability of flow task execution in weak network environments.

[0009] To achieve the above objectives, the first aspect of the present invention provides a road traffic flow task state perception scheduling method, the method comprising: acquiring a road traffic flow and abnormal congestion analysis flow operation to be executed and constructing a flow operation execution graph; simultaneously acquiring device state collection data of each computing node deployed at the roadside, intersection, regional edge, or traffic management center, and determining the node state parameters of each computing node and the link state parameters of each data transmission edge in the flow operation execution graph; wherein, the flow operation execution graph includes multiple task nodes for processing road traffic perception data and data transmission edges between task nodes; determining a target task node mapping scheme based on the flow operation execution graph, the node state parameters, and the link state parameters; deploying each task node in the flow operation execution graph on the corresponding computing node based on the target task node mapping scheme and executing the flow operation; during execution, verifying the target task node mapping scheme based on the updated device state collection data, and updating the task node mapping scheme and continuing to execute the flow operation if preset conditions are not met.

[0010] Optionally, a streaming operation execution graph containing multiple task nodes and data transmission edges between nodes is constructed based on the streaming operation configuration data of road traffic flow and abnormal congestion analysis. The multiple task nodes include multiple nodes selected from traffic perception data access nodes, data cleaning and time synchronization nodes, vehicle target detection nodes, vehicle trajectory extraction nodes, traffic flow statistics nodes, road segment status aggregation nodes, abnormal congestion identification nodes, and alarm output nodes. Device status data collected from each computing node at the roadside, intersection, area edge, or traffic management center is acquired and aggregated according to the computing node identifier to form the original status data set corresponding to each computing node. Based on the original status data set corresponding to each computing node, network latency data, available bandwidth data, and packet loss rate data of that computing node are acquired, and node status parameters are determined accordingly. For each data transmission edge used to transmit road traffic perception data or traffic status analysis results, based on the original status data set of the computing node where the source task node and the original status data set of the computing node where the target task node are located, the corresponding network latency data, available bandwidth data, and packet loss rate data are acquired as link status parameters.

[0011] Optionally, determining the target task node mapping scheme based on the streaming job execution graph, the node status parameters, and the link status parameters includes: constructing multiple candidate task node mapping schemes based on the streaming job execution graph; for each candidate task node mapping scheme, determining the execution status evaluation value of each task node based on the node status parameters, and determining the transmission status evaluation value of each data transmission edge based on the link status parameters; determining the job status evaluation value of each candidate task node mapping scheme based on the execution status evaluation value of each task node and the transmission status evaluation value of each data transmission edge; and determining the target task node mapping scheme based on the job status evaluation value of each candidate task node mapping scheme.

[0012] Optionally, constructing multiple candidate task node mapping schemes based on the streaming job execution graph includes: determining the deployable order of each task node based on the dependency relationship of each task node in the streaming job execution graph; selecting at least one optional computing node from each computing node for each task node based on the deployable order, forming a candidate computing node set for the corresponding task node; and performing a combined mapping between each task node and each computing node based on the candidate computing node set corresponding to each task node to generate multiple candidate task node mapping schemes, wherein each candidate task node mapping scheme is used to describe the correspondence between each task node and each computing node.

[0013] Optionally, for each candidate task node mapping scheme, the execution status evaluation value of each task node is determined based on the node status parameters, and the transmission status evaluation value of each data transmission edge is determined based on the link status parameters. This includes: for each candidate task node mapping scheme, obtaining the node status parameters of the computing node based on the mapping relationship between each task node and the corresponding computing node, and determining the execution status evaluation value of each corresponding task node based on the obtained node status parameters and the traffic data processing load intensity of the corresponding task node. The traffic data processing load intensity is determined based on the road traffic perception data type, video frame rate, number of road segments participating in the statistics, and / or sliding time window; for each data transmission edge in the candidate task node mapping scheme, determining the transmission status evaluation value of each corresponding data transmission edge based on the link status parameters between the computing node where the source task node is located and the computing node where the target task node is located, and in combination with the traffic data output rate of the source task node. The traffic data output rate is determined based on the video bitrate, vehicle detection result output frequency, vehicle trajectory point update frequency, and / or road segment statistical result output frequency.

[0014] Optionally, determining the target task node mapping scheme based on the job status evaluation value of each candidate task node mapping scheme includes: obtaining the job status evaluation value corresponding to each candidate task node mapping scheme and comparing the job status evaluation values ​​of each candidate task node mapping scheme; determining the candidate task node mapping scheme with the optimal job status evaluation value as the target task node mapping scheme based on the comparison results; and when there are multiple candidate task node mapping schemes with the same job status evaluation value, comparing the sum of the link status parameters corresponding to each data transmission edge in each candidate task node mapping scheme and determining the candidate task node mapping scheme with the largest sum of link status parameters as the target task node mapping scheme.

[0015] Optionally, deploying each task node in the streaming job execution graph on the corresponding computing node based on the target task node mapping scheme and executing the streaming job includes: determining the target computing node corresponding to each task node based on the target task node mapping scheme and generating a corresponding task deployment instruction; creating a task execution unit corresponding to each task node on the corresponding target computing node according to the task deployment instruction, and loading each task node into the corresponding task execution unit; after each task execution unit completes registration, triggering each task node to execute the streaming job according to the target task node mapping scheme based on the dependency relationship of each task node in the streaming job execution graph.

[0016] Optionally, during execution, the target task node mapping scheme is validated based on updated device status data. This includes: during the execution of the road traffic flow and abnormal congestion analysis streaming operation, acquiring updated device status data for each computing node, and determining the node status parameters and link status parameters of each data transmission edge based on the updated device status data; determining the current execution status evaluation value of each task node based on the mapping relationship between each task node and computing node in the target task node mapping scheme, and determining the current transmission status evaluation value of each data transmission edge based on the link status parameters corresponding to each data transmission edge; determining the current operation status evaluation value based on the current execution status evaluation value of each task node and the current transmission status evaluation value of each data transmission edge; comparing the current operation status evaluation value with the operation status evaluation value corresponding to the determination of the target task node mapping scheme, and determining that the preset condition is not met when the current operation status evaluation value is less than a preset proportion of the operation status evaluation value corresponding to the determination of the target task node mapping scheme.

[0017] Optionally, when the preset conditions are not met, the task node mapping scheme is updated and the streaming job continues to be executed, including: reconstructing the candidate task node mapping scheme based on the updated node status parameters and link status parameters, and determining the updated target task node mapping scheme; determining the task nodes that need to be adjusted and their corresponding target computing nodes based on the updated target task node mapping scheme, and generating corresponding task adjustment instructions; creating corresponding task execution units on the target computing nodes according to the task adjustment instructions, and migrating the task nodes that need to be adjusted to the corresponding task execution units; after completing the task node migration, restoring the data processing relationship between each task node based on the streaming job execution graph, so as to continue executing the streaming job.

[0018] A second aspect of the present invention provides a road traffic flow-based task status perception and scheduling system. The system includes: a parameter acquisition unit, configured to acquire a road traffic flow and abnormal congestion analysis flow-based operation to be executed and construct a flow-based operation execution graph; simultaneously acquiring device status data collected from each computing node deployed at the roadside, intersection, area edge, or traffic management center; and determining node status parameters of each computing node and link status parameters of each data transmission edge in the flow-based operation execution graph; wherein the flow-based operation execution graph includes multiple task nodes for processing road traffic perception data and data transmission edges between task nodes; a scheme determination unit, configured to determine a target task node mapping scheme based on the flow-based operation execution graph, the node status parameters, and the link status parameters; an operation execution unit, configured to deploy each task node in the flow-based operation execution graph on the corresponding computing node based on the target task node mapping scheme and execute the flow-based operation; and an operation update unit, configured to verify the target task node mapping scheme based on updated device status data during execution, and update the task node mapping scheme and continue executing the flow-based operation if preset conditions are not met.

[0019] Through the above technical solution, this invention introduces equipment status data collection during the streaming operation scheduling process for road traffic flow and abnormal congestion analysis. It uniformly models the node status parameters of computing nodes at roadside, intersections, area edges, or traffic management centers, as well as the link status parameters corresponding to data transmission edges between task nodes. Based on the execution graph structure, it determines the task node mapping scheme, ensuring that scheduling decisions not only rely on computing resources but also reflect the real-time transmission capability of traffic perception data and traffic status analysis results. Furthermore, by continuously acquiring updated equipment status data during operation execution, the existing task node mapping scheme is validated, and updated when link or node status changes. This reduces the impact of traffic data congestion on traffic statistics, road segment status aggregation, and abnormal congestion identification in weak network environments, improving the continuity and stability of road traffic analysis task execution.

[0020] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0021] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of the steps of a road traffic flow-based task state awareness and scheduling method provided in one embodiment of the present invention; Figure 2 This is a detailed flowchart of step S20 of the road traffic flow-based task state perception and scheduling method provided in one embodiment of the present invention; Figure 3 This is a schematic diagram of the execution flow of a streaming task scheduling method based on device status awareness provided in one embodiment of the present invention. Figure 4 This is a schematic diagram of a streaming task execution and resource allocation process provided by one embodiment of the present invention; Figure 5 This is a schematic diagram of a streaming task rescheduling process based on device state changes provided by one embodiment of the present invention; Figure 6 This is a structural diagram of a road traffic flow-based task status perception and scheduling system provided in one embodiment of the present invention; Figure 7 This is an internal structural diagram of a computer device provided in one embodiment of the present invention. Detailed Implementation

[0022] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0023] like Figure 1 As shown, embodiments of the present invention provide a road traffic flow-based task state-aware scheduling method, the method comprising: Step S1: Obtain the road traffic flow and abnormal congestion analysis streaming operation to be executed and construct the streaming operation execution graph. At the same time, obtain the device status data of each computing node deployed on the roadside, intersection, regional edge or traffic management center, and determine the node status parameters of each computing node and the link status parameters of each data transmission edge in the streaming operation execution graph.

[0024] Specifically, a streaming operation execution graph containing multiple task nodes and data transmission edges between nodes is constructed based on the streaming operation configuration data of road traffic flow and abnormal congestion analysis. These multiple task nodes include multiple nodes selected from traffic perception data access nodes, data cleaning and time synchronization nodes, vehicle target detection nodes, vehicle trajectory extraction nodes, traffic flow statistics nodes, road segment status aggregation nodes, abnormal congestion identification nodes, and alarm output nodes. Device status data is collected from each computing node at the roadside, intersection, area edge, or traffic management center, and aggregated according to the computing node identifier to form the original status data set corresponding to each computing node. Based on the original status data set corresponding to each computing node, network latency data, available bandwidth data, and packet loss rate data are obtained for that computing node, and the node status parameters for each computing node are determined accordingly. For each data transmission edge used to transmit road traffic perception data or traffic status analysis results, based on the original status data set of the computing node where the source task node and the computing node where the target task node are located, the corresponding network latency data, available bandwidth data, and packet loss rate data are obtained as link status parameters.

[0025] In this embodiment of the invention, user-submitted streaming job configuration data is obtained. This streaming job configuration data includes at least data source configuration, operator processing logic, and result output configuration. In a road traffic flow and abnormal congestion analysis scenario, the data source configuration includes access configurations for roadside cameras, millimeter-wave radar, geomagnetic detectors, loop detectors, roadside units, or floating car data sources. The operator processing logic includes at least several processing logics from traffic perception data cleaning, time synchronization, vehicle target detection, vehicle trajectory extraction, traffic flow statistics, road segment status aggregation, and abnormal congestion identification. The result output configuration includes road segment status result output configurations or abnormal congestion alarm result output configurations. Based on the streaming job configuration data, the job processing flow is decomposed to obtain multiple interrelated task nodes. The corresponding data transmission relationships are determined based on the data processing order between the task nodes, thereby constructing a streaming job execution graph containing multiple task nodes and data transmission edges between nodes. In a specific application, taking road traffic flow and abnormal congestion analysis as an example, the data configuration for streaming operations can correspond to processing steps such as traffic perception data access, data cleaning and time synchronization, vehicle target detection, vehicle trajectory extraction, traffic flow statistics, road segment status aggregation, abnormal congestion identification, and alarm output. Each processing step forms a corresponding task node, and data transmission edges are established according to the data processing order of road traffic perception data and traffic status analysis results to form a complete execution graph structure.

[0026] After constructing the streaming job execution graph, device status data for each computing node is acquired. This device status data consists of network status data periodically collected by each computing node during operation. In one specific implementation, it can be acquired according to a preset collection period, such as 5 seconds. The device status data includes at least network latency data, available bandwidth data, and packet loss rate data, and is aggregated according to the computing node identifier to form the original status data set corresponding to each computing node, ensuring that the source of each data point is clear during subsequent calculations. For example, for roadside computing nodes deployed at key intersections, regional edge computing nodes deployed on adjacent road segments, and central computing nodes deployed in traffic management centers, their network latency data, available bandwidth data, and packet loss rate data are aggregated according to their computing node identifiers, enabling the subsequent scheduling process to distinguish the network carrying status of computing nodes at different road locations or different levels.

[0027] Based on the original state data set corresponding to each computing node, network latency data, available bandwidth data, and packet loss rate data of that computing node are obtained respectively, and node state parameters of each computing node are determined accordingly. The node state parameters characterize the network carrying capacity of the computing node at the current moment. In one specific implementation, these parameters can be obtained by performing unified dimension processing on the network latency data, available bandwidth data, and packet loss rate data, followed by comprehensive calculation. For example, when network latency is low, available bandwidth is high, and packet loss rate is low, the node state parameter value of the corresponding computing node is high; conversely, when network latency increases or packet loss rate increases, the node state parameter value decreases. This method ensures that the node state parameters accurately reflect the availability differences of each computing node in a weak network environment. In road traffic flow and abnormal congestion analysis scenarios, when a roadside computing node needs to handle traffic video stream access or vehicle target detection tasks, if the network latency of the roadside computing node increases, the available bandwidth decreases, or the packet loss rate increases, its node state parameters decrease to reflect the reduced adaptability of the computing node to continue handling high-frequency traffic perception data processing tasks.

[0028] Furthermore, for each data transmission edge in the streaming job execution graph, based on the original state data sets of the computing nodes where the source task node and the target task node are located, the corresponding network latency data, available bandwidth data, and packet loss rate data are obtained, and the link state parameters of each data transmission edge are determined accordingly. These link state parameters characterize the data transmission capability between the source and target task nodes, and their values ​​are related to the network connection quality between the nodes. In a specific application, when the network latency between the roadside computing node and the regional edge computing node is 42 milliseconds, the available bandwidth is 35 Mbps, and the packet loss rate is 0.3%, the corresponding link state parameter value is relatively high; when the network latency between the regional edge computing node and the traffic management center computing node increases to 160 milliseconds, the available bandwidth decreases to 8 Mbps, and the packet loss rate increases to 1.8% during morning and evening peak hours, the corresponding link state parameter value decreases. By determining the link state parameters of each data transmission edge, the data transmission capability in the streaming job execution graph is quantitatively expressed. For example, when the vehicle trajectory extraction node is deployed on the roadside computing node and the abnormal congestion identification node is deployed on the regional edge computing node, the link state parameters corresponding to the data transmission edge between the two can reflect whether the trajectory point sequence and road segment statistics can be stably transmitted to the abnormal congestion identification node.

[0029] Through the above process, both node status parameters reflecting the operational status of each computing node and link status parameters reflecting the data transmission capabilities between task nodes are obtained. This provides fundamental data support for determining the task node mapping scheme based on the joint constraints of node and link status. The entire process uses a streaming job execution graph as its structural carrier, mapping the device status acquisition data to the task node and data transmission edge layers, enabling scheduling decisions to simultaneously consider the computing node status and data transmission path status. In road traffic flow and abnormal congestion analysis scenarios, the above mapping relationship allows scheduling decisions to further correlate the real-time access location of traffic perception data, the deployment location of traffic analysis tasks, and the link status between computing nodes along the road, thereby providing a scheduling basis for reducing the backlog of traffic video frames, vehicle detection results, trajectory point sequences, or road segment statistical results in weak network links.

[0030] Step S2: Determine the target task node mapping scheme based on the streaming job execution graph, the node status parameters, and the link status parameters.

[0031] The process of determining the target task node mapping scheme based on the streaming job execution graph, the node state parameters, and the link state parameters involves comprehensively optimizing the mapping relationship between task nodes and computing nodes under the constraints of the execution graph structure. Specifically, based on the dependencies between task nodes in the streaming job execution graph, multiple candidate task node mapping schemes that satisfy execution order constraints are constructed, enabling each task node to be deployed according to a predetermined data processing relationship. For each candidate task node mapping scheme, the execution state of each task node on its corresponding computing node is determined by combining the node state parameters of each computing node, and the data transmission capability between task nodes is reflected by combining the link state parameters of each data transmission edge. Based on this, the execution state of the task nodes and the transmission state of the data transmission edges are comprehensively measured to obtain the job status evaluation result of each candidate task node mapping scheme. Finally, by comparing the job status evaluation results of each candidate task node mapping scheme, the candidate task node mapping scheme with the best evaluation result is selected as the target task node mapping scheme, ensuring that the determined mapping relationship satisfies the constraints of the execution graph while taking into account both the computing node state and data transmission capability. Figure 2 Step S2 includes the following steps: Step S21: Construct a mapping scheme for multiple candidate task nodes based on the streaming job execution graph.

[0032] Specifically, based on the dependencies between task nodes in the streaming job execution graph, the deployable order of each task node is determined; based on the deployable order, for each task node, at least one optional computing node is selected from each computing node to form a candidate computing node set for the corresponding task node; based on the candidate computing node set corresponding to each task node, each task node and each computing node are combined and mapped to generate multiple candidate task node mapping schemes, wherein each candidate task node mapping scheme is used to describe the correspondence between each task node and each computing node.

[0033] In this embodiment of the invention, the process of constructing multiple candidate task node mapping schemes based on the streaming job execution graph mainly involves generating a constrained combination of possible mapping relationships between task nodes and computing nodes under the constraints of the execution graph structure, rather than unordered enumeration. Specifically, firstly, based on the data dependencies between task nodes in the streaming job execution graph, a topology analysis is performed on the task nodes to obtain a deployable order that satisfies the dependency constraints. This deployable order is used to limit the deployment sequence of task nodes to avoid situations where upstream tasks have not been completed while downstream tasks have already started. In the scenario of road traffic flow and abnormal congestion analysis, the data dependencies can be reflected as follows: traffic perception data access nodes are deployed before data cleaning and time synchronization nodes; data cleaning and time synchronization nodes are deployed before vehicle target detection nodes; vehicle target detection nodes and vehicle trajectory extraction nodes are deployed before traffic flow statistics nodes; traffic flow statistics nodes and road segment status aggregation nodes are deployed before abnormal congestion identification nodes; and abnormal congestion identification nodes are deployed before alarm output nodes. Through the above deployable order, it can be ensured that road traffic perception data and traffic status analysis results are transmitted step by step according to the actual processing link.

[0034] After determining the deployable order, for each task node, at least one candidate compute node is selected from all compute nodes to form a candidate compute node set corresponding to that task node. This selection process is not arbitrary but constrained by node state parameters. For example, if a node has high network latency or a high packet loss rate, the corresponding compute node can be removed from the candidate set. For ease of description, in one specific implementation, a node selectability function can be defined as: ; in, Represents task node Can it be deployed on a compute node? superior, Represents a computing node The node state parameters, A preset threshold is used. By employing the above method, computing nodes that do not meet the network conditions can be excluded from the candidate pool, thereby narrowing the subsequent combination space. In road traffic flow and abnormal congestion analysis scenarios, if a roadside computing node's state parameters fall below a preset threshold due to wireless backhaul link congestion, that roadside computing node will no longer be considered as a potential computing node for traffic flow statistics, road segment status aggregation, or abnormal congestion identification, to avoid a backlog of subsequent traffic state analysis results at that node.

[0035] After obtaining the set of candidate computing nodes corresponding to each task node, the task nodes and computing nodes are combined and mapped based on the set of candidate computing nodes to generate multiple candidate task node mapping schemes. Specifically, for the set of task nodes... If the corresponding candidate computing node sets are respectively , Then, by combining these, multiple mapping schemes can be formed, for example... Each candidate task node mapping scheme describes the one-to-one correspondence between each task node and each computing node. For example, a candidate task node mapping scheme could describe traffic perception data access nodes as being deployed on roadside computing nodes at key intersections, vehicle target detection nodes as being deployed on intersection computing nodes with image processing capabilities, traffic flow statistics nodes as being deployed on regional edge computing nodes, and abnormal congestion identification nodes as being deployed on traffic management center computing nodes; alternatively, it could describe vehicle target detection nodes, vehicle trajectory extraction nodes, and traffic flow statistics nodes as being jointly deployed on the same roadside computing node to reduce cross-node transmission of vehicle target box data and trajectory point sequences.

[0036] To illustrate with a specific scenario, in the process of analyzing road traffic flow and abnormal congestion, assume there are three computing nodes. Node 1 is a roadside GPU computing node deployed at key intersections, mainly accessing traffic video streams and millimeter-wave radar data from that intersection. Its network latency with local traffic sensing devices is 18ms, available bandwidth is 80Mbps, and packet loss rate is 0.05%. Node 2 is a regional edge computing node deployed on adjacent road segments, mainly used to aggregate vehicle trajectory data and traffic flow statistics from multiple intersections. Its network latency with Node 1 is 42ms, available bandwidth is 35Mbps, and packet loss rate is 0.3%. Node 3 is a central computing node deployed at the traffic management center, mainly used for cross-regional traffic status aggregation and alarm linkage. It needs to go through a backhaul network with Node 1 or Node 2. During morning and evening peak hours, its network latency is 160ms, available bandwidth is 8Mbps, and packet loss rate is 1.8%. Under the above conditions, for vehicle target detection tasks that require real-time processing of traffic video frames, Node1 is prioritized in the candidate computing node set; for data cleaning and time synchronization tasks, Node1 and Node2 can be included in the candidate computing node set; for road segment status aggregation tasks, Node2 can be included in the candidate computing node set, and Node3 will be used as a backup computing node when its link status meets a preset threshold; for abnormal congestion identification tasks, when it needs to receive statistical results from multiple road segments, Node2 can be prioritized to reduce the latency caused by transmitting traffic statistics results to the central computing node. Multiple candidate task node mapping schemes can be generated by combining the candidate computing node sets corresponding to different task nodes.

[0037] Step S22: For each candidate task node mapping scheme, determine the execution status evaluation value of each task node based on the node status parameters, and determine the transmission status evaluation value of each data transmission edge based on the link status parameters.

[0038] Specifically, for each candidate task node mapping scheme, based on the mapping relationship between each task node and its corresponding computing node, the node state parameters of the computing node are obtained, and the execution state evaluation value of each corresponding task node is determined based on the obtained node state parameters; for each data transmission edge in the candidate task node mapping scheme, based on the link state parameters between the computing node where the source task node is located and the computing node where the target task node is located, the transmission state evaluation value of each corresponding data transmission edge is determined.

[0039] In this embodiment of the invention, for any candidate task node mapping scheme, based on the mapping relationship between each task node and its corresponding computing node, the node state parameters corresponding to the computing node are obtained, and the execution state evaluation value of the task node is determined accordingly. Considering that the node state parameters already comprehensively reflect factors such as network latency, available bandwidth, and packet loss rate, to avoid the impact of differences in node capability requirements for different task types, in one specific implementation, a task load factor can be introduced to correct the node state parameters. Specifically, task nodes can be defined. At the computing node The execution status evaluation value is: ; in, Represents task node At the computing node The execution status evaluation value on the device. Represents a computing node The node state parameters, Represents task node Calculate the load intensity. This is an adjustment coefficient. This expression method allows us to reflect the impact of high-load tasks on performance even when the node state is relatively good. Therefore, when a vehicle target detection node is mapped to a roadside computing node with high node state parameters but a large image processing load, its execution state evaluation value can simultaneously reflect the node network's carrying capacity and task processing pressure, rather than solely relying on the availability of the computing node.

[0040] Furthermore, for each data transmission edge in the candidate task node mapping scheme, the transmission state evaluation value of the corresponding data transmission edge is determined based on the link state parameters between the computing node where the source task node is located and the computing node where the target task node is located. Since the link state parameters already reflect the network conditions between nodes, in one specific implementation, the link state parameters can be combined with data transmission requirements for evaluation. For example, for the source task node... With the target task node The data transmission edges between them can be defined with a transmission state evaluation value of: ; in, This represents the transmission status evaluation value of the data transmission edge. This represents the link state parameters between the source task node and the target task node. This indicates the data output rate of the source task node. This is an adjustment coefficient. By introducing the data output rate factor, the link status evaluation not only reflects network conditions but also the impact of data scale on transmission capacity. Therefore, when the number of vehicles at an intersection increases during morning and evening rush hours, leading to a rise in the amount of trajectory point sequence data output by the vehicle trajectory extraction node, even if the corresponding link status parameters do not change significantly, the transmission status evaluation value of this data transmission edge will decrease due to the increased traffic data output rate. This prompts the subsequent scheduling process to prioritize candidate task node mapping schemes with better link status or higher co-location deployment.

[0041] In the analysis of road traffic flow and abnormal congestion, assuming a candidate task node mapping scheme, the vehicle target detection task is assigned to Node1, a roadside GPU computing node deployed at key intersections, with a node state parameter of 0.91; the data cleaning and time synchronization tasks are assigned to Node2, a regional edge computing node deployed on adjacent road segments, with a node state parameter of 0.74. In this case, the execution state evaluation value for the vehicle target detection task is high, indicating that Node1 can effectively handle real-time traffic video frame processing and vehicle target box generation tasks. The execution state evaluation value for the data cleaning and time synchronization tasks is lower than that for the vehicle target detection task, but still meets the preset execution requirements, indicating that Node2 can handle the aggregation, alignment, and timestamp correction processing of multi-source traffic perception data.

[0042] Meanwhile, for the data transmission edge between Node2 and Node3, since Node3 is the central computing node deployed in the traffic management center, the data transmission between Node2 and Node3 needs to go through the backhaul network. During morning and evening peak hours, network latency may increase, available bandwidth may decrease, and packet loss rate may increase. For example, if the network latency between Node2 and Node3 is 160ms, the available bandwidth is 8Mbps, and the packet loss rate is 1.8%, while the network latency between Node1 and Node2 is 42ms, the available bandwidth is 35Mbps, and the packet loss rate is 0.3%, then the transmission status evaluation value of the data transmission edge between Node2 and Node3 is significantly lower than that of the data transmission edge between Node1 and Node2. Therefore, when the traffic flow statistics task is deployed on Node2 and the abnormal congestion identification task is deployed on Node3, the stability of the transmission of statistical results such as road segment traffic, average speed, density, and queue length to the abnormal congestion identification task will be affected by the backhaul link status, and the overall evaluation of the corresponding candidate task node mapping scheme will be suppressed in subsequent calculations.

[0043] Step S23: Based on the execution status evaluation value of each task node and the transmission status evaluation value of each data transmission edge, determine the job status evaluation value of each candidate task node mapping scheme.

[0044] Specifically, the process of determining the job status evaluation value based on the execution status evaluation value of each task node and the transmission status evaluation value of each data transmission edge is a unified measure of the overall execution effect of the candidate task node mapping scheme. Since the aforementioned execution status evaluation value already reflects the degree of operational adaptation of the task node on the corresponding computing node, and the transmission status evaluation value reflects the data transmission capability between task nodes, combining the two can more accurately characterize the feasibility and stability of the candidate task node mapping scheme in actual operation. In the scenario of road traffic flow and abnormal congestion analysis, the overall execution effect is specifically reflected in whether task nodes such as traffic perception data access, vehicle target detection, vehicle trajectory extraction, traffic flow statistics, road segment status aggregation, abnormal congestion identification, and alarm output can be stably and continuously executed between computing nodes along the road, and whether traffic state analysis results such as vehicle target box data, trajectory point sequences, road segment flow, average speed, density, and queue length can be transmitted between task nodes in a timely manner.

[0045] Specifically, for any candidate task node mapping scheme, the execution status evaluation values ​​of all its corresponding task nodes can be denoted as a set. The corresponding data transmission edge transmission state evaluation value is denoted as a set. In one implementation, a weighted aggregation method can be used to determine the job status evaluation value of the candidate task node mapping scheme, expressed as: ; in, This represents the job status evaluation value of the candidate task node mapping scheme. These are weighting coefficients used to adjust the proportion of node execution capability and link transmission capability in the overall evaluation. This represents the number of task nodes. This represents the number of data transmission edges. By normalizing the evaluation values ​​of task nodes and links separately before weighting and combining them, we can avoid the excessive influence of any one type of evaluation value on the overall result. In road traffic flow and abnormal congestion analysis scenarios, when the processing pressure of traffic video frames is high, the influence of the task node execution status evaluation value on the operation status evaluation value can be enhanced by weighting coefficients. When the statistical results of multiple road segments need to be transmitted across nodes to the abnormal congestion identification node, the influence of the data transmission status evaluation value of the data transmission edge on the operation status evaluation value can be enhanced by weighting coefficients, so that the operation status evaluation value can fit the data processing characteristics of the current traffic analysis operation.

[0046] In practical applications, such as in road traffic flow and abnormal congestion analysis scenarios, when most of the vehicle target detection nodes, vehicle trajectory extraction nodes, traffic flow statistics nodes, and abnormal congestion identification nodes in a candidate solution are deployed on computing nodes with better network conditions, and the link state parameters corresponding to key data transmission edges are high, its job status evaluation value will be large. Conversely, when some task nodes are deployed on computing nodes with poor network conditions, or when the link state corresponding to key data transmission edges is low, its job status evaluation value will decrease significantly. This approach ensures that different candidate task node mapping schemes are comparable under the same evaluation criterion, thus providing a basis for selecting the optimal mapping scheme.

[0047] Step S24: Determine the target task node mapping scheme based on the job status evaluation value of each candidate task node mapping scheme.

[0048] Specifically, the job status evaluation value corresponding to each candidate task node mapping scheme is obtained, and the job status evaluation values ​​of each candidate task node mapping scheme are compared; based on the comparison results, the candidate task node mapping scheme with the optimal job status evaluation value is determined as the target task node mapping scheme; when there are multiple candidate task node mapping schemes with the same job status evaluation value, the sum of the link status parameters corresponding to each data transmission edge in each candidate task node mapping scheme is compared, and the candidate task node mapping scheme with the largest sum of link status parameters is determined as the target task node mapping scheme.

[0049] In this embodiment of the invention, the operation status evaluation value corresponding to each candidate task node mapping scheme is obtained, and the evaluation results of all candidate schemes are summarized to form an evaluation set. Based on this, the operation status evaluation values ​​of each candidate task node mapping scheme are compared and sorted according to their numerical values. In one implementation, a descending sorting method can be used, allowing candidate schemes with larger operation status evaluation values ​​to enter the candidate range first. Through the above comparison process, the candidate task node mapping scheme with the largest operation status evaluation value can be directly selected and determined as the target task node mapping scheme, thus completing the initial selection. In the scenario of road traffic flow and abnormal congestion analysis, the target task node mapping scheme can prioritize the deployment of vehicle target detection nodes near traffic video sources and roadside computing nodes with image processing capabilities, deploy traffic flow statistics nodes or road segment status aggregation nodes on regional edge computing nodes with good link status with multiple intersections, and deploy abnormal congestion identification nodes on regional edge computing nodes or traffic management center computing nodes that can stably receive road segment status analysis results.

[0050] Furthermore, in actual operation, there may be situations where multiple candidate task node mapping schemes have the same or differing job status evaluation values ​​within a preset range. To avoid uncertainty in selection under such circumstances, this embodiment introduces a secondary discrimination mechanism based on link state parameters. Specifically, for multiple candidate task node mapping schemes with the same job status evaluation value, the link state parameters corresponding to all data transmission edges in each scheme are summarized and calculated. In one specific implementation, the link state parameters of each data transmission edge can be accumulated to obtain the sum of the link state parameters of the corresponding candidate scheme, expressed as: ; in, This represents the sum of link state parameters for a given candidate task node mapping scheme. Indicates the first Link state parameters of the data transmission edge, This represents the number of data transmission edges in the proposed scheme. Using the above method, candidate schemes with better link quality can be further distinguished.

[0051] For road traffic flow and abnormal congestion analysis, since traffic video streams, vehicle target box data, trajectory point sequences and road segment status statistics all need to be continuously transmitted according to the streaming operation execution map, when the operation status evaluation values ​​are the same or close, the candidate task node mapping scheme with the larger sum of link status parameters should be selected first, which can reduce the risk of traffic data backlog during cross-node transmission.

[0052] In practical applications, such as in road traffic flow and abnormal congestion analysis scenarios, if two candidate task node mapping schemes have the same operational status evaluation value, but one scheme's data transmission path is mainly distributed between Node1 and Node2, with a network latency of 42 milliseconds, available bandwidth of 35 Mbps, and a packet loss rate of 0.3%, while the other scheme involves a backhaul link between Node2 and Node3, with a network latency of 160 milliseconds, available bandwidth of 8 Mbps, and a packet loss rate of 1.8% during peak hours, then the former has a significantly higher total sum of link status parameters. In this case, by comparing the total sum of the link status parameters, the candidate task node mapping scheme with the better link conditions can be determined as the target task node mapping scheme. For example, in a processing link that needs to transmit traffic flow statistics results to the abnormal congestion identification task, if the traffic flow statistics node and the abnormal congestion identification node are deployed on Node2 and Node3 respectively, the backhaul link will be in poor condition. However, if the two are deployed on Node2 or within the link range between Node1 and Node2, a higher total link state parameter can be obtained. Therefore, the latter is more suitable as the target task node mapping scheme.

[0053] Step S3: Based on the target task node mapping scheme, deploy each task node in the streaming job execution graph on the corresponding computing node and execute the streaming job.

[0054] Specifically, based on the target task node mapping scheme, the target computing node corresponding to each task node is determined, and the corresponding task deployment instruction is generated; according to the task deployment instruction, a task execution unit corresponding to each task node is created on the corresponding target computing node, and each task node is loaded into the corresponding task execution unit; after each task execution unit completes registration, based on the dependency relationship of each task node in the streaming job execution graph, each task node is triggered to execute the streaming job according to the target task node mapping scheme.

[0055] In this embodiment of the invention, the target computing node corresponding to each task node is determined based on a target task node mapping scheme. The target task node mapping scheme explicitly defines the correspondence between task nodes and computing nodes. For example, in a road traffic flow and abnormal congestion analysis scenario, the traffic perception data access node and vehicle target detection node can be deployed on Node1, the data cleaning and time synchronization node and traffic flow statistics node can be deployed on Node2, and the road segment status aggregation node or abnormal congestion identification node can be deployed on Node3. Alternatively, when the backhaul link between Node2 and Node3 is in poor condition, the traffic flow statistics node, road segment status aggregation node, and abnormal congestion identification node can be jointly deployed on Node2 to reduce cross-node transmission of statistical results such as road segment flow, average speed, density, and queue length. Based on the above mapping relationship, a corresponding task deployment instruction is further generated. The task deployment instruction includes at least a task node identifier, a target computing node identifier, and execution resource configuration parameters, wherein the execution resource configuration parameters may include information such as CPU quota, memory limit, and whether GPU resources are required. In one specific implementation, for a vehicle target detection task node, the GPU resource requirement can be specified in the task deployment instruction to ensure that it runs on a computing node with GPU capabilities.

[0056] Upon receiving the task deployment instruction, task execution units corresponding to each task node are created on the corresponding target computing node according to the instruction. These task execution units can be implemented as container instances or process instances; the specific implementation method is not limited. In a typical implementation, a corresponding container instance can be created on the target computing node, and the execution logic of the corresponding task node can be loaded into the container instance. During the creation of the task execution unit, the execution environment can be initialized based on the task deployment instruction, such as loading dependent libraries, configuring runtime parameters, and establishing data communication interfaces with other task nodes. In road traffic flow and abnormal congestion analysis scenarios, the task deployment instruction can also carry road traffic perception data source identifiers, intersection numbers, road segment numbers, target detection model loading parameters, statistical period parameters, or abnormal congestion identification window parameters, enabling the corresponding task execution unit to directly access traffic perception data at the specified road location after startup and participate in subsequent streaming processing according to the target task node mapping scheme.

[0057] After the task execution unit is created, each task node is loaded into its corresponding task execution unit, making the task nodes ready for execution. Furthermore, after each task execution unit completes registration, based on the dependencies between task nodes in the streaming job execution graph, each task node is triggered to start execution in a predetermined order. Registration here refers to the task execution unit reporting its availability status to the job management process after startup and establishing communication with the scheduling process. Downstream task nodes will not be triggered to execute until all task execution units have completed registration, thus ensuring that the execution order conforms to the dependency constraints of the streaming job execution graph. For example, the vehicle target detection node will not be triggered to execute until the traffic perception data access node and the data cleaning and time synchronization node have completed registration; the traffic flow statistics node will not be triggered to execute until the vehicle target detection node and the vehicle trajectory extraction node have completed registration; and the abnormal congestion identification node will not be triggered to execute until the traffic flow statistics node and the road segment status aggregation node have completed registration. This ensures that road traffic perception data and traffic status analysis results are continuously transmitted according to the dependencies in the execution graph.

[0058] During execution, task nodes interact with each other via data transmission edges. For example, the data decompression task node sends the processed data to the format conversion task node, which then passes the data to the feature extraction task node. In a specific application, if the data source output rate is 5MB / s, the upstream task node must output data to the downstream task node at a rate no lower than this to ensure the continuity of the data processing chain. By establishing stable data transmission channels between each task execution unit, the entire streaming job execution graph can run continuously.

[0059] It should be noted that the creation, deployment, and communication methods of the aforementioned task execution units are not limited. For example, task execution units can be deployed using container orchestration or by directly starting processes at the operating system level; data transmission between task nodes can be implemented based on network sockets, shared memory, or message queues. As long as the corresponding deployment between task nodes and computing nodes can be completed based on the target task node mapping scheme, and the task nodes are executed according to the dependencies in the streaming job execution graph, they all fall within the scope of protection of this invention. In the scenario of road traffic flow and abnormal congestion analysis, as long as the traffic perception data access node, vehicle target detection node, vehicle trajectory extraction node, traffic flow statistics node, road segment status aggregation node, abnormal congestion identification node, and alarm output node can be deployed to the corresponding computing nodes according to the target task node mapping scheme, and the continuous transmission of road traffic perception data or traffic status analysis results between task nodes is maintained, it can be used as a specific implementation of this embodiment.

[0060] Step S4: During the execution process, the target task node mapping scheme is verified based on the updated device status data, and if the preset conditions are not met, the task node mapping scheme is updated and the streaming job continues to be executed.

[0061] Specifically, during execution, the target task node mapping scheme is validated based on updated device status data. This includes: during the execution of the streaming job, acquiring updated device status data for each computing node, and determining the node status parameters and link status parameters of each data transmission edge based on the updated device status data; determining the current execution status evaluation value of each task node based on the mapping relationship between each task node and computing node in the target task node mapping scheme, and determining the current transmission status evaluation value of each data transmission edge based on the link status parameters corresponding to each data transmission edge; determining the current job status evaluation value based on the current execution status evaluation value of each task node and the current transmission status evaluation value of each data transmission edge; comparing the current job status evaluation value with the job status evaluation value corresponding to the determination of the target task node mapping scheme, and determining that the preset condition is not met when the current job status evaluation value is less than a preset proportion of the job status evaluation value corresponding to the determination of the target task node mapping scheme.

[0062] Furthermore, updating the task node mapping scheme and continuing to execute the streaming job when the preset conditions are not met includes: reconstructing the candidate task node mapping scheme based on the updated node status parameters and link status parameters, and determining the updated target task node mapping scheme; determining the task nodes that need to be adjusted and their corresponding target computing nodes based on the updated target task node mapping scheme, and generating corresponding task adjustment instructions; creating corresponding task execution units on the target computing nodes according to the task adjustment instructions, and migrating the task nodes that need to be adjusted to the corresponding task execution units; after completing the task node migration, restoring the data processing relationships between each task node based on the streaming job execution graph to continue executing the streaming job.

[0063] In this embodiment of the invention, during the execution of the streaming operation for road traffic flow and abnormal congestion analysis, updated device status data of each computing node is continuously acquired. The updated device status data is consistent with the initial acquisition process, still including network latency data, available bandwidth data, and packet loss rate data, and is aggregated according to the computing node identifier. In one specific implementation, the device status can be refreshed every 5 seconds to ensure the timeliness of the status data. Based on the updated device status data, the node status parameters of each computing node are redefined, and the link status parameters of each data transmission edge are further recalculated based on the network status between nodes, so that the current system operating status is reflected in real time. For example, during morning and evening rush hours, sudden traffic accidents, or temporary traffic control, the video stream access volume, vehicle target number, and trajectory point update frequency at a certain intersection may increase significantly, and the link status between the corresponding roadside computing node and the regional edge computing node may also change. Therefore, it is necessary to continuously refresh the device status data to reflect the current operating environment of the road traffic analysis operation.

[0064] After obtaining the updated node and link status parameters, the execution status evaluation value of each task node at the current moment is determined based on the established mapping relationship between task nodes and computing nodes in the target task node mapping scheme. For example, if a task node is still deployed on Node1, and Node1 is a roadside GPU computing node at a key intersection, and its network latency increases from 18 milliseconds to 65 milliseconds, or its available bandwidth decreases from 80 Mbps to 25 Mbps, the execution status evaluation value of the corresponding vehicle target detection node or vehicle trajectory extraction node will decrease accordingly. Simultaneously, the current transmission status evaluation value of each data transmission edge is determined based on the link status parameters corresponding to each data transmission edge. For example, if the link conditions between Node1 and Node2 were originally favorable, but an increase in packet loss rate or a decrease in bandwidth occurs during operation, the transmission status evaluation value corresponding to that link will also decrease. In road traffic flow and abnormal congestion analysis scenarios, this decrease in transmission status evaluation value can manifest as a decrease in the stability of traffic video frames, vehicle target box data, trajectory point sequences, or traffic flow statistics results transmitted between task nodes.

[0065] Furthermore, based on the current execution status evaluation value of each task node and the current transmission status evaluation value of each data transmission edge, the overall operating status of the current streaming job is comprehensively measured to obtain the current job status evaluation value. In one specific implementation, this job status evaluation value can still adopt the same calculation method as the initial scheduling stage to ensure the comparability of the evaluation results. Subsequently, the current job status evaluation value is compared with the job status evaluation value corresponding to the target task node mapping scheme when it was determined. When the current job status evaluation value is less than a preset proportion of the initial job status evaluation value, it is considered that the current task node mapping scheme can no longer meet the operational requirements. For example, when the preset proportion is 0.7, if the current job status evaluation value drops to below 70% of the initial value, the subsequent adjustment process is triggered. In the scenario of road traffic flow and abnormal congestion analysis, the above triggering conditions can be used to identify whether the current task deployment relationship has caused the transmission delay risk of traffic flow statistics results, road segment status aggregation results, or abnormal congestion identification results, thereby providing a basis for judgment for subsequent rescheduling.

[0066] After determining that the preset conditions are not met, the task node mapping scheme is updated. First, based on the updated node status parameters and link status parameters, the candidate task node mapping scheme is reconstructed. This process is consistent with the initial scheduling process, still requiring constraints on task nodes based on the dependencies of the streaming job execution graph, and filtering the candidate computing node set based on node status parameters. Under the new network conditions, previously excluded computing nodes may regain availability, and conversely, previously available computing nodes may be removed, thus forming a new candidate mapping space. For example, when the backhaul link between the traffic management center computing node Node3 and the regional edge computing node Node2 experiences bandwidth degradation during peak hours, Node3 may be removed from the preferred candidate computing node set for abnormal congestion identification nodes; when the peak period ends and the backhaul link recovers, Node3 can re-enter the corresponding candidate computing node set.

[0067] After obtaining new candidate task node mapping schemes, the updated target task node mapping scheme is determined according to a predetermined evaluation method. Subsequently, based on the updated target task node mapping scheme, the task nodes requiring adjustment and their corresponding target computing nodes are identified. The task nodes requiring adjustment are typically those whose current deployment location differs from the updated target location, thus avoiding a complete migration of all task nodes. In one specific implementation, for example, an abnormal congestion identification node originally deployed on Node3, after being remapped to Node2 due to deterioration of the backhaul link between Node3 and Node2, is identified as a task node requiring adjustment. Similarly, a traffic flow statistics node originally deployed on Node1, after being remapped to Node2 due to increased video access pressure on Node1, is identified as a task node requiring adjustment to free up Node1's processing capacity for vehicle target detection tasks.

[0068] Furthermore, corresponding task adjustment instructions are generated, and new task execution units are created on the target computing node according to these instructions. The execution logic of the corresponding task node is loaded into the new task execution unit, while retaining the running status information of the original task execution unit. In one implementation, a checkpoint mechanism can be used to save and restore the task execution status, allowing the task to continue execution from its predetermined progress after migration without having to start from scratch. In road traffic flow and abnormal congestion analysis scenarios, the running status information may include processed traffic video frame numbers, vehicle target tracking status, trajectory point cache, statistical time windows, road segment status aggregation cache, or abnormal congestion identification window status, to ensure that road traffic perception data can still be processed continuously after task migration.

[0069] After creating the task execution unit, the task nodes that need adjustment are migrated to the new task execution unit, and the operation of the original task execution units is gradually stopped. After the migration is completed, the data processing relationship between each task node is restored based on the streaming job execution graph, so that the data flow is transmitted again according to the updated mapping scheme. For example, when the abnormal congestion identification node is migrated from Node3 to Node2, the data output of its upstream task node needs to be redirected to Node2 to ensure the continuity of the data processing link. Specifically, the road segment flow, average speed, density, and queue length statistics output by the road segment status aggregation node no longer pass through the backhaul link between Node2 and Node3, but are directly transmitted to the migrated abnormal congestion identification node on the Node2 side, thereby reducing the impact of the decline in the status of the backhaul link on the output of abnormal congestion identification results.

[0070] It should be noted that the above task migration process can be executed in a step-by-step manner, that is, prioritizing the migration of critical task nodes that have a greater impact on current performance, thereby reducing the disturbance to the overall system. Furthermore, the data buffering and state synchronization methods of each task node during the migration process are not limited, as long as the consistency of data processing logic before and after the migration can be guaranteed, they all fall within the scope of protection of this invention.

[0071] Through the aforementioned verification and update process, the streaming operation for road traffic flow and abnormal congestion analysis can continuously sense changes in equipment status and dynamically adjust the task node mapping scheme during operation, thereby avoiding a decline in overall processing performance due to network degradation. Thus, even in weak network environments, the stability and continuity of streaming operations for traffic sensing data access, vehicle target detection, traffic flow statistics, road segment status aggregation, abnormal congestion identification, and alarm output can be maintained.

[0072] In one specific implementation, such as Figure 3 As shown, this embodiment takes the complete process of streaming job submission to execution as the main line and explains the interaction relationship between Web interaction components, scheduling service hub, Kubernetes, cluster orchestration controller and streaming computing engine.

[0073] In this implementation, a job creation request is first submitted via a web interaction component. Upon receiving the request, the scheduling service hub constructs Flink Cluster resources based on the job configuration and calls the Kubernetes interface to complete resource creation. After the cluster orchestration controller detects the Flink Cluster resource creation event, it creates a Job Manager Pod and starts the corresponding job, thereby constructing a streaming job execution graph and feeding the execution graph information back to the scheduling service hub.

[0074] After the execution graph is generated, the scheduling service hub obtains the execution graph structure based on the JobId and, in conjunction with the node status parameters and link status parameters reported by the device status awareness component, calls the scheduling policy service to generate the mapping relationship between task nodes and computing nodes. This mapping relationship is then sent back to the streaming computing engine as a scheduling result and synchronously written to the FlinkCluster resources to drive the subsequent resource allocation process.

[0075] Furthermore, the cluster orchestration controller creates corresponding Task ManagerPods based on the task node mapping relationship and configures node affinity for each Pod, enabling task nodes to be deployed to target compute nodes. After the Task Manager starts and completes registration, streaming jobs begin execution according to the execution graph dependencies. Simultaneously, the job running status is continuously recorded and persisted to support subsequent scheduling verification and rescheduling processes.

[0076] like Figure 4 The diagram illustrates the task scheduling and resource allocation process during the execution phase of a streaming job. In this process, after a job is submitted, the Job Manager parses the job configuration and generates execution tasks, requesting the Resource Manager to allocate computing resources. After verifying the current resource status, the Resource Manager allocates the corresponding Slot resources to the Task Manager, which then executes the specific tasks. During task execution, the Task Manager reports the execution status to the Job Manager, which ultimately summarizes the job status and provides feedback to the user.

[0077] It should be noted that, Figure 4 The illustrated process primarily reflects the task execution mechanism within the streaming computing engine. Its resource allocation and task scheduling are mainly based on resource availability, without considering device network status or the data transmission link status between task nodes. Building upon this, this invention optimizes the task node mapping relationship by introducing a device status awareness mechanism and incorporating node status parameters and link status parameters during the scheduling phase. Figure 4 The execution process shown is enhanced with pre-scheduling to enable task execution to adapt to dynamic changes in a weak network environment.

[0078] In another embodiment, such as Figure 5The diagram illustrates the streaming task rescheduling process of this invention in a weak network environment. In this embodiment, when the status acquisition agent detects a decrease in the node status parameters of a computing node or the link status parameters corresponding to the data transmission edges between task nodes, it generates abnormal status information and reports it to the scheduling service center. The scheduling service center identifies the affected task nodes based on the abnormal status information and triggers the scheduling policy service to recalculate the task node mapping relationship.

[0079] The scheduling policy service generates a new task node mapping scheme based on the updated node status parameters and link status parameters, and returns the scheduling result to the scheduling service hub. Subsequently, the rescheduling plan is sent to the intelligent adaptive scheduler, triggering a state saving operation during execution. After the state saving is completed, the task allocation relationship is updated based on the updated task node mapping scheme.

[0080] On the resource side, the cluster orchestration controller creates new task execution units on the target compute node according to the updated task node mapping scheme, completes resource registration for the new node, and terminates the corresponding task execution units on the original node. After the task migration is completed, the data processing relationships between each task node are restored based on the streaming job execution graph, allowing the task to continue running from its original execution state.

[0081] Through the above process, a closed-loop control is achieved from changes in equipment status to task rescheduling and execution recovery, enabling streaming operations to run continuously and stably in weak network environments.

[0082] In another implementation, after determining the link state parameters, for each data transmission edge, the link capacity margin is calculated by combining the link state parameters corresponding to that data transmission edge with the data output rate of the source task node. When the link capacity margin is lower than a preset threshold, a data compression processing node is inserted on the corresponding data transmission edge, and the compression processing node and the source task node are deployed on the same computing node to reduce the amount of data transmitted across nodes; a data decompression processing node is set on the target task node side to restore the data structure. In one specific application, when the output rate of the source task node is 5MB / s, and the available bandwidth of the corresponding link is 3Mbps, it is determined that there is a transmission bottleneck on the data transmission edge. By introducing a compression processing node, the effective amount of transmitted data is reduced to within the link's capacity. Through the above method, the data transmission path is adaptively adjusted without changing the original task node mapping relationship, thereby reducing the impact of weak network environment on streaming job execution and reducing the system overhead caused by frequent task migration.

[0083] In other implementations, during execution, when the link state parameter corresponding to a data transmission edge remains below a preset threshold, instead of directly migrating the task node across nodes, a local reconstruction of the affected data processing path is performed based on the streaming job execution graph. Specifically, the source and target task nodes on both sides of the data transmission edge are divided into the same task sub-chain, and this task sub-chain is redeployed on the same computing node based on the node state parameters, thereby eliminating cross-node data transmission. In one specific application, when the link state parameter between the feature extraction task node and the target recognition task node is low, both are redeployed to the same computing node with computing capabilities, allowing data to be transferred locally. By adjusting the spatial distribution structure of task nodes, the dependence on weak links is reduced, thereby reducing performance degradation caused by link fluctuations and improving the continuity of streaming job execution.

[0084] like Figure 6 As shown, this invention provides a road traffic flow-based task status perception and scheduling system. The system includes: a parameter acquisition unit, used to acquire road traffic flow and abnormal congestion analysis flow-based operations to be executed and construct a flow-based operation execution graph; simultaneously, it acquires device status data collected from each computing node deployed on the roadside, intersection, area edge, or traffic management center, and determines the node status parameters of each computing node and the link status parameters of each data transmission edge in the flow-based operation execution graph; wherein, the flow-based operation execution graph includes multiple task nodes for processing road traffic perception data and data transmission edges between task nodes; a scheme determination unit, used to determine a target task node mapping scheme based on the flow-based operation execution graph, the node status parameters, and the link status parameters; an operation execution unit, used to deploy each task node in the flow-based operation execution graph on the corresponding computing node based on the target task node mapping scheme and execute the flow-based operation; and an operation update unit, used to verify the target task node mapping scheme based on the updated device status data during execution, and update the task node mapping scheme and continue executing the flow-based operation if preset conditions are not met.

[0085] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7As shown, the computer device includes a processor A01, a network interface A02, memory (not shown), and a database (not shown) connected via a system bus. The processor A01 provides computing and control capabilities. The memory includes internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02, and a database (not shown). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 stored in the non-volatile storage medium A04. The network interface A02 is used for communication with external terminals via a network connection. When the computer program B02 is executed by the processor A01, it implements a road traffic flow-based task state-aware scheduling method.

[0086] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0087] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details described above. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not further describe the various possible combinations.

[0088] Furthermore, various different embodiments of the present invention can be combined in any way, as long as they do not violate the spirit of the embodiments of the present invention, they should also be regarded as the content disclosed by the embodiments of the present invention.

Claims

1. A road traffic flow task state perception scheduling method, characterized in that, The method includes: The system acquires the road traffic flow and abnormal congestion analysis streaming operations to be executed and constructs a streaming operation execution graph. Simultaneously, it acquires device status data from each computing node deployed at roadside locations, intersections, area edges, or traffic management centers, and determines the node status parameters of each computing node and the link status parameters of each data transmission edge in the streaming operation execution graph. The streaming operation execution graph includes multiple task nodes for processing road traffic perception data and data transmission edges between task nodes. The target task node mapping scheme is determined based on the streaming job execution graph, the node status parameters, and the link status parameters; Based on the target task node mapping scheme, each task node in the streaming job execution graph is deployed on the corresponding computing node and the streaming job is executed; During execution, the target task node mapping scheme is verified based on updated device status data. If the preset conditions are not met, the target task node mapping scheme is updated and the streaming job continues to be executed. During execution, the target task node mapping scheme is validated based on updated equipment status data. This includes: during the execution of the road traffic flow and abnormal congestion analysis streaming operation, acquiring updated equipment status data for each computing node, and determining the node status parameters and link status parameters of each data transmission edge based on the updated equipment status data; determining the current execution status evaluation value of each task node based on the mapping relationship between each task node and computing node in the target task node mapping scheme, and determining the current transmission status evaluation value of each data transmission edge based on the link status parameters corresponding to each data transmission edge; determining the current operation status evaluation value based on the current execution status evaluation value of each task node and the current transmission status evaluation value of each data transmission edge; comparing the current operation status evaluation value with the operation status evaluation value corresponding to the determination of the target task node mapping scheme, and determining that the preset condition is not met when the current operation status evaluation value is less than a preset proportion of the operation status evaluation value corresponding to the determination of the target task node mapping scheme. When preset conditions are not met, the target task node mapping scheme is updated and the streaming job continues to be executed, including: reconstructing the candidate task node mapping scheme based on the updated node status parameters and link status parameters, and determining the updated target task node mapping scheme; based on the updated target task node mapping scheme, determining the task nodes that need to be adjusted and the corresponding target computing nodes, and generating corresponding task adjustment instructions; according to the task adjustment instructions, creating corresponding task execution units on the target computing nodes, and migrating the task nodes that need to be adjusted to the corresponding task execution units; after completing the task node migration, restoring the data processing relationship between each task node based on the streaming job execution graph, so as to continue executing the streaming job; During execution, when the link status parameter corresponding to a data transmission edge is continuously lower than the preset threshold, the affected data processing path is partially reconstructed based on the streaming job execution graph. The source task nodes and target task nodes on both sides of the data transmission edge are divided into the same task sub-chain, and the task sub-chain is redeployed on the same computing node based on the node status parameter.

2. The method of claim 1, wherein, The process involves acquiring the road traffic flow and abnormal congestion analysis streaming job to be executed and constructing the streaming job execution graph. Simultaneously, it acquires the device status data of each computing node and determines the node status parameters of each computing node and the link status parameters of each data transmission edge in the streaming job execution graph, including: A streaming operation execution graph, containing multiple task nodes and data transmission edges between nodes, is constructed based on streaming operation configuration data derived from road traffic flow and abnormal congestion analysis; among which... The multiple task nodes include multiple nodes from the following: traffic perception data access node, data cleaning and time synchronization node, vehicle target detection node, vehicle trajectory extraction node, traffic flow statistics node, road segment status aggregation node, abnormal congestion identification node, and alarm output node. Acquire device status data from each computing node at the roadside, intersection, area edge, or traffic management center, and aggregate the data according to the computing node identifier to form the original status data set for each computing node. Based on the original state data set corresponding to each computing node, the network latency data, available bandwidth data and packet loss rate data of the computing node are obtained, and the node state parameters of each computing node are determined accordingly. For each data transmission edge used to transmit road traffic perception data or traffic state analysis results, based on the original state data set of the computing node where the source task node is located and the original state data set of the computing node where the target task node is located, the corresponding network latency data, available bandwidth data and packet loss rate data are obtained as link state parameters.

3. The road traffic flow-based task state awareness and scheduling method according to claim 1, characterized in that, Determining the target task node mapping scheme based on the streaming job execution graph, the node status parameters, and the link status parameters includes: Based on the aforementioned streaming job execution graph, construct a mapping scheme for multiple candidate task nodes; For each candidate task node mapping scheme, the execution status evaluation value of each task node is determined based on the node status parameters, and the transmission status evaluation value of each data transmission edge is determined based on the link status parameters. Based on the execution status evaluation value of each task node and the transmission status evaluation value of each data transmission edge, the job status evaluation value of each candidate task node mapping scheme is determined. The target task node mapping scheme is determined based on the job status evaluation value of each candidate task node mapping scheme.

4. The road traffic flow-based task state awareness and scheduling method according to claim 3, characterized in that, Based on the streaming job execution graph, a mapping scheme for multiple candidate task nodes is constructed, including: Based on the dependencies between task nodes in the streaming job execution graph, the deployable order of each task node is determined. Based on the deployable order, for each task node, at least one optional computing node is selected from each computing node to form a candidate computing node set for the corresponding task node. Based on the set of candidate computing nodes corresponding to each task node, a combination mapping is performed between each task node and each computing node to generate multiple candidate task node mapping schemes. Each candidate task node mapping scheme is used to describe the correspondence between each task node and each computing node.

5. The road traffic flow-based task state awareness and scheduling method according to claim 3, characterized in that, For each candidate task node mapping scheme, the execution status evaluation value of each task node is determined based on the node status parameters, and the transmission status evaluation value of each data transmission edge is determined based on the link status parameters, including: For each candidate task node mapping scheme, based on the mapping relationship between each task node and the corresponding computing node, the node state parameters of the computing node are obtained, and the execution status evaluation value of each task node is determined based on the obtained node state parameters and the traffic data processing load intensity of the corresponding task node. The traffic data processing load intensity is determined based on the road traffic perception data type, video frame rate, number of road segments participating in the statistics and / or sliding time window. For each data transmission edge in the candidate task node mapping scheme, based on the link state parameters between the computing node where the source task node is located and the computing node where the target task node is located, and combined with the traffic data output rate of the source task node, the transmission state evaluation value of each data transmission edge is determined. The traffic data output rate is determined based on the video bitrate, the vehicle detection result output frequency, the vehicle trajectory point update frequency and / or the road segment statistical result output frequency.

6. The road traffic flow-based task state awareness and scheduling method according to claim 3, characterized in that, The target task node mapping scheme is determined based on the job status evaluation values ​​of each candidate task node mapping scheme, including: Obtain the job status evaluation value corresponding to each candidate task node mapping scheme, and compare the job status evaluation values ​​of each candidate task node mapping scheme; Based on the comparison results, the candidate task node mapping scheme with the optimal job status evaluation value is determined as the target task node mapping scheme. When multiple candidate task node mapping schemes have the same job status evaluation value, the sum of the link status parameters corresponding to each data transmission edge in each candidate task node mapping scheme is compared, and the candidate task node mapping scheme with the largest sum of link status parameters is determined as the target task node mapping scheme.

7. The road traffic flow-based task state awareness and scheduling method according to claim 1, characterized in that, Based on the target task node mapping scheme, the task nodes in the streaming job execution graph are deployed on the corresponding computing nodes and the streaming job is executed, including: Based on the target task node mapping scheme, the target computing node corresponding to each task node is determined, and the corresponding task deployment instructions are generated. According to the task deployment instructions, a task execution unit corresponding to each task node is created on the corresponding target computing node, and each task node is loaded into the corresponding task execution unit; After each task execution unit completes its registration, based on the dependency relationships between each task node in the streaming job execution graph, each task node is triggered to execute the streaming job according to the target task node mapping scheme.

8. A road traffic flow-based task status perception and scheduling system, characterized in that, The system is used to execute the road traffic flow-based task state-aware scheduling method according to any one of claims 1-7, and the system comprises: The parameter acquisition unit is used to acquire the road traffic flow and abnormal congestion analysis streaming operation to be executed and construct the streaming operation execution graph. Simultaneously, it acquires the device status data collected from each computing node deployed at the roadside, intersection, area edge, or traffic management center, and determines the node status parameters of each computing node and the link status parameters of each data transmission edge in the streaming operation execution graph; wherein, The streaming operation execution graph includes multiple task nodes for processing road traffic perception data and data transmission edges between task nodes. The scheme determination unit is used to determine the target task node mapping scheme based on the streaming job execution graph, the node status parameters, and the link status parameters; The job execution unit is used to deploy each task node in the streaming job execution graph on the corresponding computing node based on the target task node mapping scheme and execute the streaming job. The job update unit is used to verify the target task node mapping scheme based on the updated device status data during the execution process, and update the target task node mapping scheme and continue to execute the streaming job if the preset conditions are not met.

Images