A method for identifying and warning illegal behavior based on a graph neural network

Abstract

The application discloses a kind of based on graph neural network's illegal behavior AI identification and early warning method, including the following steps: obtaining and pre-processing the multiple-source monitoring data corresponding to illegal behavior identification task;Extract subject element, behavior element and scene element;Build node and relationship edge, obtain behavior graph data sequence;Carry out feature coding, obtain node feature matrix set and edge feature matrix set;Through the time modeling and state evolution processing of improved DGNN network for stage perception;Based on illegal behavior graph representation set carries out illegal behavior correlation reasoning;Carry out illegal risk assessment;Carry out early warning trigger determination and grade mapping, generate structured illegal warning information.The application adopts the dynamic graph neural network method of stage perception, realizes illegal behavior identification, risk assessment and integrated processing of hierarchical early warning, with the advantages of strong continuous modeling capability, accurate risk determination and high interpretability of early warning result.

Description

Technical Field

[0001] This invention relates to the field of intelligent safety monitoring technology, and in particular to an AI-based method for identifying and issuing early warnings of violations based on graph neural networks. Background Technology

[0002] With the continuous development of video surveillance systems, spatial positioning technology, and scene perception devices, a multi-source monitoring data foundation covering personnel, vehicles, equipment, and work scenarios has been formed in work areas and public environments. Existing methods for identifying violations mainly rely on target detection, behavior classification, or judgment methods based on empirical rules, performing frame-by-frame or short-time analysis of video footage or sensor data to determine whether there are violations or abnormal behaviors.

[0003] However, existing technologies still have significant limitations in practical applications. Most methods lack the ability to comprehensively model the relationships between the actors, action elements, and scene elements, making it difficult to depict the evolution of violations over time and the impact of multi-agent interactions. Furthermore, risk assessment and early warning are typically based on static thresholds or single scoring results, failing to comprehensively reflect the severity of violations, the risk intensity of the actors, and the trend of behavioral development. This results in coarse early warning level classifications and unclear triggering criteria, making it difficult to meet the needs of refined violation identification and tiered early warning in complex scenarios.

[0004] Therefore, how to provide an AI-based method for identifying and warning about traffic violations based on graph neural networks is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose an AI-based method for identifying and warning of traffic violations based on graph neural networks. This invention employs a stage-aware dynamic graph neural network method to achieve integrated processing of violation identification, risk assessment, and tiered warning. It has the advantages of strong continuous modeling capabilities, accurate risk judgment, and high interpretability of warning results.

[0006] According to an embodiment of the present invention, an AI-based method for identifying and issuing early warnings of traffic violations based on graph neural networks includes the following steps: Acquire multi-source monitoring data corresponding to the violation identification task and perform standardized preprocessing to obtain a standardized monitoring dataset; Based on the standardized monitoring dataset, extract subject elements, behavioral elements and scene elements, and output the set of behavioral subjects, the set of behavioral elements and the set of scene elements; Based on the set of behavioral subjects, the set of behavioral elements, and the set of scene elements, nodes and relational edges are constructed to obtain a behavioral graph data sequence arranged in chronological order. Based on the behavioral graph data sequence, feature encoding is performed on nodes and relation edges to obtain a set of node feature matrices and a set of edge feature matrices. Based on the improved DGNN network, the node feature matrix set and the edge feature matrix set are subjected to stage-aware temporal modeling and state evolution processing to obtain the graph representation set of violation behavior. Based on the graph representation set of violation behavior, the relationship between the main actions is constructed, and violation type inference is performed by combining temporal, spatial and semantic constraints to generate a set of violation identification results. Based on the violation identification result set and the violation behavior graph representation set, a violation risk assessment result set is generated. Based on the result set of violation risk assessment, early warning trigger determination and level mapping are performed to generate structured violation early warning information.

[0007] Optionally, the multi-source monitoring data includes video image sequence data, timestamp data, spatial positioning data, and scene element data synchronized with video acquisition. The preprocessing includes time alignment, missing value imputation, outlier removal, data format unification, and numerical normalization.

[0008] Optionally, the output of the set of behavioral subjects, the set of behavioral elements, and the set of scene elements specifically includes: The video image sequence data, spatial positioning data and scene element data corresponding to each timestamp are read from the standardized monitoring dataset to construct frame-level data units arranged in chronological order. Perform subject detection processing on each frame-level data unit to obtain a set of candidate boxes for human subjects and a set of candidate boxes for vehicle subjects, and generate a subject appearance feature vector and a subject spatial location feature vector for each subject candidate box; Multi-target association tracking is performed based on the similarity of the appearance feature vectors of adjacent timestamps and the continuity of the spatial location of the subject. Subject identifiers are assigned to the candidate boxes of personnel and vehicles and subject trajectory indexes are generated. The results are then summarized to obtain the set of behavioral subjects. For each subject identifier in the set of behavioral subjects, extract the subject posture key point sequence, subject motion state sequence and subject interaction object sequence in continuous time segments based on video image sequence data, and encapsulate them into a candidate set of action event elements; Perform action category determination and action time boundary location on the candidate set of action event elements to generate a set of action event elements; Trajectory reconstruction is performed based on the main trajectory index and spatial positioning data to generate a trajectory element set. The action event element set and the trajectory element set are then combined to obtain a behavior element set. Scene element extraction is performed based on scene element data and video image sequence data to generate a set of regional elements and a set of facility elements. These are then combined to obtain a set of scene elements, and the sets of behavioral subjects, behavioral elements, and scene elements are output.

[0009] Optionally, obtaining the behavior graph data sequence specifically includes: A sliding time window sequence is constructed based on the timestamp order of the standardized monitoring dataset, and the standardized monitoring dataset is segmented according to the start and end times of each time window to obtain a time window segment data set that corresponds one-to-one with each time window. For each time window, the data set of the time window segment corresponding to each time window is selected from the set of behavioral subjects, the subject identifiers whose timestamps fall into the current time window are selected, a set of behavioral subject nodes is constructed, and subject category field, subject identifier field, subject timestamp field and subject spatial location field are written for each behavioral subject node; For each time window, the data set of the time window segment corresponding to each time window is selected from the set of behavioral elements. Action event elements and trajectory elements whose timestamps fall into the current time window are selected to construct a set of behavioral element nodes. For each action event element node, an action type field, an action start time field, an action end time field, and an action confidence field are written. For each trajectory element node, a trajectory point sequence field, a trajectory point timestamp sequence field, and a trajectory motion feature field are written. For each time window, the data set of the time window segment corresponding to each time window is selected from the scene element set to filter the regional elements and facility elements that are related to the spatial range of the current time window, construct the scene element node set, and write the regional type field, regional boundary field and regional spatial positioning field for each regional element node, and write the facility type field, facility location field and facility availability status field for each facility element node. The set of nodes for the main actors, the set of nodes for the main actors, and the set of nodes for the elements of the scene are merged to obtain a node set; Construct temporal relationship edges based on the temporal relationships between action event element nodes; Construct spatial relationship edges based on the spatial relationships between behavioral subject nodes and scene element nodes; Construct interaction relationship edges based on the attribution relationship between the behavioral subject node and the behavioral element node; The set of nodes and the set of edges consisting of temporal relationship edges, spatial relationship edges and interaction relationship edges are encapsulated into behavioral graph data corresponding to the current time window, and arranged in chronological order of the time windows to obtain a behavioral graph data sequence.

[0010] Optionally, obtaining the set of node feature matrices and the set of edge feature matrices specifically includes: For a behavioral graph data sequence arranged in chronological order, the node set and edge set corresponding to each time point are read sequentially. Perform feature encoding on the behavioral subject nodes at each time step to generate a set of feature vectors for the behavioral subject nodes; Perform action event node feature encoding on action event element nodes at each time point to generate a set of action event element node feature vectors; Perform trajectory node feature encoding on trajectory element nodes at each time point to generate a set of trajectory element node feature vectors. Then, summarize the set of action event element node feature vectors and the set of trajectory element node feature vectors to obtain a set of behavior element node feature vectors. Perform scene node feature encoding on the regional element nodes and facility element nodes at each time point to generate a set of scene element node feature vectors; The feature vector sets of the subject node, the feature vector sets of the behavior element node, and the feature vector sets of the scene element node at the same time are aligned and spliced ​​according to the writing order of the nodes in the node sets to generate the node feature matrix corresponding to each time. The node feature matrix of each time is then stored in the node feature matrix set in chronological order. Perform edge feature encoding on the temporal relation edges, spatial relation edges, and interaction relation edges at each time point to generate a set of feature vectors for temporal relation edges, a set of feature vectors for spatial relation edges, and a set of feature vectors for interaction relation edges; The feature vector sets of temporal relation edges, spatial relation edges, and interaction relation edges at the same time are aligned and concatenated according to the writing order of the edges in the edge sets to generate the edge feature matrix corresponding to each time. The edge feature matrix of each time is then stored in the edge feature matrix set in chronological order.

[0011] Optionally, obtaining the set of representations of the violation behavior graphs specifically includes: For a behavioral graph data sequence arranged in chronological order, the node set and edge set corresponding to each time point are read sequentially and aligned with the node feature matrix set and edge feature matrix set in time. The behavior graph data sequence is divided into behavior stages based on the temporal evolution of the node set and edge set. The behavior graph data sequence is divided into the pre-behavior stage, the behavior formation stage, and the behavior completion or diffusion stage, and a behavior stage identifier is written for each time point. Based on the behavior stage identifier, the node state update rules and information propagation constraints are configured differently. Prior suppression propagation constraints are set for the behavior pre-occurrence stage, interaction enhancement propagation constraints are set for the behavior formation stage, and diffusion suppression and stable propagation constraints are set for the behavior completion or diffusion stage. Semantic mapping and fusion processing are performed based on the set of behavioral elements and the set of scene elements to determine the semantic parameters of violation risk. At each time step, the node feature matrix, edge feature matrix, node set, and edge set are input into the temporal modeling structure of the improved DGNN network. Based on the edge set, message computation and message aggregation are performed on adjacent nodes to obtain the node message aggregation result. The improved DGNN network, while maintaining the graph representation learning structure of the original network, makes targeted adjustments to the internal processing logic of the time modeling structure and the state update and evolution structure. Specifically, it introduces a violation behavior stage perception mechanism to segment the time evolution of the dynamic graph structure and introduces violation risk semantic constraints in the state evolution process of the graph neural network. At each moment, the node's historical state, the node message aggregation result, and the violation risk semantic parameters are fed into the state update and evolution structure as input items, and the node state evolution update is performed to obtain the node's current state. In the graph representation learning structure, a set of node representations and a set of edge representations are generated based on the current state of the nodes and the set of edges. In the readout substructure of the graph representation learning structure, the node representation set and edge representation set are aggregated and summarized according to time and behavior stage to obtain the subgraph representation set, and the node representation set, edge representation set and subgraph representation set are encapsulated together into the violation behavior graph representation set.

[0012] Optionally, the generation of the violation identification result set specifically includes: Read the node representation set, edge representation set, and subgraph representation set corresponding to each time step sequentially from the graph representation set of violation behavior, and construct a set of inferenceable representation units based on the node representation set and edge representation set; Based on the temporal and interactive relationship edge representations in the edge representation set, the subject representation unit and action representation unit are associated and combined to generate a set of subject-action association pairs; Based on the spatial relationship edge representation in the edge representation set, spatial association matching is performed on the subject action association pair set and the scene element node representation to generate a subject action association pair set with position constraints, and the violation occurrence location field is written for the subject action association pair that meets the spatial association conditions. Perform violation type inference calculation on the set of subject action associations with position constraints to generate a candidate set of violation types; Score normalization and threshold filtering are performed on the candidate set of violation types to generate a violation type field, and the violation occurrence time field is determined based on the action start time field and the action end time field. Based on the subject identifier field in the subject action association pair, the violation subject field is determined, and the violation type field, violation subject field, violation occurrence time field, and violation occurrence location field are encapsulated to generate a violation identification result set.

[0013] Optionally, the generation of the violation risk assessment result set specifically includes: The violation type field, violation subject field, violation occurrence time field, and violation occurrence location field are read sequentially from the violation identification result set. The node representation set, edge representation set, and subgraph representation set corresponding to the time of the violation occurrence time field are read from the violation behavior graph representation set. Risk benchmark values ​​are generated by performing risk benchmark mapping processing based on the violation type field and the violation location field. Based on the risk intensity extraction and processing of the execution subject's risk intensity set based on the graph representation of the violation behavior, a subject risk intensity value is generated; Based on the violation occurrence time field, perform trend risk calculation processing on the set of violation behavior graph representations to generate trend risk values; Risk scoring is calculated based on the risk benchmark value, the subject risk intensity value, and the trend risk value to generate risk score fields. The risk score field is matched with the set of level thresholds to output the corresponding risk level field. The risk level field and the risk score field are then encapsulated to generate a set of violation risk assessment results.

[0014] Optionally, the generation of the structured violation warning information specifically includes: The risk level field in the violation risk assessment result set is matched with the warning level trigger threshold set, and the risk score field is compared with the score trigger threshold set. When the interval matching result meets the warning trigger level condition and the threshold comparison result meets the warning trigger score condition, the warning trigger mark field is set as the trigger mark, and the warning trigger mark field is generated. For records where the warning trigger flag field is a trigger flag, the warning level is determined according to the level mapping rules, and a warning level field is generated; Triggering criteria fields are generated based on risk scoring fields, risk benchmark values, subject risk intensity values, and trend risk values. A timestamp field is generated based on the violation occurrence time field; a triggering subject field is generated based on the violation subject field; and a triggering location field is generated based on the violation occurrence location field. The warning level field, trigger basis field, timestamp field, trigger subject field, and trigger location field are encapsulated in a structured manner to generate structured violation warning information.

[0015] The beneficial effects of this invention are: This invention introduces a graph neural network-based holistic modeling mechanism for violations, unifying and associating the dispersed subject information, action information, and scene information from multi-source monitoring data. This enables the formation of a continuous behavioral graph data sequence over time, allowing for a dynamic depiction of violations from their pre-occurrence, formation, completion, or spread stages. Compared to existing technologies that rely solely on single-frame recognition or local rule-based judgment, this invention, through stage-aware dynamic graph modeling and state evolution processing, fully expresses the interaction relationships between behavioral subjects, the evolution of actions, and their influence on scene constraints. This significantly improves the stability, continuity, and overall consistency of violation recognition in complex scenarios.

[0016] In terms of violation risk assessment, this invention no longer relies solely on a single identification result or static threshold for judgment. Instead, it integrates multi-dimensional information such as violation type, location of occurrence, intensity of interaction between the parties involved, and behavioral development trends to construct a risk benchmark value, a party risk intensity value, and a trend risk value. These are then weighted and fused to generate risk scoring and risk level fields. This approach more realistically reflects the potential risk level and changing trends of violations, making the risk assessment results more interpretable and discriminative, and providing a reliable basis for subsequent early warning triggering and level mapping.

[0017] At the early warning output level, this invention achieves dual constraint determination of risk level and risk score through clearly defined early warning triggering level conditions and early warning triggering scoring conditions. Based on this, it generates structured violation early warning information including early warning level, triggering basis, timestamp, triggering subject, and triggering location. By refining and encapsulating the triggering basis and marking the main triggering basis, the early warning result not only indicates the risk status but also clearly reflects the cause and background of the early warning, which is beneficial for subsequent handling decisions and accountability. Overall, this invention achieves closed-loop processing from violation behavior identification and risk assessment to tiered early warning output, improving the refinement, interpretability, and practical value of the intelligent violation behavior identification and early warning system in practical applications. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Fig. 1 This is a flowchart of an AI-based method for identifying and warning of traffic violations based on graph neural networks, as proposed in this invention. Fig. 2 This is a flowchart illustrating the construction of a behavior graph data sequence for an AI-based method for identifying and warning of traffic violations based on graph neural networks, as proposed in this invention. Fig. 3This is a flowchart illustrating the process of generating a graph representation set of traffic violations for an AI-based method for identifying and warning traffic violations based on graph neural networks, as proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figs. 1-3 A method for AI-based identification and early warning of traffic violations based on graph neural networks includes the following steps: Acquire multi-source monitoring data corresponding to the violation identification task and perform standardized preprocessing to obtain a standardized monitoring dataset; Based on the standardized monitoring dataset, extract subject elements, behavioral elements and scene elements, and output the set of behavioral subjects, the set of behavioral elements and the set of scene elements; Based on the set of behavioral subjects, the set of behavioral elements, and the set of scene elements, nodes and relational edges are constructed to obtain a behavioral graph data sequence arranged in chronological order. Based on the behavioral graph data sequence, feature encoding is performed on nodes and relation edges to obtain a set of node feature matrices and a set of edge feature matrices. Based on the improved DGNN network, the node feature matrix set and the edge feature matrix set are subjected to stage-aware temporal modeling and state evolution processing to obtain the graph representation set of violation behavior. Based on the graph representation set of violation behavior, the relationship between the main actions is constructed, and violation type inference is performed by combining temporal, spatial and semantic constraints to generate a set of violation identification results. Based on the violation identification result set and the violation behavior graph representation set, a violation risk assessment result set is generated. Based on the result set of violation risk assessment, early warning trigger determination and level mapping are performed to generate structured violation early warning information.

[0021] In this embodiment, the multi-source monitoring data includes video image sequence data, timestamp data, spatial positioning data, and scene element data synchronized with video acquisition. The preprocessing includes time alignment, missing value imputation, outlier removal, data format unification, and numerical normalization.

[0022] In this embodiment, the outputs of the set of behavioral subjects, the set of behavioral elements, and the set of scene elements specifically include: The video image sequence data, spatial positioning data and scene element data corresponding to each timestamp are read from the standardized monitoring dataset to construct frame-level data units arranged in chronological order. The frame-level data unit includes image frames, timestamps, spatial positioning information and synchronous scene element information. Perform subject detection processing on each frame-level data unit to obtain a set of candidate boxes for human subjects and a set of candidate boxes for vehicle subjects, and generate a subject appearance feature vector and a subject spatial location feature vector for each subject candidate box; Multi-target association tracking is performed based on the similarity of the appearance feature vectors of adjacent timestamps and the continuity of the spatial location of the subject. Subject identifiers are assigned to the candidate boxes of personnel and vehicles and subject trajectory indexes are generated. The results are then summarized to obtain the set of behavioral subjects. The similarity of the main body appearance feature vector is obtained by extracting color distribution features, texture statistical features, and shape structure features from the image regions corresponding to the same category of main body candidate boxes under adjacent time stamps, normalizing them, and then concatenating them to form the main body appearance feature vector; then, the feature difference is calculated dimension by dimension for the main body appearance feature vectors corresponding to adjacent time stamps and the overall difference is obtained. The overall difference is converted into a similarity value by reverse normalization. The similarity value is used to characterize the consistency of the main body appearance under adjacent time stamps. The continuity of the subject's spatial position is obtained by calculating the position change of the subject's spatial position feature vector corresponding to the subject's candidate box under adjacent timestamps. The subject's spatial position feature vector is formed by normalizing the spatial positioning coordinates in the spatial positioning data. The position change calculation includes calculating the displacement distance, displacement direction change, and time interval between the subject's spatial positions at adjacent timestamps. After normalizing the displacement distance, displacement direction change, and time interval respectively, the comprehensive value of position change is obtained according to the weighted combination rule. Then, the comprehensive value of position change is converted into a spatial continuity metric value by reverse normalization. The spatial continuity metric value is used to characterize the degree of spatial motion continuity of the subject in the time dimension. For each subject identifier in the set of behavioral subjects, extract the subject posture key point sequence, subject motion state sequence and subject interaction object sequence in continuous time segments based on video image sequence data, and encapsulate them into a candidate set of action event elements; Action category determination and action time boundary localization are performed on the candidate set of action event elements to generate a set of action event elements. The set of action event elements includes an action type field, an action start time field, an action end time field, an action confidence field, and an attribution subject identifier field. The attribution subject identifier field is taken from the subject identifier field of the action subject corresponding to the action event. The action confidence is equal to the action detection score multiplied by the trajectory continuity weight multiplied by the scene consistency weight. The action detection score represents the score output by the action category determination. The trajectory continuity weight represents the weight corresponding to the degree of continuity of the spatial position change of the subject identifier within the continuous time range before and after the action. The scene consistency weight represents the weight corresponding to the degree of matching between the action occurrence location and the synchronous scene element information in the frame-level data unit. Trajectory reconstruction is performed based on the main trajectory index and spatial positioning data to generate a trajectory element set. The action event element set and the trajectory element set are then combined to obtain a behavior element set. The trajectory element set includes a trajectory point sequence field, a trajectory point timestamp sequence field, and a trajectory motion feature field. The trajectory motion feature field includes displacement change, velocity change, and direction change. The displacement change represents the magnitude of the spatial position difference between adjacent trajectory points, the velocity change represents the ratio of the displacement change at adjacent moments to the time interval between adjacent moments, and the direction change represents the magnitude of the change in direction between adjacent trajectory segments. Scene element extraction is performed based on scene element data and video image sequence data to generate a set of regional elements and a set of facility elements. The scene element set is then summarized and output as a set of behavioral subjects, a set of behavioral elements, and a set of scene elements. The set of regional elements includes a regional type field, a regional boundary field, and a regional spatial positioning field. The set of facility elements includes a facility type field, a facility location field, and a facility availability status field.

[0023] In this embodiment, obtaining the behavior graph data sequence specifically includes: A sliding time window sequence is constructed based on the timestamp order of the standardized monitoring dataset, and the standardized monitoring dataset is segmented according to the start and end times of each time window to obtain a time window segment data set that corresponds one-to-one with each time window. For each time window, the data set of the time window segment corresponding to each time window is selected from the set of behavioral subjects, the subject identifiers whose timestamps fall into the current time window are selected, a set of behavioral subject nodes is constructed, and subject category field, subject identifier field, subject timestamp field and subject spatial location field are written for each behavioral subject node; For each time window, the data set of the time window segment corresponding to each time window is selected from the set of behavioral elements. Action event elements and trajectory elements whose timestamps fall into the current time window are selected to construct a set of behavioral element nodes. For each action event element node, an action type field, an action start time field, an action end time field, and an action confidence field are written. For each trajectory element node, a trajectory point sequence field, a trajectory point timestamp sequence field, and a trajectory motion feature field are written. For each time window, the data set of the time window segment corresponding to each time window is selected from the scene element set to filter the regional elements and facility elements that are related to the spatial range of the current time window, construct the scene element node set, and write the regional type field, regional boundary field and regional spatial positioning field for each regional element node, and write the facility type field, facility location field and facility availability status field for each facility element node. The set of nodes for the main actors, the set of nodes for the main actors, and the set of nodes for the elements of the scene are merged to obtain a node set; Construct temporal relationship edges based on the temporal relationships between action event element nodes; The construction of the temporal relationship edge includes: calculating the sequential relationship between action event element nodes based on the action start time field and action end time field of the action event element node; when the start time of the later action is later than the end time of the earlier action, a temporal relationship edge is established between the corresponding two action event element nodes, and a sequential direction field and a time interval field are written into the temporal relationship edge, wherein the time interval field is equal to the time difference between the start time of the later action and the end time of the earlier action. Construct spatial relationship edges based on the spatial relationship between behavioral subject nodes and scene element nodes; The construction of the spatial relationship edge includes: calculating the spatial association based on the subject spatial location field of the behavior subject node and the region boundary field and facility location field of the scene element node; when the subject spatial location is inside the region boundary or the spatial distance between the subject spatial location and the facility location is less than the spatial association threshold, a spatial relationship edge is established between the corresponding behavior subject node and the scene element node, and a spatial distance field and a region inclusion marker field are written into the spatial relationship edge. The spatial distance field represents the distance between the subject spatial location and the facility location, and the region inclusion marker field is used to characterize whether the subject spatial location is inside the region boundary. Construct interaction relationship edges based on the attribution relationship between the behavioral subject node and the behavioral element node; The construction of the interaction relationship edge includes: establishing an interaction association based on the subject identifier field of the behavior subject node and the subject identifier field of the behavior element node; when the subject identifier of the action event element node is consistent with the subject identifier of the behavior subject node, an interaction relationship edge is established between the corresponding behavior subject node and the action event element node; when the subject identifier of the trajectory element node is consistent with the subject identifier of the behavior subject node, an interaction relationship edge is established between the corresponding behavior subject node and the trajectory element node; and writing the interaction category field and the interaction intensity field into the interaction relationship edge. The interaction intensity field is equal to the product of the action confidence field and the trajectory continuity weight. The trajectory continuity weight represents the weight corresponding to the degree of continuity of the spatial position change of the subject identifier within the continuous time range from the start time to the end time of the action. The set of nodes and the set of edges consisting of temporal relationship edges, spatial relationship edges and interaction relationship edges are encapsulated into behavioral graph data corresponding to the current time window, and arranged in chronological order of the time windows to obtain a behavioral graph data sequence.

[0024] In this embodiment, obtaining the set of node feature matrices and the set of edge feature matrices specifically includes: For a behavioral graph data sequence arranged in chronological order, the node set and edge set corresponding to each time point are read sequentially. Perform subject node feature encoding on the subject node at each time point to generate a set of subject node feature vectors. The subject node feature encoding includes class encoding of the subject category field, numerical normalization encoding of the subject spatial location field, conversion of the subject timestamp field into time location encoding, and concatenating the category encoding, spatial location encoding and time location encoding in a fixed order to form the subject node feature vector. Action event node feature encoding is performed on action event element nodes at each time point to generate a set of action event element node feature vectors. The action event node feature encoding includes class encoding of the action type field, converting the action start time field and action end time field into action duration encoding and action time location encoding, performing numerical normalization encoding on the action confidence field, and concatenating the action type encoding, action duration encoding, action time location encoding and action confidence encoding in a fixed order to form the action event element node feature vector. Trajectory node feature encoding is performed on trajectory element nodes at each time point to generate a set of trajectory element node feature vectors. The set of action event element node feature vectors and the set of trajectory element node feature vectors are then combined to obtain a set of behavior element node feature vectors. The trajectory node feature encoding includes numerical normalization encoding of displacement change, velocity change, and direction change in the trajectory motion feature field, and sequence statistical encoding of the trajectory point sequence field and the trajectory point timestamp sequence field. The sequence statistical encoding includes trajectory point quantity encoding, trajectory duration encoding, and trajectory change stability encoding. The trajectory duration encoding is equal to the time difference between the last timestamp and the first timestamp in the trajectory point timestamp sequence. The trajectory change stability encoding is equal to the weighted sum of the dispersion of the displacement change sequence and the dispersion of the direction change sequence. The displacement change encoding, velocity change encoding, direction change encoding, trajectory point quantity encoding, trajectory duration encoding, and trajectory change stability encoding are then concatenated in a fixed order to form the trajectory element node feature vector. Scene node feature encoding is performed on the regional element nodes and facility element nodes at each time point to generate a set of scene element node feature vectors. The scene node feature encoding includes class encoding of the regional type field and facility type field, converting the regional boundary field and regional spatial positioning field into regional geometric encoding, converting the facility location field into facility location encoding, converting the facility availability status field into status encoding, and concatenating the class encoding, regional geometric encoding, facility location encoding and status encoding in a fixed order corresponding to the node type to form the scene element node feature vector. The feature vector sets of the subject node, the feature vector sets of the behavior element node, and the feature vector sets of the scene element node at the same time are aligned and spliced ​​according to the writing order of the nodes in the node sets to generate the node feature matrix corresponding to each time. The node feature matrix of each time is then stored in the node feature matrix set in chronological order. Edge feature encoding is performed on the temporal relationship edges, spatial relationship edges, and interaction relationship edges at each time point to generate a set of feature vectors for temporal relationship edges, a set of feature vectors for spatial relationship edges, and a set of feature vectors for interaction relationship edges. The edge feature encoding includes directional encoding of the sequential direction field, numerical normalization encoding of the time interval field, numerical normalization encoding of the spatial distance field, label encoding of the region inclusion field, category encoding of the interaction category field, and numerical normalization encoding of the interaction intensity field. The direction encoding, time interval encoding, spatial distance encoding, region inclusion label encoding, interaction category encoding, and interaction intensity encoding are then concatenated in a fixed order to form the edge feature vector. The feature vector sets of temporal relation edges, spatial relation edges, and interaction relation edges at the same time are aligned and concatenated according to the writing order of the edges in the edge sets to generate the edge feature matrix corresponding to each time. The edge feature matrix of each time is then stored in the edge feature matrix set in chronological order.

[0025] In this embodiment, obtaining the set of representations of the violation behavior graph specifically includes: For a behavioral graph data sequence arranged in chronological order, the node set and edge set corresponding to each time point are read sequentially and aligned with the node feature matrix set and edge feature matrix set in time. The behavior graph data sequence is divided into behavior stages based on the temporal evolution of the node set and edge set. The behavior graph data sequence is divided into the pre-behavior stage, the behavior formation stage, and the behavior completion or diffusion stage. A behavior stage identifier is written for each time point. The behavior stage is determined based on the frequency change characteristics and trajectory change amplitude characteristics of the behavior elements in the behavior graph data sequence. Based on the behavior stage identifier, the node state update rules and information propagation constraints are configured differently. Prior suppression propagation constraints are set for the behavior pre-occurrence stage, interaction enhancement propagation constraints are set for the behavior formation stage, and diffusion suppression and stable propagation constraints are set for the behavior completion or diffusion stage. The node state update rule is implemented by setting historical retention weights corresponding to the behavior stage in the node state update operator. The information propagation constraint acts on the neighborhood message passing process of the graph neural network by restricting the message propagation weights of different types of edges in the behavior graph. The prior suppression propagation constraint includes: in the stage before the behavior occurs, setting initial suppression conditions for information propagation between nodes. The initial suppression conditions are used to limit the propagation intensity and propagation range of node messages between adjacent nodes. They are determined based on the node's historical state and the node's characteristics at the current moment. Propagation attenuation weights are applied to node messages that do not meet the prior behavior triggering conditions, so that node state updates are mainly based on maintaining the historical state. The interaction enhancement propagation constraint includes: during the behavior formation stage, applying interaction enhancement conditions to node pairs with interaction relationship edges, the interaction enhancement conditions being used to increase the weight of node messages propagated through the interaction relationship edges, and enabling enhancement propagation factors for behavior subject nodes associated with action event element nodes and trajectory element nodes, so that messages between interactively associated nodes play a dominant role in the node state update process. The diffusion suppression and stable propagation constraints include: during the behavior completion or diffusion phase, applying diffusion restriction conditions to the propagation range and propagation intensity of node messages, and applying state preservation conditions to nodes that have formed a stable state. The diffusion restriction conditions are used to suppress the diffusion propagation of node messages to non-directly related nodes, and the state preservation conditions are used to reduce the change amplitude of node states between adjacent time points in order to maintain the stability of the overall state of the behavior graph. Based on the semantic mapping and fusion processing of the behavioral element set and the scene element set, the semantic parameters of violation risk are determined. The semantic parameters of violation risk include risk level parameters, behavior severity parameters, and violation trend intensity parameters. The risk level parameter is obtained by the action type field, the area type field, and the facility availability status field according to the level mapping rule. The behavior severity parameter is obtained by the action confidence field, the interaction intensity field, and the area sensitivity value mapped from the area type field according to the weighted combination rule. The violation trend intensity parameter is obtained by the change in the number of action events, the change in interaction intensity, and the change in the number of spatial relationship edges at adjacent time points according to the weighted combination rule. At each time step, the node feature matrix, edge feature matrix, node set, and edge set are input into the temporal modeling structure of the improved DGNN network. Based on the edge set, message computation and message aggregation are performed on adjacent nodes to obtain the node message aggregation result. The message computation is obtained by the linear transformation rule after concatenation of the edge feature vector and the feature vector of the adjacent node. The message aggregation is obtained by the weighted summation rule of the adjacent edge messages under the control of behavior stage identification and information propagation constraints. The improved DGNN network, while maintaining the graph representation learning structure of the original network, makes targeted adjustments to the internal processing logic of the time modeling structure and the state update and evolution structure. Specifically, it introduces a violation behavior stage perception mechanism to segment the time evolution of the dynamic graph structure and introduces violation risk semantic constraints in the state evolution process of the graph neural network. At each time point, the node's historical state, the node's message aggregation result, and the violation risk semantic parameters are fed into the state update and evolution structure as inputs. The node state evolution update is then performed to obtain the node's current state. This node state evolution update includes: at the current time, weighting the node's historical state according to historical retention weights to obtain a historical state weighted result; weighting the node's message aggregation result according to stage propagation weights to obtain a message aggregation weighted result; weighting the violation risk semantic parameters according to risk modulation weights to obtain a risk semantic weighted result; aligning the historical state weighted result, message aggregation weighted result, and risk semantic weighted result numerically, then fusing them according to a weighted summation rule to obtain a fused state vector; and performing state normalization and state constraint processing on the fused state vector to generate the node's current state. In the graph representation learning structure, a node representation set and an edge representation set are generated based on the current state of the nodes and the edge set. The node representation set is obtained by arranging the current states of each node in the order of the node set. The edge representation set is obtained by concatenating the current states of the nodes at both ends of the edge and the edge feature vectors according to the linear transformation rule, and then arranging them in the order of the edge set. In the readout substructure of the graph representation learning structure, the node representation set and edge representation set are aggregated and summarized according to time and behavior stage to obtain the subgraph representation set, and the node representation set, edge representation set and subgraph representation set are encapsulated together into the violation behavior graph representation set.

[0026] In this embodiment, the generation of the violation identification result set specifically includes: The node representation set, edge representation set, and subgraph representation set corresponding to each time moment are read sequentially from the set of graph representations of violations. Based on the set of node representation sets and edge representation sets, a set of inferable representation units is constructed. The set of inferable representation units includes a subject representation unit with the behavior subject node representation as the core, an action representation unit with the action event element node representation as the core, and a trajectory representation unit with the trajectory element node representation as the core. The subject representation unit includes a subject identifier field and a subject representation vector field. The action representation unit includes an action type field, an action start time field, an action end time field, and an action representation vector field. The trajectory representation unit includes a trajectory point timestamp sequence field, a trajectory point sequence field, and a trajectory representation vector field. Based on the temporal relationship edge representation and interaction relationship edge representation in the edge representation set, the subject representation unit and action representation unit are associated and combined to generate a subject action association pair set. The generation of the subject action association pair set includes: when the subject identifier field of the behavior subject node connected by the interaction relationship edge is consistent with the subject identifier field of the action event element node, and the temporal order constraint of the corresponding temporal relationship edge representation satisfies that the action start time field is within the association time range, the corresponding subject representation unit and action representation unit are combined into a subject action association pair, and the association time field and interaction intensity field are written. The association time field is taken as the time interval corresponding to the automatic action start time field and the action end time field. Based on the spatial relationship edge representation in the edge representation set, spatial association matching is performed on the subject action association pair set and the scene element node representation to generate a subject action association pair set with position constraints. For subject action association pairs that meet the spatial association conditions, a violation occurrence location field is written. The spatial association matching includes: when the region containment marker field of the spatial relationship edge indicates that the subject is located inside the boundary of the corresponding region element, the violation occurrence location field is taken from the region spatial positioning field of the corresponding region element; when the region containment marker field of the spatial relationship edge indicates that the subject is not located inside the boundary of the region element and the spatial distance field is less than the spatial association threshold, the violation occurrence location field is taken from the facility location field of the corresponding facility element. A violation type inference calculation is performed on the set of subject action associations with position constraints to generate a candidate set of violation types. The violation type inference calculation includes: concatenating the subject representation vector field, action representation vector field, trajectory representation vector field, and subgraph representation vector at the same time in the subgraph representation set to obtain a joint representation vector, and calculating the inference score of each violation type based on the joint representation vector. The inference score is determined by the similarity value calculated between the joint representation vector and the violation type template vector, and then weighted by the position constraint matching weight. The similarity value is used to characterize the degree of consistency between the joint representation vector and the semantic pattern of the violation type, and the position constraint matching weight is used to characterize the degree of consistency between the violation occurrence location field and the scene constraint corresponding to the violation type. The violation type template vector is taken from a preset violation type template library. The types of violations include: violations exceeding work authority, violations entering prohibited or restricted areas, violations not following the prescribed route or trajectory, violations of abnormally lingering in dangerous or sensitive areas, and violations where the work behavior does not match the availability or operational status of the facilities. Score normalization and threshold filtering are performed on the candidate set of violation types to generate a violation type field, and the violation occurrence time field is determined based on the action start time field and the action end time field, wherein the violation occurrence time field is the time interval corresponding to the action start time field and the action end time field. The violation subject field is determined based on the subject identifier field in the subject action association pair. The violation type field, violation subject field, violation occurrence time field, and violation occurrence location field are encapsulated to generate a violation identification result set. The violation subject includes the behavioral subject corresponding to the violation type field in the behavioral subject set. The behavioral subject includes personnel subject, vehicle subject, equipment subject, or facility operation subject.

[0027] In this embodiment, the generation of the violation risk assessment result set specifically includes: The violation type field, violation subject field, violation occurrence time field, and violation occurrence location field are read sequentially from the violation identification result set. The node representation set, edge representation set, and subgraph representation set corresponding to the time of the violation occurrence time field are read from the violation behavior graph representation set. Risk benchmark mapping is performed based on the violation type field and the violation location field to generate a risk benchmark value. The risk benchmark mapping process includes mapping the violation type field to a type risk benchmark value, mapping the area type field or facility availability status field corresponding to the violation location field to a scenario risk benchmark value, and fusing the type risk benchmark value and the scenario risk benchmark value according to a weighted combination rule to obtain the risk benchmark value. Based on the set of graph representations of violations, the subject risk intensity extraction process is performed to generate a subject risk intensity value. The subject risk intensity extraction process includes: locating the subject representation vector field corresponding to the subject field of the violation from the set of node representations; locating the interaction relationship edge representation associated with the subject field of the violation from the set of edge representations and reading the interaction intensity field; reading the subgraph representation vector corresponding to the violation occurrence time field from the set of subgraph representations; and merging the subject representation vector field, the interaction intensity field and the subgraph representation vector according to the weighted combination rule to obtain the subject risk intensity value. Based on the violation occurrence time field, a trend risk calculation process is performed on the violation behavior graph representation set to generate a trend risk value. The trend risk calculation process includes: statistically analyzing the changes in the number of action event element nodes, the changes in interaction intensity, and the changes in the number of spatial relationship edges within the time interval corresponding to the violation occurrence time field; and then normalizing the changes in the number of action event element nodes, the changes in interaction intensity, and the changes in the number of spatial relationship edges respectively, and then merging them according to a weighted combination rule to obtain the trend risk value. Risk scoring is performed based on the risk benchmark value, the subject risk intensity value, and the trend risk value to generate a risk score field. The risk score field is obtained by fusing the risk benchmark value, the subject risk intensity value, and the trend risk value according to a weighted summation rule. The risk benchmark value is obtained by mapping the violation type field and the violation location field to obtain the type risk benchmark value and the scenario risk benchmark value, respectively, and then fusing them according to weights. The subject risk intensity value is obtained by weighted fusing the subject representation vector corresponding to the violation subject, the interaction intensity field represented by the interaction relationship edge, and the subgraph representation vector at the same time. The trend risk value is obtained by weighted fusing the change in the number of action events, the change in interaction intensity, and the change in the number of spatial relationship edges within the violation occurrence time interval. The risk score field is matched with the set of level thresholds to output the corresponding risk level field. The risk level field and the risk score field are then encapsulated to generate a set of violation risk assessment results.

[0028] In this embodiment, the generation of the structured violation warning information specifically includes: The risk level field in the violation risk assessment result set is matched with the warning level trigger threshold set, and the risk score field is compared with the score trigger threshold set. When the interval matching result meets the warning trigger level condition and the threshold comparison result meets the warning trigger score condition, the warning trigger mark field is set as the trigger mark, and the warning trigger mark field is generated. The conditions for triggering the warning level include: after the risk level field is matched with the set of warning level triggering thresholds, the risk level corresponding to the matching result is not lower than the preset warning triggering level, or the matching result falls into the level range that needs to trigger the warning. The warning trigger scoring conditions include: after comparing the risk score field with the scoring trigger threshold set, the value of the risk score field is greater than or equal to the warning trigger scoring threshold of the corresponding level. For records where the warning trigger marker field is a trigger marker, the warning level is determined according to the level mapping rule, and a warning level field is generated. The level mapping rule includes mapping the risk level field to a warning level field that is consistent with the risk level field or mapping it to a warning level field that is higher than the risk level field. The warning levels include low risk warning level, medium risk warning level and high risk warning level. The low risk warning level is used to indicate that the violation risk is in a state of concern, the medium risk warning level is used to indicate that the violation risk is in a state of intervention, and the high risk warning level is used to indicate that the violation risk is in a state of immediate handling. Triggering basis fields are generated based on risk scoring fields, risk benchmark values, subject risk intensity values, and trend risk values. The triggering basis fields include risk scoring field values, risk level field values, risk benchmark value values, subject risk intensity value values, trend risk value values, and comparison results of the interval matching results of the early warning level triggering threshold set and the threshold of the scoring triggering threshold set. The risk scoring field values ​​and risk level field values ​​are marked as the main triggering basis in the triggering basis fields. A timestamp field is generated based on the violation occurrence time field, a triggering subject field is generated based on the violation subject field, and a triggering location field is generated based on the violation occurrence location field. The timestamp field is taken from the start or end time of the time interval corresponding to the violation occurrence time field and converted according to the warning output time format rules. The triggering subject field is taken from the subject identification information and subject category information corresponding to the violation subject field and encapsulated according to the warning output subject format rules. The triggering location field is taken from the area spatial positioning field or facility location field corresponding to the violation occurrence location field and encapsulated according to the warning output location format rules. The warning level field, trigger basis field, timestamp field, trigger subject field, and trigger location field are encapsulated in a structured manner to generate structured violation warning information.

[0029] Example 1: To verify the feasibility of this invention in practice, it was applied to a daily safety supervision scenario in a large industrial park. This scenario involves personnel movement, vehicle traffic, equipment operation, and multiple types of work areas, resulting in a complex spatial structure and significant differences in work rules. Traditional methods relying on manual inspections and single video rule judgments are insufficient to promptly identify violations occurring across time and areas. In particular, there is a lack of effective linkage analysis capabilities for abnormal personnel trajectories, unauthorized operations, and behaviors incompatible with facility status, leading to identification delays and misjudgments.

[0030] In this scenario, video capture devices deployed at key locations throughout the park continuously acquire video image sequences, simultaneously obtaining time information, spatial positioning information, and status information of the area and facilities. The system first performs unified processing on the multi-source data to ensure consistency in time and space. Based on this, the system automatically identifies personnel, vehicles, and equipment operators, and extracts their actions and trajectories from continuous footage. Simultaneously, it extracts scene elements from the environment, such as work area boundaries, facility locations, and facility availability. This information is then organized into behavioral subjects, behavioral elements, and scene elements, and constructed into a behavioral graph data sequence containing multiple relationships within a time window.

[0031] Subsequently, the system utilizes an improved dynamic graph neural network to model the behavioral graph data. By introducing a behavioral stage segmentation mechanism, the system can distinguish between different stages, such as before a behavior occurs, during behavior formation, and during behavior diffusion. It dynamically adjusts the information dissemination method at each stage, ensuring that potential violations are addressed early on while preventing the excessive spread of irrelevant information later. During the modeling process, behavioral elements and scene elements are further mapped to semantic parameters related to violation risk, so that the evolution of node states not only reflects the behavior itself but also its environment and rule constraints.

[0032] During the violation identification phase, the system automatically constructs the relationship between the subject and the action based on the graph representation results, and combines temporal sequence, spatial location, and scene semantics to perform comprehensive reasoning on the behavior, thereby identifying various violations such as unauthorized operation, entry into restricted areas, abnormal trajectory, and mismatch with facility status. Compared with methods that rely solely on single-frame judgment, this approach can completely reconstruct the formation process of the violation, significantly reducing misjudgments caused by instantaneous images.

[0033] In the risk assessment and early warning stage, the system further integrates the type of violation, location of occurrence, intensity of interaction between the parties involved, and behavioral evolution trend to classify and assess the risk of violations and generate structured early warning information. In practical applications, the system can provide early warnings before violations cause obvious consequences, allowing managers to take intervention measures at appropriate times and locations. Continuous operation has shown that the system's identification of violations is more stable in complex operating environments, and its response to high-risk behaviors is more timely, significantly improving the continuity and reliability of overall safety supervision. This fully demonstrates the technical advantages and practical value of this invention in real-world application scenarios.

[0034] Table 1. Performance Comparison of the Invention with Traditional Violation Identification and Early Warning Methods

[0035] As can be clearly seen from Table 1, the method of the present invention is superior to the traditional method in many indicators.

[0036] In terms of the accuracy of violation identification, the traditional method achieves 87.2%, while the method of this invention improves it to 92.6%, an increase of 5.4 percentage points. This improvement is mainly due to the fact that the present invention does not rely on single-frame or single-node judgments in the identification stage, but rather on a unified model of the behavior subject, action elements, and scene elements based on a set of violation behavior graph representations. This allows the identification results to be subject to the combined effects of temporal, spatial, and semantic constraints, thereby reducing erroneous judgments caused by isolated features.

[0037] Regarding the recall rate for violation identification, the traditional method achieves 83.5%, while the method of this invention reaches 87.7%. This improved recall rate indicates that the present invention has a stronger ability to capture covert and progressive violations. This is because the improved DGNN network introduces a behavior stage segmentation mechanism, allowing information from the pre-beginning and behavior formation stages to be continuously accumulated, thus avoiding the problem of missed detections due to the lack of obvious instantaneous features in traditional methods.

[0038] The false alarm rate was reduced from 9.4% with the traditional method to 6.8%, a decrease of 3.6 percentage points. This improvement is mainly due to the introduction of position constraint matching weights and violation risk semantic parameters in the violation type reasoning process. This ensures that the system must simultaneously meet the matching conditions of behavioral patterns and scenario rules when judging violations, thereby effectively suppressing false alarms caused by normal work behavior and similar actions.

[0039] Regarding the average recognition latency, the traditional method is 2.6s, while the method of this invention is 1.9s. The reason for the reduced latency is not simply the compression of the calculation process, but rather the use of a time window organization method for the behavioral graph data sequence, which allows behavioral information to be continuously updated in the graph structure, reducing redundant calculations. At the same time, the graph representation results can be directly reused in the risk assessment and early warning stages.

[0040] The stability in complex scenarios improved from 84.1% to 89.3%, reflecting the superior control of recognition result fluctuations in scenarios with dense crowds and frequent behavioral overlap. This improvement stems from the information propagation constraint mechanism introduced during node state evolution, effectively preventing irrelevant node information from interfering with complex graph structures.

[0041] Regarding the success rate of cross-temporal violation identification, the traditional method achieves 78.9%, while the method of this invention reaches 86.5%. This improvement is directly related to the temporal modeling structure of behavior stage perception, enabling the system to correlate and reason about behavioral fragments scattered across multiple time windows, thereby identifying violations formed across time.

[0042] The early warning rate for high-risk violations increased from 72.3% to 81.6%, indicating that the present invention can trigger effective warnings before the risks fully manifest. This is because a trend risk value was introduced in the risk assessment stage, which jointly models the changes in the number of action events, the changes in interaction intensity, and the changes in spatial relationship edges, enabling the risk evolution trend to be reflected in advance.

[0043] Finally, in terms of the consistency index of early warning levels, the method of this invention achieved 91.4%, significantly higher than the 85.0% of the traditional method. This improvement stems from the adoption of a dual-condition judgment mechanism using risk level and risk score fields in the early warning triggering stage, which ensures a higher consistency between the early warning level and the actual risk status, avoiding issues such as level jumps or instability.

[0044] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for AI-based identification and early warning of traffic violations based on graph neural networks, characterized in that, Includes the following steps: Acquire multi-source monitoring data corresponding to the violation identification task and perform standardized preprocessing to obtain a standardized monitoring dataset; Based on the standardized monitoring dataset, extract subject elements, behavioral elements and scene elements, and output the set of behavioral subjects, the set of behavioral elements and the set of scene elements; Based on the set of behavioral subjects, the set of behavioral elements, and the set of scene elements, nodes and relational edges are constructed to obtain a behavioral graph data sequence arranged in chronological order. Based on the behavioral graph data sequence, feature encoding is performed on nodes and relation edges to obtain a set of node feature matrices and a set of edge feature matrices. Based on the improved DGNN network, the node feature matrix set and the edge feature matrix set are subjected to stage-aware temporal modeling and state evolution processing to obtain the graph representation set of violation behavior. Based on the graph representation set of violation behavior, the relationship between the main actions is constructed, and violation type inference is performed by combining temporal, spatial and semantic constraints to generate a set of violation identification results. Based on the violation identification result set and the violation behavior graph representation set, a violation risk assessment result set is generated. Based on the result set of violation risk assessment, early warning trigger determination and level mapping are performed to generate structured violation early warning information.

2. The method for AI-based identification and early warning of traffic violations based on graph neural networks according to claim 1, characterized in that, The multi-source monitoring data includes video image sequence data, timestamp data, spatial positioning data, and scene element data synchronized with video acquisition. The preprocessing includes time alignment, missing value imputation, outlier removal, data format unification, and numerical normalization.

3. The method for AI-based identification and early warning of traffic violations based on graph neural networks according to claim 1, characterized in that, The outputs of the set of behavioral subjects, the set of behavioral elements, and the set of scene elements specifically include: The video image sequence data, spatial positioning data and scene element data corresponding to each timestamp are read from the standardized monitoring dataset to construct frame-level data units arranged in chronological order. Perform subject detection processing on each frame-level data unit to obtain a set of candidate boxes for human subjects and a set of candidate boxes for vehicle subjects, and generate a subject appearance feature vector and a subject spatial location feature vector for each subject candidate box; Multi-target association tracking is performed based on the similarity of the appearance feature vectors of adjacent timestamps and the continuity of the spatial location of the subject. Subject identifiers are assigned to the candidate boxes of personnel and vehicles and subject trajectory indexes are generated. The results are then summarized to obtain the set of behavioral subjects. For each subject identifier in the set of behavioral subjects, extract the subject posture key point sequence, subject motion state sequence and subject interaction object sequence in continuous time segments based on video image sequence data, and encapsulate them into a candidate set of action event elements; Perform action category determination and action time boundary location on the candidate set of action event elements to generate a set of action event elements; Trajectory reconstruction is performed based on the main trajectory index and spatial positioning data to generate a trajectory element set. The action event element set and the trajectory element set are then combined to obtain a behavior element set. Scene element extraction is performed based on scene element data and video image sequence data to generate a set of regional elements and a set of facility elements. These are then combined to obtain a set of scene elements, and the sets of behavioral subjects, behavioral elements, and scene elements are output.

4. The method for AI-based identification and early warning of traffic violations based on graph neural networks according to claim 1, characterized in that, The acquisition of the behavior graph data sequence specifically includes: A sliding time window sequence is constructed based on the timestamp order of the standardized monitoring dataset, and the standardized monitoring dataset is segmented according to the start and end times of each time window to obtain a time window segment data set that corresponds one-to-one with each time window. For each time window, the data set of the time window segment corresponding to each time window is selected from the set of behavioral subjects, the subject identifiers whose timestamps fall into the current time window are selected, a set of behavioral subject nodes is constructed, and subject category field, subject identifier field, subject timestamp field and subject spatial location field are written for each behavioral subject node; For each time window, the data set of the time window segment corresponding to each time window is selected from the set of behavioral elements. Action event elements and trajectory elements whose timestamps fall into the current time window are selected to construct a set of behavioral element nodes. For each action event element node, an action type field, an action start time field, an action end time field, and an action confidence field are written. For each trajectory element node, a trajectory point sequence field, a trajectory point timestamp sequence field, and a trajectory motion feature field are written. For each time window, the data set of the time window segment corresponding to each time window is selected from the scene element set to filter the regional elements and facility elements that are related to the spatial range of the current time window, construct the scene element node set, and write the regional type field, regional boundary field and regional spatial positioning field for each regional element node, and write the facility type field, facility location field and facility availability status field for each facility element node. The set of nodes for the main actors, the set of nodes for the main actors, and the set of nodes for the elements of the scene are merged to obtain a node set; Construct temporal relationship edges based on the temporal relationships between action event element nodes; Construct spatial relationship edges based on the spatial relationship between behavioral subject nodes and scene element nodes; Construct interaction relationship edges based on the attribution relationship between the behavioral subject node and the behavioral element node; The set of nodes and the set of edges consisting of temporal relationship edges, spatial relationship edges and interaction relationship edges are encapsulated into behavioral graph data corresponding to the current time window, and arranged in chronological order of the time windows to obtain a behavioral graph data sequence.

5. The method for AI-based identification and early warning of traffic violations based on graph neural networks according to claim 1, characterized in that, The specific steps for obtaining the set of node feature matrices and the set of edge feature matrices include: For a behavioral graph data sequence arranged in chronological order, the node set and edge set corresponding to each time point are read sequentially. Perform feature encoding on the behavioral subject nodes at each time step to generate a set of feature vectors for the behavioral subject nodes; Perform action event node feature encoding on action event element nodes at each time point to generate a set of action event element node feature vectors; Perform trajectory node feature encoding on trajectory element nodes at each time point to generate a set of trajectory element node feature vectors. Then, summarize the set of action event element node feature vectors and the set of trajectory element node feature vectors to obtain a set of behavior element node feature vectors. Perform scene node feature encoding on the regional element nodes and facility element nodes at each time point to generate a set of scene element node feature vectors; The feature vector sets of the subject node, the feature vector sets of the behavior element node, and the feature vector sets of the scene element node at the same time are aligned and spliced ​​according to the writing order of the nodes in the node sets to generate the node feature matrix corresponding to each time. The node feature matrix of each time is then stored in the node feature matrix set in chronological order. Perform edge feature encoding on the temporal relation edges, spatial relation edges, and interaction relation edges at each time point to generate a set of feature vectors for temporal relation edges, a set of feature vectors for spatial relation edges, and a set of feature vectors for interaction relation edges; The feature vector sets of temporal relation edges, spatial relation edges, and interaction relation edges at the same time are aligned and concatenated according to the writing order of the edges in the edge sets to generate the edge feature matrix corresponding to each time. The edge feature matrix of each time is then stored in the edge feature matrix set in chronological order.

6. The method for AI-based identification and early warning of traffic violations based on graph neural networks according to claim 1, characterized in that, The specific methods for obtaining the set of graph representations of the violations include: For a behavioral graph data sequence arranged in chronological order, the node set and edge set corresponding to each time point are read sequentially and aligned with the node feature matrix set and edge feature matrix set in time. The behavior graph data sequence is divided into behavior stages based on the temporal evolution of the node set and edge set. The behavior graph data sequence is divided into the pre-behavior stage, the behavior formation stage, and the behavior completion or diffusion stage, and a behavior stage identifier is written for each time point. Based on the behavior stage identifier, the node state update rules and information propagation constraints are configured differently. Prior suppression propagation constraints are set for the behavior pre-occurrence stage, interaction enhancement propagation constraints are set for the behavior formation stage, and diffusion suppression and stable propagation constraints are set for the behavior completion or diffusion stage. Semantic mapping and fusion processing are performed based on the set of behavioral elements and the set of scene elements to determine the semantic parameters of violation risk. At each time step, the node feature matrix, edge feature matrix, node set, and edge set are input into the temporal modeling structure of the improved DGNN network. Based on the edge set, message computation and message aggregation are performed on adjacent nodes to obtain the node message aggregation result. The improved DGNN network, while maintaining the graph representation learning structure of the original network, makes targeted adjustments to the internal processing logic of the time modeling structure and the state update and evolution structure. Specifically, it introduces a violation behavior stage perception mechanism to segment the time evolution of the dynamic graph structure and introduces violation risk semantic constraints in the state evolution process of the graph neural network. At each moment, the node's historical state, the node message aggregation result, and the violation risk semantic parameters are fed into the state update and evolution structure as input items, and the node state evolution update is performed to obtain the node's current state. In the graph representation learning structure, a set of node representations and a set of edge representations are generated based on the current state of the nodes and the set of edges. In the readout substructure of the graph representation learning structure, the node representation set and edge representation set are aggregated and summarized according to time and behavior stage to obtain the subgraph representation set, and the node representation set, edge representation set and subgraph representation set are encapsulated together into the violation behavior graph representation set.

7. The method for AI-based identification and early warning of traffic violations based on graph neural networks according to claim 1, characterized in that, The generation of the violation identification result set specifically includes: Read the node representation set, edge representation set, and subgraph representation set corresponding to each time step sequentially from the graph representation set of violation behavior, and construct a set of inferenceable representation units based on the node representation set and edge representation set; Based on the temporal and interactive relationship edge representations in the edge representation set, the subject representation unit and action representation unit are associated and combined to generate a set of subject-action association pairs; Based on the spatial relationship edge representation in the edge representation set, spatial association matching is performed on the subject action association pair set and the scene element node representation to generate a subject action association pair set with position constraints, and the violation occurrence location field is written for the subject action association pair that meets the spatial association conditions. Perform violation type inference calculation on the set of subject action associations with position constraints to generate a candidate set of violation types; Score normalization and threshold filtering are performed on the candidate set of violation types to generate a violation type field, and the violation occurrence time field is determined based on the action start time field and the action end time field. Based on the subject identifier field in the subject action association pair, the violation subject field is determined, and the violation type field, violation subject field, violation occurrence time field, and violation occurrence location field are encapsulated to generate a violation identification result set.

8. The method for AI-based identification and early warning of traffic violations based on graph neural networks according to claim 1, characterized in that, The generation of the violation risk assessment result set specifically includes: The violation type field, violation subject field, violation occurrence time field, and violation occurrence location field are read sequentially from the violation identification result set. The node representation set, edge representation set, and subgraph representation set corresponding to the time of the violation occurrence time field are read from the violation behavior graph representation set. Risk benchmark values ​​are generated by performing risk benchmark mapping processing based on the violation type field and the violation location field. Based on the risk intensity extraction and processing of the execution subject's risk intensity set based on the graph representation of the violation behavior, a subject risk intensity value is generated; Based on the violation occurrence time field, perform trend risk calculation processing on the set of violation behavior graph representations to generate trend risk values; Risk scoring is calculated based on the risk benchmark value, the subject risk intensity value, and the trend risk value to generate risk score fields. The risk score field is matched with the set of level thresholds to output the corresponding risk level field. The risk level field and the risk score field are then encapsulated to generate a set of violation risk assessment results.

9. The method for AI-based identification and early warning of traffic violations based on graph neural networks according to claim 1, characterized in that, The generation of the structured violation warning information specifically includes: The risk level field in the violation risk assessment result set is matched with the warning level trigger threshold set, and the risk score field is compared with the score trigger threshold set. When the interval matching result meets the warning trigger level condition and the threshold comparison result meets the warning trigger score condition, the warning trigger mark field is set as the trigger mark, and the warning trigger mark field is generated. For records where the warning trigger flag field is a trigger flag, the warning level is determined according to the level mapping rules, and a warning level field is generated; Triggering criteria fields are generated based on risk scoring fields, risk benchmark values, subject risk intensity values, and trend risk values. A timestamp field is generated based on the violation occurrence time field; a triggering subject field is generated based on the violation subject field; and a triggering location field is generated based on the violation occurrence location field. The warning level field, trigger basis field, timestamp field, trigger subject field, and trigger location field are encapsulated in a structured manner to generate structured violation warning information.

Images