Vision-based expressway abnormal event intelligent identification system

Abstract

The invention discloses a vision-based highway abnormal event intelligent identification system, and particularly relates to the technical field of computer vision and intelligent traffic, and the system comprises an event center dynamic representation module, a multi-modal causal reasoning module, a memory enhancement identification module and a meta learning optimization module. Abnormal events are dynamically captured through an event triggering mechanism and a center point detection network, environmental interference is eliminated by utilizing causal intervention and anti-factual reasoning, situational recognition is realized by adopting a memory bank, and rapid model self-adaption is realized based on a meta-learning framework. The method effectively solves the problems that a traditional method is high in false alarm rate and poor in environmental adaptability, the recognition accuracy and the system robustness are remarkably improved, meanwhile, the continuous learning ability is achieved, and the method is suitable for complex and changeable expressway scenes.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent transportation technology, and more specifically, to a vision-based intelligent recognition system for abnormal events on highways. Background Technology

[0002] With the rapid development of intelligent transportation systems, computer vision-based automatic identification technology for highway anomalies has become a key means of ensuring road traffic safety. Traditional identification methods mainly rely on fixed target detection models (such as YOLO, FCF, etc.) combined with predefined rules or simple temporal analysis. These methods locate and track vehicles in video streams to determine whether there are anomalies such as congestion, accidents, or illegal parking. However, these methods face significant challenges in the complex and ever-changing real-world highway scenarios. Their core shortcomings lie in the fragmentation of feature representation and shallow semantic understanding. They typically rely on manually designed features or general deep features, making it difficult to effectively capture the overall semantics and dynamic evolution of anomalies in the spatiotemporal dimensions, resulting in insufficient ability to identify complex events (such as chain-reaction rear-end collisions or abnormal flows induced by temporary traffic control).

[0003] Another prominent problem with existing technical solutions is their high environmental sensitivity and false alarm rate. Highway monitoring scenarios are greatly affected by environmental factors such as changes in lighting, weather conditions (rain, snow, fog), and camera shake. Traditional methods lack the ability to effectively distinguish between the essential characteristics of events and environmental interference factors, easily misjudging non-abnormal conditions such as shadows, tree swaying, and rain / snow reflections as events, generating a large number of false alarms. In addition, existing methods generally lack deep causal reasoning mechanisms and cannot answer the key question of "whether the event was caused by environmental factors," relying solely on pattern matching based on apparent correlations, which limits the robustness and reliability of the system in complex environments.

[0004] Furthermore, existing systems suffer from significant deficiencies in their adaptive capabilities and knowledge accumulation mechanisms. Most systems employ static models; once deployed, their recognition logic and knowledge base become fixed, making it difficult to adapt to unique traffic patterns on different road segments, seasonal changes, and newly emerging rare event types. Simultaneously, these systems lack effective continuous learning capabilities, failing to accumulate experience and optimize performance from historical recognition results, and are prone to catastrophic forgetting problems. The models also exhibit weak generalization ability; models trained and optimized on one road segment often experience a sharp performance drop when directly transferred to another road segment with drastically different characteristics. These limitations collectively restrict the large-scale and reliable application of vision-based highway anomaly event recognition systems in diverse and complex environments. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a vision-based intelligent recognition system for highway anomaly events.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A vision-based intelligent identification system for abnormal events on highways includes the following modules: The event center dynamic representation module dynamically captures abnormal events on highways through event triggering mechanisms and center point detection networks. It aggregates multi-scale features using spatiotemporal graph convolution and establishes event evolution trajectory tracking to form a complete event lifecycle representation system. The multimodal causal reasoning module constructs a structured causal model to distinguish the essential characteristics of events from environmental interference factors. It eliminates environmental confounding effects through causal intervention reasoning and counterfactual scenario simulation, and finally outputs a reliable causal confidence score based on adaptive weight fusion of multiple evidence sources. The memory-enhanced recognition module establishes a multimodal memory bank to store historical event knowledge, uses attention mechanisms and multi-dimensional reordering to achieve accurate memory retrieval, and constructs an event recognition system with continuous learning capabilities through contextualized recognition and dynamic memory update mechanisms. The meta-learning optimization module enables the model to quickly adapt to new road segments based on the meta-learning framework. It balances recognition performance indicators through multi-objective optimization and significantly improves system inference efficiency while ensuring accuracy by utilizing a parameter dynamic routing mechanism.

[0007] Specifically, the event center dynamic representation module includes: An event-triggered mechanism is adopted, which triggers event detection when the pixel-level optical flow change between consecutive video frames exceeds an adaptive threshold. The adaptive threshold is dynamically adjusted according to ambient brightness and traffic flow. The event center point cloud representation is constructed through a center point detection network. The center point detection network adopts a heatmap-based encoder-decoder structure. The encoder backbone network is HRNet-32, and the decoder consists of three upsampling layers and convolutional layers. The output is a single-channel heatmap that is proportional to the size of the input image. The center point detection network uses FocalLoss as the loss function and is trained with the Adam optimizer. The training process employs a multi-stage optimization strategy, including cosine annealing learning rate scheduling, online hard example mining, gradient pruning, and exponential moving average parameter updates.

[0008] Specifically, the event center dynamic representation module also includes: A spatiotemporal graph structure is constructed with the event center point as the graph node. Nodes within 50 pixels of Euclidean distance in space and nodes with the same tracking ID within 5 adjacent frames in time are connected to each other. The edge weight is determined by the spatiotemporal distance and motion consistency. A three-layer spatiotemporal graph convolutional network combined with a graph attention mechanism is used to aggregate neighborhood information and generate enhanced event representations; By using a hierarchical clustering algorithm, events are grouped into three levels—macro, meso, and micro—based on their spatiotemporal similarity. A lifecycle model is constructed for each event instance to record its entire process from generation to development to extinction and to extract trajectory dynamics features.

[0009] Specifically, the implementation of the multimodal causal reasoning module includes: A structured causal model is constructed, which includes five core variables: event type E, visual features V, environmental conditions C, traffic state T, and spatiotemporal context S. The causal relationship paths between the variables are defined as C→V, C→T, S→E, T→E, and V→E. Environmental factors are decoupled through conditional adversarial training. A gradient inversion layer is used for adversarial training of the environment domain classifier to learn environment-invariant visual feature representations.

[0010] Specifically, the multimodal causal reasoning module further includes: For suspected anomalous events, a do-calculus intervention is performed to sever the association between environmental condition C and visual feature V and fix V as the observed value. The confounding effect is assessed by calculating the change in event probability before and after the intervention, and it is determined whether the false alarm is caused by the environment. A decoupled representation of features is learned using a variational autoencoder, counterfactual samples are generated by fixing the environment encoding, and the semantic distance between the actual event and the counterfactual event is calculated to assess the reliability of the event. Based on evidence from three aspects—actual observation, causal intervention, and counterfactual reasoning—an adaptive weighting mechanism is used to generate a causal credibility score. Only events with scores exceeding a preset threshold are included in the subsequent processing flow.

[0011] Specifically, the implementation of the memory-enhanced recognition module includes: Construct a multimodal memory bank, including a visual memory matrix that stores the appearance features of typical events, a semantic memory graph that records the logical relationships between events, and a spatiotemporal pattern memory chain that preserves recurring spatiotemporal patterns; K-nearest neighbor search based on attention mechanism is used for memory retrieval. The attention score between the current event query vector and the key vector in the memory bank is calculated to obtain the most relevant memory segments. A three-dimensional evaluation system consisting of semantic similarity, spatiotemporal similarity, and event structure similarity is introduced. The weight coefficients are adaptively adjusted according to the contextual characteristics of the query event to reorder the retrieved memory fragments.

[0012] Specifically, the memory-enhanced recognition module further includes: The current event features and the reordered memory fragments are input into the relational reasoning network. The current observations and historical knowledge are integrated through a multi-round message passing mechanism to output contextualized event classification results. An importance score is calculated based on the confidence level of an event and the recent frequency of similar events. Event features that are above the threshold are written into the long-term memory bank, and the rest are stored in the short-term memory buffer. A memory replay mechanism is executed periodically to sample memory fragments from the short-term memory buffer and retrain the relational reasoning network together with long-term memory. At the same time, an explanatory identification report containing similar historical cases, key discriminative features, and confidence decomposition is generated.

[0013] Specifically, the implementation of the meta-learning optimization module includes: Data is collected from multiple highway segments with different characteristics to construct a meta-training task set. Each task contains a support set and a query set. The tasks share common knowledge while maintaining characteristic differences. The MAML algorithm framework is used to train a four-layer convolutional neural network as the basic model. The model is equipped with the ability to quickly adapt to a small number of samples by fine-tuning the inner loop for specific road segments and optimizing the initialization parameters in the outer loop. When deployed to a new road segment, the monitoring data from the previous 24 hours is used as a support set for online meta-adaptation, and key parameters are updated through gradient descent to absorb the characteristics of the road segment.

[0014] Specifically, the meta-learning optimization module also includes: The Pareto optimization method is used to balance the three objectives of accuracy, recall, and false positive rate, and the objective weights are adjusted according to the actual needs of different road segments. A supernetwork consisting of an encoder and a parameter generator is constructed. The encoder maps the input event features into a conditional vector, and the parameter generator dynamically generates the convolutional layer weight biases of the main recognition network according to the magnitude of the vector. This enables conditional computation with lightweight parameters in simple scenarios and activation of complex computational paths in complex scenarios.

[0015] The technical effects and advantages of this invention are as follows: By combining event-centric dynamic representation with multimodal causal reasoning, the accuracy and robustness of anomalous event identification are fundamentally improved. Event-centric representation effectively overcomes the feature fragmentation problem caused by traditional bounding box detection, and can completely capture the spatiotemporal evolution of events; while the causal reasoning mechanism, through intervention and counterfactual analysis, accurately removes environmental confounding factors, significantly reducing the false alarm rate caused by interference from lighting, weather, etc., enabling the system to maintain high reliability even in complex environments.

[0016] The system leverages memory enhancement and meta-learning mechanisms to achieve continuous self-evolution and efficient adaptation. The memory enhancement module accumulates valuable road segment-specific knowledge through a multimodal memory bank and contextualized recognition, endowing the system with continuous learning capabilities. The meta-learning optimization module enables the model to quickly adapt to new road segments using a small number of samples. Combined with parameter dynamic routing technology, it significantly improves computational efficiency while ensuring recognition accuracy, laying a solid foundation for the large-scale deployment and application of the system. Attached Figure Description

[0017] Figure 1 This is a system block diagram of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, the modules of the vision-based intelligent recognition system for abnormal events on highways are as follows: The event center dynamic representation module dynamically captures abnormal events on highways through an event triggering mechanism and a center point detection network. It aggregates multi-scale features using spatiotemporal graph convolution and establishes an event evolution trajectory tracking system to form a complete event lifecycle representation system. The specific process is as follows: The processing flow is initialized using an event-triggered mechanism. Event detection is triggered when the pixel-level optical flow change between consecutive video frames exceeds an adaptive threshold. The threshold is dynamically adjusted according to ambient brightness and traffic flow to avoid meaningless calculations. We construct an event center point cloud representation and use a center point detection network to replace bounding box regression. Each event is defined by three core parameters: center point coordinates, influence radius, and event intensity. We obtain a 256-dimensional event embedding vector by directly regressing through the network. The center point detection network employs a heatmap-based encoder-decoder structure. The encoder backbone uses HRNet-32 to extract multi-resolution features while maintaining high spatial accuracy. The decoder consists of three upsampling layers and convolutional layers, ultimately outputting a single-channel heatmap proportional to the input image size (e.g., 1:4), where each peak point corresponds to the center location of a potential event. The network uses FocalLoss as the loss function to address the imbalance between positive and negative samples (event center and background). During training, the Adam optimizer is used, with an initial learning rate set to... .

[0020] A multi-stage optimization strategy is employed during training: First, cosine annealing is used for learning rate scheduling, with the learning rate determined by the formula... Changes, among which The learning rate at the current moment. To minimize the learning rate, To achieve the maximum learning rate, , This represents the current iteration number. The maximum number of iterations is set to 50 epochs; In the middle of training (after the 30th epoch), an online hard example discovery strategy is introduced, defining hard examples as those whose loss function values ​​rank in the top 20%. ,in Let i be the loss function value for the i-th sample. for, This represents the 80th percentile of the loss values ​​in the current batch; For these difficult examples, a variant of the focus loss function is used. ,in The loss value for the difficult sample. To predict probabilities, The adjustment factor for focus loss is set to 2. Number of difficult cases; Simultaneously, a gradient clipping mechanism is introduced, where the gradient norm exceeds a threshold. At that time, according to the formula Pruning is performed to ensure training stability, among which The gradient after clipping. This is the original gradient vector. The L2 norm of the gradient; Finally, at the end of training, exponential moving average is used to update the model parameters. The parameter update formula is as follows: ,in These are the model parameters after exponential moving average. These are the current model parameters. The decay factor was set to 0.995 to obtain a more robust final model.

[0021] Spatiotemporal graph convolutional aggregation is implemented, using event center points as graph nodes to construct a spatiotemporal graph structure. The connections between nodes follow these rules: spatially, nodes with an Euclidean distance of less than 50 pixels are connected; temporally, nodes with the same tracking ID within 5 adjacent frames are connected; edge weights are determined jointly by spatiotemporal distance and motion consistency (calculated through cosine similarity of optical flow directions). The aggregation process uses a three-layer spatiotemporal graph convolutional network (ST-GCN), with each layer outputting 128 dimensions. Neighborhood information is aggregated through a graph attention mechanism (GAT), and finally, the updated features of each node are concatenated with the original features to generate an enhanced 512-dimensional event representation. Multi-granularity event clustering is performed, and a hierarchical clustering algorithm is used to group events into three levels based on spatiotemporal similarity: macro events, meso events, and micro events. Each level maintains an independent event memory unit.

[0022] Establish an event evolution trajectory tracking system, construct a life cycle model for each event instance, record the entire process from generation and development to extinction, and extract trajectory dynamic features as the basic input for subsequent causal inference.

[0023] The multimodal causal reasoning module constructs a structured causal model to distinguish between the essential characteristics of an event and environmental interference factors. It eliminates environmental confounding effects through causal intervention reasoning and counterfactual scenario simulation, and finally outputs a reliable causal confidence score based on adaptive weighted fusion of multiple evidence sources. The specific process is as follows: Construct a causal graph for a highway scenario, defining five core variables: event type E, visual features V, environmental conditions C, traffic state T, and spatiotemporal context S, and establish a complete causal network structure to clarify the causal relationship paths between variables.

[0024] Environmental factors are decoupled, and the visual feature representation that is invariant to the environment is learned through conditional adversarial training. This ensures that the same event has consistent feature representation under different weather and lighting conditions. Specifically, a gradient inversion layer is used to implement adversarial training of the environmental domain classifier.

[0025] Perform causal intervention reasoning, conduct do-calculus intervention for each suspected abnormal event, evaluate the change in event probability after fixing environmental variables, and determine that the probability of several prognostic events has decreased significantly as a false alarm caused by the environment; otherwise, it is confirmed as a real anomaly.

[0026] The constructed structured causal model (SCM) comprises five core variables: event type E, visual feature V, environmental condition C, traffic state T, and spatiotemporal context S; its causal graph structure is defined as: C→V, C→T, S→E, T→E, V→E. For each suspected anomalous event, execution is performed... The intervention involves severing the boundary between the environment C and the visual feature V, fixing V to the observed value v. The confounding effect is assessed by calculating the probability change τ = P(E|do(V=v),C,T,S) - P(E|V=v,C,T,S). If |τ|>0.2, it is considered a false alarm caused by the environment. In the formula: τ is the magnitude of the intervention effect (probability change), P(E|do(V=v),C,T,S) is the conditional probability after the intervention, and P(E|V=v,C,T,S) is the conditional probability before the intervention. Counterfactual scenario simulations are conducted. For observed event instances (V=v, C=c,...), counterfactual samples (V=v', C=c,...) are generated, where v' is obtained by perturbing the feature vector v in the latent space along an environment-invariant direction. Specifically, a variational autoencoder (VAE) is used to learn the decoupled representation of the features, and sampling is performed with the environment encoding fixed. The semantic distance between the actual event and the counterfactual event is calculated as: d=1-cossim(f(v),f(v')), where d is the semantic distance, cossim() is the cosine similarity function, f(v) is the feature representation of the actual event, f(v') is the feature representation of the counterfactual event, and f is the semantic extraction function. When d<0.1, it indicates that the event is not sensitive to environmental changes and has high reliability.

[0027] The system outputs a causal confidence score, integrating evidence from observation, causal intervention, and counterfactual reasoning to generate a causal confidence score for each event. Only events with scores exceeding 0.85 proceed to further processing. An adaptive weighting mechanism, rather than a fixed weighting, is used when integrating evidence from observation, intervention, and counterfactual reasoning. The specific process is as follows: First, an accuracy tracker is established for each source of evidence to record the results of the most recent N=100 manual verifications or high-confidence automatic verifications. The historical accuracy of each source of evidence is calculated using a time decay weighting method: ,in Let be the accuracy of the i-th evidence source at time t. Let be the accuracy of the i-th evidence source at time t-1. As the attenuation factor, Determine whether evidence source i is correct in the current event (1 or 0); Calculate adaptive weights based on the updated accuracy: ,in Let be the weight of the i-th evidence source at time t. Let be the accuracy of the i-th evidence source at time t. Let be the accuracy of the j-th evidence source at time t. The temperature parameter controls the steepness of the weight distribution. Simultaneously, an evidence consistency constraint is introduced, and the correlation coefficient matrix between the scores of each evidence source is calculated. When a high degree of inconsistency occurs (i.e.) This automatically reduces the weight of all evidence sources and triggers a conservative decision-making mode. The final causal confidence score is calculated as follows: ,in The final causal confidence score, The standardized score (range 0-1) for each source of evidence. =0.1 is a conservative factor to ensure that there is no overconfidence when there is conflict of evidence; In addition, a weight stability check is set up to check the weight change over 10 consecutive events. At that time, the weight update is frozen to prevent overfitting, where The magnitude of the weight change, Let be the weight of the i-th evidence source at time t-10.

[0028] The memory-enhanced recognition module establishes a multimodal memory bank to store historical event knowledge, employs attention mechanisms and multi-dimensional reordering to achieve accurate memory retrieval, and constructs an event recognition system with continuous learning capabilities through contextualized recognition and dynamic memory update mechanisms; the specific process is as follows: A multimodal memory bank is constructed, consisting of three parts: a visual memory matrix, a semantic memory graph, and a spatiotemporal pattern memory chain. The visual memory matrix stores the appearance features of typical events, the semantic memory graph records the logical relationships between events, and the spatiotemporal pattern memory chain preserves the spatiotemporal patterns of recurring events.

[0029] The system performs memory retrieval and alignment, searches for similarity in the memory bank for the features of the current event, uses an attention mechanism to calculate the association weight with historical memories, and retrieves the K most relevant memory fragments as identification references.

[0030] Memory retrieval employs an attention-based K-nearest neighbor search. Given the current event query vector q, it computes its relationship with all key vectors in the memory. Attention score: ,in The attention score for the i-th memory segment. For query vector, Let i be the key vector of the i-th memory segment. Let d be the learnable weight matrix, where d is the vector dimension. Retrieve the K=10 memory segments with the highest attention scores. and their corresponding association weights As a reference for identification.

[0031] After the initial retrieval is completed, a re-ranking mechanism based on multi-dimensional similarity is introduced to improve retrieval quality; firstly, a three-dimensional similarity evaluation system is constructed: semantic similarity. , To query the semantic feature vector of the event, The semantic feature vector of the memory segment is used to calculate the cosine similarity based on the semantic feature vector; Spatiotemporal similarity ,in For spatiotemporal similarity, To query the timestamp of the event, For the timestamp of a memory fragment, To query the spatial coordinates of an event, The spatial coordinates of the memory fragment. (1 hour) is the time decay parameter. Meters are spatial attenuation parameters; Event structure similarity ,in For event structure similarity, To query the adjacency matrix of the event graph structure, For the adjacency matrix of the memory fragment event graph structure, This represents the Frobenius norm.

[0032] The weight coefficients are adaptively adjusted based on the context characteristics of the current query: when the query event is in a complex scenario with multiple intertwined events, the weight coefficients are set. ; When the query event has a clear spatiotemporal pattern, set ; When the query event structure is complex, set .

[0033] Final similarity score , These are the semantic similarity weight coefficient, the spatiotemporal similarity weight coefficient, and the event structure similarity weight coefficient, respectively; the 10 memory fragments initially retrieved are then weighted according to... The fragments were reordered, and the top 5 best fragments were selected for subsequent identification. At the same time, the contribution of each dimension was recorded for the generation of explanatory reports.

[0034] Contextualized event recognition is performed by inputting current event features and retrieved memory fragments into a relational reasoning network. Through a multi-round message passing mechanism, current observations and historical knowledge are integrated to output event classification results based on contextual understanding.

[0035] Memory updating and consolidation are carried out. The memory bank is dynamically updated based on the importance score of the recognition results. Important event features are written into long-term memory, while ordinary event features are temporarily stored in short-term memory. Key knowledge is consolidated regularly through a memory playback mechanism.

[0036] Memory updates follow an importance-weighted mechanism. Each event has an importance score. From the formula Calculation, where To identify confidence levels, The frequency of similar events occurring within a recent time window. and These are the confidence weighting coefficient and the frequency weighting coefficient (default values ​​are 0.7 and 0.3, respectively). Importance scores above the threshold... Event characteristics are written into long-term memory, otherwise stored in short-term memory buffer. The system performs a memory replay every 24 hours, randomly sampling a batch of memory fragments from the buffer and retraining the relational reasoning network together with long-term memory to consolidate key knowledge and prevent catastrophic forgetting.

[0037] The system generates interpretable identification reports that not only output the event type but also provide information such as similar historical cases, key discriminant features, and confidence decomposition, enhancing the interpretability and credibility of the results.

[0038] The meta-learning optimization module, based on the meta-learning framework, enables the model to quickly adapt to new road segments. It balances recognition performance metrics through multi-objective optimization and utilizes a dynamic parameter routing mechanism to significantly improve system inference efficiency while maintaining accuracy. The specific process is as follows: A meta-training task set is constructed by collecting data from multiple highway segments with different characteristics. A recognition task is constructed for each segment, forming a meta-learning support set and query set. The tasks share common knowledge while maintaining their unique characteristics.

[0039] A model-agnostic meta-learning approach was implemented, using the MAML algorithm framework to train the base model. This enabled the model to quickly adapt to new road segments using a small number of samples. The inner loop fine-tuned the model for specific road segments, while the outer loop optimized the initialization parameters. The base model was a four-layer convolutional neural network. During the meta-training phase, each task's support set contained 16 samples for that road segment, and the query set contained 8 samples. The inner loop used gradient descent for 5 iterations of rapid adaptation with a learning rate of 0.01. The outer loop used the Meta-SGD optimizer to update the model's initial parameters with a meta-learning rate of 0.001. Meta-training was performed for 50,000 iterations on a task set containing data from 20 different road segments. Online meta-adaptation is performed. When the system is deployed to a new road segment, the monitoring data from the previous 24 hours is used as a support set for rapid model adaptation. Key parameters are updated through gradient descent, so that the model can absorb the characteristics of the road segment while maintaining its general capabilities.

[0040] Multi-objective optimization and balancing are carried out simultaneously, optimizing accuracy, recall, and false positive rate. The Pareto optimization method is used to find the optimal balance point, and the target weights are adjusted according to the actual needs of different road segments.

[0041] This system implements dynamic parameter routing by constructing a hypernetwork structure that dynamically generates model parameters based on input features. This enables conditional computation, using lightweight parameters for simple scenarios and activating more complex computational paths for complex scenarios, thus improving efficiency while maintaining performance. The hypernetwork consists of an encoder and a parameter generator. The encoder maps input event features to a conditional vector z. The parameter generator takes z as input and dynamically generates the weight biases for specific convolutional layers in the main recognition network through a fully connected layer. For simple scenarios (indicated by a smaller z magnitude), fewer parameters are generated, equivalent to a lightweight convolution; for complex scenarios (with a larger z magnitude), complete convolutional parameters are generated, activating more complex computational paths.

[0042] The above formulas are all dimensionless calculations. Dimensionless calculations can be performed using various methods such as standardization, which will not be elaborated here. The formulas are derived from software simulations based on a large amount of collected data, and the preset parameters in the formulas can be set by those skilled in the art according to the actual situation.

[0043] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, ATA hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state ATA hard disk.

[0044] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0045] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0046] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0047] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0048] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0049] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable ATA hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0050] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A vision-based intelligent recognition system for abnormal events on highways, characterized in that: Includes the following modules: The event center dynamic representation module dynamically captures abnormal events on highways through event triggering mechanisms and center point detection networks. It aggregates multi-scale features using spatiotemporal graph convolution and establishes event evolution trajectory tracking to form a complete event lifecycle representation system. The multimodal causal reasoning module constructs a structured causal model to distinguish the essential characteristics of events from environmental interference factors. It eliminates environmental confounding effects through causal intervention reasoning and counterfactual scenario simulation, and finally outputs a reliable causal confidence score based on adaptive weight fusion of multiple evidence sources. The memory-enhanced recognition module establishes a multimodal memory bank to store historical event knowledge, uses attention mechanisms and multi-dimensional reordering to achieve accurate memory retrieval, and constructs an event recognition system with continuous learning capabilities through contextualized recognition and dynamic memory update mechanisms. The meta-learning optimization module enables the model to quickly adapt to new road segments based on the meta-learning framework. It balances recognition performance indicators through multi-objective optimization and significantly improves system inference efficiency while ensuring accuracy by utilizing a parameter dynamic routing mechanism.

2. The vision-based intelligent identification system for abnormal events on highways according to claim 1, characterized in that, The event center dynamic representation module specifically includes: An event-triggered mechanism is adopted, which triggers event detection when the pixel-level optical flow change between consecutive video frames exceeds an adaptive threshold. The adaptive threshold is dynamically adjusted according to ambient brightness and traffic flow. The event center point cloud representation is constructed through a center point detection network. The center point detection network adopts a heatmap-based encoder-decoder structure. The encoder backbone network is HRNet-32, and the decoder consists of three upsampling layers and convolutional layers. The output is a single-channel heatmap that is proportional to the size of the input image. The center point detection network uses FocalLoss as the loss function and is trained with the Adam optimizer. The training process employs a multi-stage optimization strategy, including cosine annealing learning rate scheduling, online hard example mining, gradient pruning, and exponential moving average parameter updates.

3. The vision-based intelligent identification system for abnormal events on highways according to claim 2, characterized in that, The event center dynamic representation module also includes: A spatiotemporal graph structure is constructed with the event center point as the graph node. Nodes within 50 pixels of Euclidean distance in space and nodes with the same tracking ID within 5 adjacent frames in time are connected to each other. The edge weight is determined by the spatiotemporal distance and motion consistency. A three-layer spatiotemporal graph convolutional network combined with a graph attention mechanism is used to aggregate neighborhood information and generate enhanced event representations; By using a hierarchical clustering algorithm, events are grouped into three levels—macro, meso, and micro—based on their spatiotemporal similarity. A lifecycle model is constructed for each event instance to record its entire process from generation to development to extinction and to extract trajectory dynamics features.

4. The vision-based intelligent identification system for abnormal events on highways according to claim 1, characterized in that, The specific implementation of the multimodal causal reasoning module includes: A structured causal model is constructed, which includes five core variables: event type E, visual features V, environmental conditions C, traffic state T, and spatiotemporal context S. The causal relationship paths between the variables are defined as C→V, C→T, S→E, T→E, and V→E. Environmental factors are decoupled through conditional adversarial training. A gradient inversion layer is used for adversarial training of the environment domain classifier to learn environment-invariant visual feature representations.

5. The vision-based intelligent identification system for abnormal events on highways according to claim 4, characterized in that, The multimodal causal reasoning module also includes: For suspected anomalous events, a do-calculus intervention is performed to sever the association between environmental condition C and visual feature V and fix V as the observed value. The confounding effect is assessed by calculating the change in event probability before and after the intervention, and it is determined whether the false alarm is caused by the environment. A decoupled representation of features is learned using a variational autoencoder, counterfactual samples are generated by fixing the environment encoding, and the semantic distance between the actual event and the counterfactual event is calculated to assess the reliability of the event. Based on evidence from three aspects—actual observation, causal intervention, and counterfactual reasoning—an adaptive weighting mechanism is used to generate a causal credibility score. Only events with scores exceeding a preset threshold are included in the subsequent processing flow.

6. The vision-based intelligent identification system for abnormal events on highways according to claim 1, characterized in that, The specific implementation of the memory-enhanced recognition module includes: Construct a multimodal memory bank, including a visual memory matrix that stores the appearance features of typical events, a semantic memory graph that records the logical relationships between events, and a spatiotemporal pattern memory chain that preserves recurring spatiotemporal patterns; K-nearest neighbor search based on attention mechanism is used for memory retrieval. The attention score between the current event query vector and the key vector in the memory bank is calculated to obtain the most relevant memory segments. A three-dimensional evaluation system consisting of semantic similarity, spatiotemporal similarity, and event structure similarity is introduced. The weight coefficients are adaptively adjusted according to the contextual characteristics of the query event to reorder the retrieved memory fragments.

7. The vision-based intelligent identification system for abnormal events on highways according to claim 6, characterized in that, The memory-enhanced recognition module also includes: The current event features and the reordered memory fragments are input into the relational reasoning network. The current observations and historical knowledge are integrated through a multi-round message passing mechanism to output contextualized event classification results. An importance score is calculated based on the confidence level of an event and the recent frequency of similar events. Event features that are above the threshold are written into the long-term memory bank, and the rest are stored in the short-term memory buffer. A memory replay mechanism is executed periodically to sample memory fragments from the short-term memory buffer and retrain the relational reasoning network together with long-term memory. At the same time, an explanatory identification report containing similar historical cases, key discriminative features, and confidence decomposition is generated.

8. The vision-based intelligent identification system for abnormal events on highways according to claim 1, characterized in that, The specific implementation of the meta-learning optimization module includes: Data is collected from multiple highway segments with different characteristics to construct a meta-training task set. Each task contains a support set and a query set. The tasks share common knowledge while maintaining characteristic differences. The MAML algorithm framework is used to train a four-layer convolutional neural network as the basic model. The model is equipped with the ability to quickly adapt to a small number of samples by fine-tuning the inner loop for specific road segments and optimizing the initialization parameters in the outer loop. When deployed to a new road segment, the monitoring data from the previous 24 hours is used as a support set for online meta-adaptation, and key parameters are updated through gradient descent to absorb the characteristics of the road segment.

9. The vision-based intelligent identification system for abnormal events on highways according to claim 8, characterized in that, The meta-learning optimization module also includes: The Pareto optimization method is used to balance the three objectives of accuracy, recall, and false positive rate, and the objective weights are adjusted according to the actual needs of different road segments. A supernetwork consisting of an encoder and a parameter generator is constructed. The encoder maps the input event features into a conditional vector, and the parameter generator dynamically generates the convolutional layer weight biases of the main recognition network according to the magnitude of the vector. This enables conditional computation with lightweight parameters in simple scenarios and activation of complex computational paths in complex scenarios.

Images