High-speed danger early warning method and system based on machine vision

Abstract

The invention discloses a highway danger early warning method and system based on machine vision. The highway danger early warning method comprises the following steps of 1, collecting and preprocessing multi-source vision video data; 2, carrying out region division on image frames of the multi-source visual video data by adopting multi-scale super-pixel region aggregation; 3, extracting spatial context features and time sequence features in continuous image frames; 4, constructing a multi-target space-time interaction diagram according to the target feature vector set; 5, performing causal analysis on the multi-target space-time interaction diagram by using an improved NOTEARS algorithm, and generating a causal structure diagram containing a danger-triggered causal chain; and 6, carrying out risk scoring on each danger triggering causal chain, determining risk grade division, and outputting a corresponding early warning response instruction. According to the method, superpixel region aggregation and an improved NOTEARS algorithm are fused, and intelligent risk early warning of a high-speed scene is realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent sensing and hazard warning technology, and in particular to a high-speed hazard warning method and system based on machine vision. Background Technology

[0002] In high-speed environments, especially in complex urban traffic, highway intersections, or intelligent vehicle systems, rapid identification and early warning of potential hazards have always been key research directions in fields such as intelligent driving, traffic safety, and automated inspection. Existing machine vision-based hazard identification methods mainly rely on image information from a single viewpoint for target detection and motion trend prediction, such as using a forward-looking camera or simply fusing multiple images and extracting target trajectories and behavioral features through convolutional neural networks (CNNs) and long short-term memory networks (LSTMs). However, these methods often suffer from response delays, high false recognition rates, and severe missed detections in scenarios involving frequent multi-target interactions, severe blind spots, and unpredictable emergencies at high speeds, making it difficult to model and deduce potential hazard relationships between multiple targets.

[0003] Furthermore, existing technologies typically employ rule-based or shallow statistical risk assessment mechanisms, failing to delve deeply into the potential causal relationships between different targets during their time-series evolution. This results in an inability to effectively understand the triggering paths of risks. For example, some methods judge risk levels solely based on distance or speed differences between targets, lacking the ability to model the evolutionary trends, behavioral correlations, and temporal evolution patterns of target states, and thus failing to identify hidden hazard chains. In scenarios with multiple targets present simultaneously and complex high-speed convergence, the formation of hazards often exhibits causal transmission characteristics, such as a target's emergency lane change triggering a chain reaction of vehicles behind, or a target in the peripheral area suddenly cutting in, causing a target in the central area to be unable to avoid it. These causal chain evolution processes are difficult to accurately capture and quantify in existing systems.

[0004] Therefore, how to provide a high-speed hazard warning method and system based on machine vision is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a high-speed hazard warning method and system based on machine vision. This invention integrates multi-source visual information with an improved NOTEARS algorithm to construct a multi-target spatiotemporal interactive graph, enabling the identification and quantitative analysis of potential hazard triggering chains in high-speed motion scenes. Through a risk scoring and grading mechanism, it outputs tiered warnings and intervention commands, improving the accuracy and timeliness of risk perception in complex dynamic environments, and achieving intelligent hazard warning and safety decision support in high-speed scenarios.

[0006] The high-speed hazard warning method based on machine vision according to an embodiment of the present invention includes the following steps: Step 1: Collect and preprocess multi-source visual video data to generate multi-source visual video data with a unified structure; Step 2: Multi-scale superpixel region aggregation is used to divide the image frames of multi-source visual video data into regions to obtain a structured visual image block set; Step 3: Input the structured visual image patch set into the spatiotemporal graph convolutional encoder to extract the spatial context features and temporal series features within consecutive image frames, and generate a set of target feature vectors; Step 4: Construct a multi-target spatiotemporal interaction graph based on the target feature vector set; Step 5: Use the improved NOTEARS algorithm to perform causal analysis on the multi-objective spatiotemporal interaction graph to generate a causal structure graph containing causal chains that trigger dangers. Step Six: Assess the risk of each hazard-triggered causal chain, determine the risk level classification, and output the corresponding early warning response instructions.

[0007] Optionally, the multi-source visual video data includes a sequence of images from the front-view camera in the direction of travel, a sequence of images from the side-view cameras covering the blind spots on both sides, and an image of the surrounding dynamic environment from an aerial view taken by the UAV. The preprocessing includes grayscale normalization, brightness normalization, motion blur repair, edge enhancement, and time synchronization for different types of video images.

[0008] Optionally, step two specifically involves: For each image frame of the multi-source visual video data, perform initial superpixel segmentation. The initial superpixel segmentation steps are as follows: Calculate the two-dimensional spatial position of each pixel in the image frame in the image coordinate system; The image frame is divided into regular grid regions with a fixed step size, which serve as the initial cluster centers; Within a fixed radius near each initial cluster center, the color feature value of each pixel is extracted, and the color feature value includes the brightness values ​​of the red, green and blue channels; For each pixel in the image frame, the difference in color feature value and spatial distance between it and each initial cluster center are calculated simultaneously, and then fused into a comprehensive distance index by a preset weighting ratio. The spatial distance is the Euclidean spatial distance between the pixel and the initial cluster center. Each pixel is assigned to an initial cluster center with the minimum comprehensive distance, generating the corresponding superpixel unit; Three sets of scale parameters are set for each superpixel unit, and the average color feature value, center coordinates and boundary texture gradient of each superpixel unit are recalculated at each scale. At the current scale, all adjacent superpixel units are traversed, and a region merging operation is performed. The region merging operation is to merge superpixel units whose color feature value difference is lower than a set threshold, whose center coordinates are smaller than a specified pixel range, and whose texture direction consistency between boundaries is higher than a preset ratio into a new superpixel unit. The superpixel units that have been aggregated at three different scales are fused to obtain a set of structured visual image blocks.

[0009] Optionally, step three specifically includes: The set of structured visual image patches is constructed into a frame-level graph sequence in chronological order; The frame-level graph sequence is input into the graph convolutional encoding module of the spatiotemporal graph convolutional encoder. Within each image frame, the spatial context features of each image block within the current image frame are extracted through graph convolution operations. The spatial context features include the texture, color, edge information of the image block and the semantic association relationship with neighboring image blocks. Between adjacent image frames, temporal convolutional units are used to model the state changes of the same image patch in consecutive image frames, extract the motion trend and change magnitude of the image patch in the time dimension, and obtain time series features; Spatial context features and temporal series features are fused to form a spatiotemporal joint feature representation of each image patch in a continuous image frame sequence. Multi-image frame information is then integrated through a feature aggregation mechanism to generate spatiotemporal context-enhanced image patch feature vectors. The image patch feature vector is input into the target recognition submodule, and the dynamic target category corresponding to each image patch is identified based on a multi-class classifier. The dynamic target categories include vehicles, pedestrians, and obstacles. Extract the corresponding target feature vectors from the identified target category image blocks and generate a target feature vector set. The target feature vectors include the center position coordinates of the target category image block in the image coordinate system, the motion speed in the time dimension, the motion direction, and the appearance feature representation.

[0010] Optionally, step four specifically includes: Based on the target feature vector set, a multi-target spatiotemporal interaction graph is constructed. The construction steps are as follows: Different target feature vectors are used as graph nodes in the multi-target spatiotemporal interaction graph; Pair up all graph nodes and calculate the spatial distance, velocity difference, and motion direction angle difference between each pair of graph nodes to obtain the feature difference vector of the graph node pair. Map the feature difference vectors to the Riemannian manifold space; Analyze the geodesic path from one graph node to another in the Riemannian manifold space, and compare the deviation of the geodesic path from the straight-line distance between the two nodes; The measured spatial distance, velocity difference, motion direction angle difference, and straight-line distance deviation are weighted and combined according to preset weights to obtain the edge weights between graph node pairs.

[0011] Optionally, the improved NOTEARS algorithm is specifically as follows: For each pair of graph nodes in a multi-objective spatiotemporal interaction graph, a causal connection hypothesis is established sequentially. The steps for establishing the connection hypothesis are as follows: For each pair of graph nodes, obtain the timestamp of the corresponding image frame, determine the temporal sequence, and ascertain whether the temporal causal sequence is satisfied. Extract the center coordinates of graph nodes in consecutive image frames at fixed time intervals and calculate the spatial distance; If a node in one graph is earlier than a node in another graph in time, and the spatial distance between the two decreases continuously over multiple time frames, then it is determined that there is a spatial proximity causal trend. Analyze the velocity and direction of motion of the graph nodes to determine the direction of state change; For graph node pairs that satisfy the temporal causal sequence and simultaneously satisfy the spatial proximity causal trend and the consistency of the direction of state change in three or more consecutive time frames, establish a causal connection hypothesis and record the starting node and the target node. Based on the connectivity hypothesis, the state change order of each graph node pair in different time frames is analyzed. Graph node pairs that show a fixed sequential state change relationship in multiple time frames and have the same direction of state change in at least three consecutive time windows are selected as causal connectivity candidates. Loop closure detection is performed on all causal connection candidates. If a group of causal connections forms a closed path, the connection edges are removed according to the time order and edge weight, generating a causal structure graph. The removal steps are as follows: Preserve the connecting edges that originate from the node that occurred earlier in time; If the time is equivalent, then retain the connection edge with the higher edge weight; If the edge weights are equivalent, then randomly remove one connecting edge until there is no closed path left; The directed edges of the causal structure graph represent the triggering or guiding effect of one target state on another target state, the direction of the directed edges represents the causal sequence, and the weight of the directed edges represents the degree of influence. The causal chain that triggers danger is identified in the causal structure graph. The causal chain that triggers danger is a path of potential risk events, including: The sudden stop following causal chain is formed by the high-speed approach phenomenon of the target before the sudden change of the state of the target in front. The high-speed approach phenomenon is that the target is in a rear position in time, the speed is higher than that of the target in front in multiple consecutive time frames, and the speed difference between the two is greater than the set approach threshold, and the spatial distance decreases continuously in three consecutive image frames. A lateral conflict-type causal chain in which a target makes an avoidance maneuver after a lateral displacement occurs between two adjacent targets in space; The blind zone intrusion causal chain is triggered by the target rapidly entering the high-density area from the edge area. The high-density area is defined as the area in which the number of targets within a unit space is greater than a set density threshold and is located in the non-visible area of ​​the subjective target or the sensor detection blind zone.

[0012] Optionally, step six specifically includes: A risk score is performed on each hazard-triggered causal chain in the causal structure diagram. The risk scoring steps include: Obtain the set of directed edges that constitute each causal chain of danger triggering, count the edge weight of each directed edge and the path length of each causal chain of danger triggering, and calculate the average edge weight. The risk score for each causal chain is calculated by combining the path length and average edge weight based on preset weighting coefficients. Based on the comparison between the risk score and a preset set of risk score thresholds, the risk is divided into different risk levels, including: Warning level: The risk score is between the low and medium thresholds, indicating that there is potential risk. Warning level: The risk score is between the medium and high thresholds, indicating that there is a foreseeable risk. Intervention level: The risk score exceeds the high threshold, indicating an urgent risk requiring immediate response. The system maps risk levels to preset response strategy logic and outputs an early warning response command corresponding to the current risk level. The early warning response command includes: When the risk level is warning, output a visual interface alert or voice reminder message; When the risk level is warning level, an auxiliary behavior guidance instruction is output, which includes movement path adjustment or target center of gravity shift guidance. When the risk level is intervention-level, a mandatory intervention command is output, which includes an emergency stop or a preset avoidance behavior.

[0013] The high-speed hazard warning system based on machine vision according to an embodiment of the present invention includes the following modules: The data acquisition and preprocessing module is used to acquire and preprocess multi-source visual video data to generate multi-source visual video data with a unified structure. The superpixel region aggregation module is used to perform multi-scale superpixel region aggregation processing on image frames of multi-source visual video data to generate a set of structured visual image blocks. The spatiotemporal feature extraction module is used to input a set of structured visual image blocks into a spatiotemporal graph convolutional encoder, extract spatial context features and temporal series features from consecutive image frames, identify dynamic target categories, and output a set of target feature vectors. The spatiotemporal interaction graph construction module is used to construct a multi-target spatiotemporal interaction graph based on the target feature vector set. Different target feature vectors are used as graph nodes. The module calculates the spatial distance, velocity difference and directional angle difference between nodes. It analyzes the geodesic path between nodes in the Riemannian manifold space and calculates the straight-line distance deviation. It generates the edge weights between nodes according to the preset weight combination. The causal structure learning module is used to perform improved NOTEARS causal analysis on multi-objective spatiotemporal interaction graphs. It establishes causal connection hypotheses for each graph node pair, generates causal connection candidates based on temporal sequence, spatial proximity trend and consistency of state changes, and generates a causal structure graph through loop detection and edge weight priority elimination. The identified dangers trigger causal chains. The risk assessment and early warning response module is used to score the risk of each hazard-triggered causal chain in the causal structure graph, calculate the risk score value by weighting the path length and average edge weight, and classify the risk into three levels: prompt level, warning level and intervention level according to the risk score threshold set, and output the corresponding early warning response instructions.

[0014] The beneficial effects of this invention are: This invention constructs an interactive graph that integrates the spatiotemporal relationships of multiple targets and introduces an improved NOTEARS causal structure recognition method, enabling in-depth modeling and analysis of potential dangerous relationships between multiple targets in high-speed motion scenes. Compared to existing technologies that rely solely on single-target trajectory prediction or static risk assessment, this method accurately identifies the state evolution patterns of multiple targets across continuous time frames and reveals the formation path of risks through causal chains. This allows for dynamic perception and prediction of typical risk patterns such as sudden stop chasing, lateral collisions, and blind spot intrusion. Furthermore, by incorporating a weighted risk scoring mechanism, this method can quantitatively evaluate different causal chains and output hierarchical warnings and intervention commands based on risk levels. This allows the system to flexibly match appropriate response strategies when facing sudden or progressive risks, significantly improving the accuracy, timeliness, and safety assurance capabilities of hazard warnings. It overcomes the technical bottleneck of traditional vision systems' insufficient causal risk recognition in complex dynamic scenes. Attached Figure Description

[0015] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of the high-speed hazard warning method based on machine vision proposed in this invention; Figure 2 This is a schematic diagram of the high-speed hazard warning system based on machine vision proposed in this invention. Detailed Implementation

[0016] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0017] refer to Figure 1 High-speed hazard warning methods based on machine vision include: Step 1: Collect and preprocess multi-source visual video data to generate multi-source visual video data with a unified structure; Step 2: Multi-scale superpixel region aggregation is used to divide the image frames of multi-source visual video data into regions to obtain a structured visual image block set; Step 3: Input the structured visual image patch set into the spatiotemporal graph convolutional encoder to extract the spatial context features and temporal series features within consecutive image frames, and generate a set of target feature vectors; Step 4: Construct a multi-target spatiotemporal interaction graph based on the target feature vector set; Step 5: Use the improved NOTEARS algorithm to perform causal analysis on the multi-objective spatiotemporal interaction graph to generate a causal structure graph containing causal chains that trigger dangers. Step Six: Assess the risk of each hazard-triggered causal chain, determine the risk level classification, and output the corresponding early warning response instructions.

[0018] This step integrates multi-source visual data and performs multi-scale image analysis to construct an interactive graph structure with spatiotemporal semantics. It utilizes the improved NOTEARS algorithm to achieve efficient causal reasoning, identify potential dangerous causal chains, and thus achieve quantitative scoring and graded response to risk events. This effectively improves the accuracy and timeliness of the early warning system's perception of sudden risks and is suitable for real-time intelligent early warning scenarios in complex and dynamic environments.

[0019] In this embodiment, the multi-source visual video data includes a sequence of images from the front-view camera in the direction of travel, a sequence of images of the blind spots on both sides captured by the side camera, and an overall dynamic environment image of the surrounding area captured by the UAV from an aerial view. The preprocessing includes grayscale normalization, brightness normalization, motion blur repair, edge enhancement, and time synchronization processing for different types of video images.

[0020] This step introduces multi-source visual video data from forward, lateral, and overhead angles to comprehensively cover the front, blind spots, and overall environmental dynamics, achieving a higher-dimensional perception capability. Combined with multi-dimensional preprocessing steps such as grayscale normalization, brightness standardization, blur repair, and time synchronization, it effectively improves the consistency of image quality and temporal alignment accuracy, providing a more stable and reliable foundation for subsequent image region segmentation and spatiotemporal modeling, and helping to enhance the robustness and perception accuracy of the overall early warning system.

[0021] In this embodiment, step two specifically includes: For each image frame of the multi-source visual video data, perform initial superpixel segmentation. The initial superpixel segmentation steps are as follows: Calculate the two-dimensional spatial position of each pixel in the image frame in the image coordinate system; The image frame is divided into regular grid regions with a fixed step size, which serve as the initial cluster centers; Within a fixed radius near each initial cluster center, the color feature value of each pixel is extracted, and the color feature value includes the brightness values ​​of the red, green and blue channels; For each pixel in the image frame, the difference in color feature value and spatial distance between it and each initial cluster center are calculated simultaneously, and then fused into a comprehensive distance index by a preset weighting ratio. The spatial distance is the Euclidean spatial distance between the pixel and the initial cluster center. Each pixel is assigned to an initial cluster center with the minimum comprehensive distance, generating the corresponding superpixel unit; Three sets of scale parameters are set for each superpixel unit, and the average color feature value, center coordinates and boundary texture gradient of each superpixel unit are recalculated at each scale. At the current scale, all adjacent superpixel units are traversed, and a region merging operation is performed. The region merging operation is to merge superpixel units whose color feature value difference is lower than a set threshold, whose center coordinates are smaller than a specified pixel range, and whose texture direction consistency between boundaries is higher than a preset ratio into a new superpixel unit. The superpixel units that have been aggregated at three different scales are fused to obtain a set of structured visual image blocks.

[0022] This step introduces a multi-scale superpixel region aggregation mechanism to construct structured visual image patches with spatial consistency, color similarity, and boundary texture constraints on image frames, significantly enhancing the accuracy of region segmentation and the structural representation of image content. The fusion of multi-scale merging strategies effectively eliminates visual noise and redundant details, preserving key visual features while improving the robustness of image segmentation and its adaptability to complex scenes. This provides a clear and compact image region foundation for subsequent target feature extraction and causal modeling.

[0023] In this embodiment, step three specifically includes: The set of structured visual image patches is constructed into a frame-level graph sequence in chronological order; The frame-level graph sequence is input into the graph convolutional encoding module of the spatiotemporal graph convolutional encoder. Within each image frame, the spatial context features of each image block within the current image frame are extracted through graph convolution operations. The spatial context features include the texture, color, edge information of the image block and the semantic association relationship with neighboring image blocks. Between adjacent image frames, temporal convolutional units are used to model the state changes of the same image patch in consecutive image frames, extract the motion trend and change magnitude of the image patch in the time dimension, and obtain time series features; Spatial context features and temporal series features are fused to form a spatiotemporal joint feature representation of each image patch in a continuous image frame sequence. Multi-image frame information is then integrated through a feature aggregation mechanism to generate spatiotemporal context-enhanced image patch feature vectors. The image patch feature vector is input into the target recognition submodule, and the dynamic target category corresponding to each image patch is identified based on a multi-class classifier. The dynamic target categories include vehicles, pedestrians, and obstacles. Extract the corresponding target feature vectors from the identified target category image blocks and generate a target feature vector set. The target feature vectors include the center position coordinates of the target category image block in the image coordinate system, the motion speed in the time dimension, the motion direction, and the appearance feature representation.

[0024] This step utilizes a spatiotemporal graph convolutional encoder to jointly model image patches in both spatial and temporal dimensions, achieving accurate target identification and state representation in complex dynamic scenes. The graph convolution module effectively captures the spatial contextual semantic relationships within image frames, while the temporal convolution unit characterizes the dynamic changes of the target. The resulting spatiotemporal joint feature vector enhances the understanding of moving target behavior. A multi-class classifier is then used to identify dynamic target categories and generate a set of target feature vectors containing position, velocity, orientation, and appearance, providing high-quality input data for subsequent causal mapping and risk prediction.

[0025] In this embodiment, step four specifically includes: Based on the target feature vector set, a multi-target spatiotemporal interaction graph is constructed. The construction steps are as follows: Different target feature vectors are used as graph nodes in the multi-target spatiotemporal interaction graph; Pair up all graph nodes and calculate the spatial distance, velocity difference, and motion direction angle difference between each pair of graph nodes to obtain the feature difference vector of the graph node pair. Map the feature difference vectors to the Riemannian manifold space; Analyze the geodesic path from one graph node to another in the Riemannian manifold space, and compare the deviation of the geodesic path from the straight-line distance between the two nodes; The measured spatial distance, velocity difference, motion direction angle difference, and straight-line distance deviation are weighted and combined according to preset weights to obtain the edge weights between graph node pairs.

[0026] This step constructs a multi-target spatiotemporal interaction graph based on target feature vectors and maps the feature difference vectors between nodes to a Riemannian manifold space, effectively characterizing the nonlinear spatiotemporal interaction relationships between moving targets. By analyzing the deviation between geodesic paths and straight-line distances, the ability to quantify the coupling degree of target behavior in complex dynamic environments is further enhanced. Edge weights are calculated using multi-dimensional indicators such as spatial position, velocity difference, and directional angle difference, providing a more accurate node association measurement basis for subsequent causal structure identification, thereby improving the response accuracy of the early warning system.

[0027] In this embodiment, the improved NOTEARS algorithm is specifically as follows: For each pair of graph nodes in a multi-objective spatiotemporal interaction graph, a causal connection hypothesis is established sequentially. The steps for establishing the connection hypothesis are as follows: For each pair of graph nodes, obtain the timestamp of the corresponding image frame, determine the temporal sequence, and ascertain whether the temporal causal sequence is satisfied. Extract the center coordinates of graph nodes in consecutive image frames at fixed time intervals and calculate the spatial distance; If a node in one graph is earlier than a node in another graph in time, and the spatial distance between the two decreases continuously over multiple time frames, then it is determined that there is a spatial proximity causal trend. Analyze the velocity and direction of motion of the graph nodes to determine the direction of state change; For graph node pairs that satisfy the temporal causal sequence and simultaneously satisfy the spatial proximity causal trend and the consistency of the direction of state change in three or more consecutive time frames, establish a causal connection hypothesis and record the starting node and the target node. Based on the connectivity hypothesis, the state change order of each graph node pair in different time frames is analyzed. Graph node pairs that show a fixed sequential state change relationship in multiple time frames and have the same direction of state change in at least three consecutive time windows are selected as causal connectivity candidates. Loop closure detection is performed on all causal connection candidates. If a group of causal connections forms a closed path, the connection edges are removed according to the time order and edge weight, generating a causal structure graph. The removal steps are as follows: Preserve the connecting edges that originate from the node that occurred earlier in time; If the time is equivalent, then retain the connection edge with the higher edge weight; If the edge weights are equivalent, then randomly remove one connecting edge until there is no closed path left; The directed edges of the causal structure graph represent the triggering or guiding effect of one target state on another target state, the direction of the directed edges represents the causal sequence, and the weight of the directed edges represents the degree of influence. The causal chain that triggers danger is identified in the causal structure graph. The causal chain that triggers danger is a path of potential risk events, including: The sudden stop following causal chain is formed by the high-speed approach phenomenon of the target before the sudden change of the state of the target in front. The high-speed approach phenomenon is that the target is in a rear position in time, the speed is higher than that of the target in front in multiple consecutive time frames, and the speed difference between the two is greater than the set approach threshold, and the spatial distance decreases continuously in three consecutive image frames. A lateral conflict-type causal chain in which a target makes an avoidance maneuver after a lateral displacement occurs between two adjacent targets in space; The blind zone intrusion causal chain is triggered by the target rapidly entering the high-density area from the edge area. The high-density area is defined as the area in which the number of targets within a unit space is greater than a set density threshold and is located in the non-visible area of ​​the subjective target or the sensor detection blind zone.

[0028] This step improves the NOTEARS algorithm by introducing temporal sequence judgment, dynamic changes in spatial distance, and state consistency analysis to establish causal connection hypotheses under multiple conditions, significantly enhancing the accuracy and robustness of causal relationship identification. Through multi-frame continuity verification and loop elimination strategies, a loop-free causal structure graph is constructed to ensure clear causal chain logic. Furthermore, by combining edge weights and temporal sequence, three typical danger-triggered causal chains—sudden stop following, lateral conflict, and blind zone intrusion—are identified, providing a highly reliable causal graph foundation for subsequent risk scoring and response strategies, thus improving the practicality and intelligence of the early warning system in high-speed dynamic scenarios.

[0029] In this embodiment, step six specifically includes: A risk score is performed on each hazard-triggered causal chain in the causal structure diagram. The risk scoring steps include: Obtain the set of directed edges that constitute each causal chain of danger triggering, count the edge weight of each directed edge and the path length of each causal chain of danger triggering, and calculate the average edge weight. The risk score for each causal chain is calculated by combining the path length and average edge weight based on preset weighting coefficients. Based on the comparison between the risk score and a preset set of risk score thresholds, the risk is divided into different risk levels, including: Warning level: The risk score is between the low and medium thresholds, indicating that there is potential risk. Warning level: The risk score is between the medium and high thresholds, indicating that there is a foreseeable risk. Intervention level: The risk score exceeds the high threshold, indicating an urgent risk requiring immediate response. The system maps risk levels to preset response strategy logic and outputs an early warning response command corresponding to the current risk level. The early warning response command includes: When the risk level is warning, output a visual interface alert or voice reminder message; When the risk level is warning level, an auxiliary behavior guidance instruction is output, which includes movement path adjustment or target center of gravity shift guidance. When the risk level is intervention-level, a mandatory intervention command is output, which includes an emergency stop or a preset avoidance behavior.

[0030] This step quantifies and evaluates the path length and edge weights of the causal chain triggering danger, constructing a risk scoring mechanism based on weighted coefficient fusion to achieve accurate classification and judgment of potential risks. Based on the scoring results, three risk levels are categorized: alert, warning, and intervention, and corresponding response strategies are matched, ranging from mild reminders to mandatory intervention, demonstrating good hierarchical early warning and intelligent response capabilities. This mechanism improves the real-time performance and accuracy of risk handling in complex dynamic scenarios, effectively enhancing the intelligence level and safety intervention efficiency of the early warning system.

[0031] refer to Figure 2 A high-speed hazard warning system based on machine vision includes the following modules: The data acquisition and preprocessing module is used to acquire and preprocess multi-source visual video data to generate multi-source visual video data with a unified structure. The superpixel region aggregation module is used to perform multi-scale superpixel region aggregation processing on image frames of multi-source visual video data to generate a set of structured visual image blocks. The spatiotemporal feature extraction module is used to input a set of structured visual image blocks into a spatiotemporal graph convolutional encoder, extract spatial context features and temporal series features from consecutive image frames, identify dynamic target categories, and output a set of target feature vectors. The spatiotemporal interaction graph construction module is used to construct a multi-target spatiotemporal interaction graph based on the target feature vector set. Different target feature vectors are used as graph nodes. The module calculates the spatial distance, velocity difference and directional angle difference between nodes. It analyzes the geodesic path between nodes in the Riemannian manifold space and calculates the straight-line distance deviation. It generates the edge weights between nodes according to the preset weight combination. The causal structure learning module is used to perform improved NOTEARS causal analysis on multi-objective spatiotemporal interaction graphs. It establishes causal connection hypotheses for each graph node pair, generates causal connection candidates based on temporal sequence, spatial proximity trend and consistency of state changes, and generates a causal structure graph through loop detection and edge weight priority elimination. The identified dangers trigger causal chains. The risk assessment and early warning response module is used to score the risk of each hazard-triggered causal chain in the causal structure graph, calculate the risk score value by weighting the path length and average edge weight, and classify the risk into three levels: prompt level, warning level and intervention level according to the risk score threshold set, and output the corresponding early warning response instructions.

[0032] This step involves constructing a modular high-speed hazard early warning system, achieving a closed-loop process from multi-source visual data acquisition to structured image processing, spatiotemporal feature extraction, causal relationship mining, and hierarchical response control. The efficient data flow and clear logic between functional modules ensure the system's ability to model potential risks from multiple angles and respond in real-time in high-speed scenarios. By integrating and improving the NOTEARS causal analysis and Riemannian manifold mapping mechanism, the system's accuracy in identifying complex spatiotemporal interactions is enhanced, effectively improving the accuracy of hazard prediction and the intelligence level of dynamic intervention.

[0033] Example 1: To verify the feasibility of this invention in practice, it was applied to a test section of a highway interchange approximately 15 kilometers long, characterized by complex lanes, small turning radii, and overlapping blind spots. This test section was equipped with a multi-source visual acquisition system, including a forward-looking camera, dual-side blind-spot cameras, and a top-down camera mounted on an unmanned aerial vehicle (UAV) platform. All camera modules acquired image data at a frequency of 30 frames per second, with a video resolution of 1920×1080 pixels.

[0034] The system performs grayscale normalization, brightness standardization, edge enhancement, and blur restoration on each frame of image while traveling at high speed, achieving initial sharpening and structural unification. Subsequently, a multi-scale superpixel region aggregation algorithm is used to perform spatial region segmentation on each frame. Combined with region merging strategies at three scales, an average of approximately 85 stable visual image patches are generated per frame. These image patches are then input sequentially into an improved spatiotemporal graph convolutional encoder to extract spatial contextual features such as vehicle spacing, texture features, and occlusion status, as well as temporal series features such as speed change trends and directional angle shifts. The model automatically identifies dynamic obstacles on the travel path, forming target feature vectors for multiple categories of targets, including vehicles, pedestrians, and roadblocks.

[0035] Based on the aforementioned feature vectors, a multi-target spatiotemporal interaction graph is constructed, and the system analyzes the motion relationships between different targets in a high-speed interactive environment. By calculating the spatial distance, velocity difference, and motion direction angle difference between nodes and mapping them to the Riemannian manifold space, a more realistic dynamic proximity relationship is measured, thereby assigning reasonable interaction weights to the graph edges. Subsequently, the system uses an improved NOTEARS causal inference algorithm to identify multiple causal chains that could potentially trigger accidents in the graph. The system detects a spatiotemporal causal structure in a group of targets that includes continuous rear-end collision acceleration increases, abrupt changes in the state of the vehicle in front, and rapidly shifting targets on both sides, automatically classifying it as a composite causal chain of "emergency stop following type" and "lateral conflict type".

[0036] To quantify the level of risk, the system calculates a weighted risk score for each causal chain, combining the average edge weights and path length. Scores exceeding a set threshold trigger an early warning response: a prompt level provides a visual alert, a warning level initiates a path adjustment command, and an intervention level directly triggers an emergency braking command. During the experimental phase, a total of 672 high-dynamic driving events were collected, including 63 manually labeled accident risk cases. The system detected 61 high-risk causal chains. The following table compares the performance of the method of this invention with three existing mainstream early warning systems during the experiment.

[0037] Table 1: Performance Comparison of Multiple Methods in High-Speed ​​Dynamic Early Warning As shown in Table 1, the method of this invention significantly outperforms existing methods in terms of target recognition accuracy, early warning response delay, multi-target hazard chain capture capability, blind zone target recognition capability, system false alarm rate, and emergency intervention triggering accuracy. It is particularly innovative in constructing interaction models between multiple targets, deriving potential causal chains, and quantifying risks. In a typical high-speed blind zone intrusion scenario, traditional YOLO-based detection systems missed a group of targets entering from the blind zone due to unclear edge detection. However, the method of this invention identifies the intruding target by combining top-view and side-view blocks, and derives a blind zone intrusion causal chain based on speed difference and directional abrupt changes. It issues an early warning before the target and the intruded object intersect, successfully avoiding potential risks. This embodiment verifies the accuracy and response efficiency of the system in handling hazardous event perception in high-speed, complex visual environments, and has broad engineering application prospects.

[0038] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A high-speed hazard warning method based on machine vision, characterized in that, include: Step 1: Collect and preprocess multi-source visual video data to generate multi-source visual video data with a unified structure; Step 2: Multi-scale superpixel region aggregation is used to divide the image frames of multi-source visual video data into regions to obtain a structured visual image block set; Step 3: Input the structured visual image patch set into the spatiotemporal graph convolutional encoder to extract the spatial context features and temporal series features within consecutive image frames, and generate a set of target feature vectors; Step 4: Construct a multi-target spatiotemporal interaction graph based on the target feature vector set; Step 5: Use the improved NOTEARS algorithm to perform causal analysis on the multi-objective spatiotemporal interaction graph to generate a causal structure graph containing causal chains that trigger dangers. Step Six: Assess the risk of each hazard-triggered causal chain, determine the risk level classification, and output the corresponding early warning response instructions.

2. The high-speed hazard warning method based on machine vision according to claim 1, characterized in that, The multi-source visual video data includes a sequence of images from the front-view camera in the direction of travel, a sequence of images of the blind spots on both sides captured by the side camera, and images of the surrounding dynamic environment captured by the UAV from an aerial view. The preprocessing includes grayscale normalization, brightness normalization, motion blur repair, edge enhancement, and time synchronization processing for different types of video images.

3. The high-speed hazard warning method based on machine vision according to claim 1, characterized in that, Step two specifically involves: For each image frame of the multi-source visual video data, perform initial superpixel segmentation. The initial superpixel segmentation steps are as follows: Calculate the two-dimensional spatial position of each pixel in the image frame in the image coordinate system; The image frame is divided into regular grid regions with a fixed step size, which serve as the initial cluster centers; Within a fixed radius near each initial cluster center, the color feature value of each pixel is extracted, and the color feature value includes the brightness values ​​of the red, green and blue channels; For each pixel in the image frame, the difference in color feature value and spatial distance between it and each initial cluster center are calculated simultaneously, and then fused into a comprehensive distance index by a preset weighting ratio. The spatial distance is the Euclidean spatial distance between the pixel and the initial cluster center. Each pixel is assigned to an initial cluster center with the minimum comprehensive distance, generating the corresponding superpixel unit; Three sets of scale parameters are set for each superpixel unit, and the average color feature value, center coordinates and boundary texture gradient of each superpixel unit are recalculated at each scale. At the current scale, all adjacent superpixel units are traversed, and a region merging operation is performed. The region merging operation is to merge superpixel units whose color feature value difference is lower than a set threshold, whose center coordinates are smaller than a specified pixel range, and whose texture direction consistency between boundaries is higher than a preset ratio into a new superpixel unit. The superpixel units that have been aggregated at three different scales are fused to obtain a set of structured visual image blocks.

4. The high-speed hazard warning method based on machine vision according to claim 1, characterized in that, Step three specifically involves: The set of structured visual image patches is constructed into a frame-level graph sequence in chronological order; The frame-level graph sequence is input into the graph convolutional encoding module of the spatiotemporal graph convolutional encoder. Within each image frame, the spatial context features of each image block within the current image frame are extracted through graph convolution operations. The spatial context features include the texture, color, edge information of the image block and the semantic association relationship with neighboring image blocks. Between adjacent image frames, temporal convolutional units are used to model the state changes of the same image patch in consecutive image frames, extract the motion trend and change magnitude of the image patch in the time dimension, and obtain time series features; Spatial context features and temporal series features are fused to form a spatiotemporal joint feature representation of each image patch in a continuous image frame sequence. Multi-image frame information is then integrated through a feature aggregation mechanism to generate spatiotemporal context-enhanced image patch feature vectors. The image patch feature vector is input into the target recognition submodule, and the dynamic target category corresponding to each image patch is identified based on a multi-class classifier. The dynamic target categories include vehicles, pedestrians, and obstacles. Extract the corresponding target feature vectors from the identified target category image blocks and generate a target feature vector set. The target feature vectors include the center position coordinates of the target category image block in the image coordinate system, the motion speed in the time dimension, the motion direction, and the appearance feature representation.

5. The high-speed hazard warning method based on machine vision according to claim 1, characterized in that, Step four specifically involves: Based on the target feature vector set, a multi-target spatiotemporal interaction graph is constructed. The construction steps are as follows: Different target feature vectors are used as graph nodes in the multi-target spatiotemporal interaction graph; Pair up all graph nodes and calculate the spatial distance, velocity difference, and motion direction angle difference between each pair of graph nodes to obtain the feature difference vector of the graph node pair. Map the feature difference vectors to the Riemannian manifold space; Analyze the geodesic path from one graph node to another in the Riemannian manifold space, and compare the deviation of the geodesic path from the straight-line distance between the two nodes; The measured spatial distance, velocity difference, motion direction angle difference, and straight-line distance deviation are weighted and combined according to preset weights to obtain the edge weights between graph node pairs.

6. The high-speed hazard warning method based on machine vision according to claim 1, characterized in that, The improved NOTEARS algorithm is specifically as follows: For each pair of graph nodes in a multi-objective spatiotemporal interaction graph, a causal connection hypothesis is established sequentially. The steps for establishing the connection hypothesis are as follows: For each pair of graph nodes, obtain the timestamp of the corresponding image frame, determine the temporal sequence, and ascertain whether the temporal causal sequence is satisfied. Extract the center coordinates of graph nodes in consecutive image frames at fixed time intervals and calculate the spatial distance; If a node in one graph is earlier than a node in another graph in time, and the spatial distance between the two decreases continuously over multiple time frames, then it is determined that there is a spatial proximity causal trend. Analyze the velocity and direction of motion of the graph nodes to determine the direction of state change; For graph node pairs that satisfy the temporal causal sequence and simultaneously satisfy the spatial proximity causal trend and the consistency of the direction of state change in three or more consecutive time frames, establish a causal connection hypothesis and record the starting node and the target node. Based on the connectivity hypothesis, the state change order of each graph node pair in different time frames is analyzed. Graph node pairs that show a fixed sequential state change relationship in multiple time frames and have the same direction of state change in at least three consecutive time windows are selected as causal connectivity candidates. Loop closure detection is performed on all causal connection candidates. If a group of causal connections forms a closed path, the connection edges are removed according to the time order and edge weight, generating a causal structure graph. The removal steps are as follows: Preserve the connecting edges that originate from the node that occurred earlier in time; If the time is equivalent, then retain the connection edge with the higher edge weight; If the edge weights are equivalent, then randomly remove one connecting edge until there is no closed path left; The directed edges of the causal structure graph represent the triggering or guiding effect of one target state on another target state, the direction of the directed edges represents the causal sequence, and the weight of the directed edges represents the degree of influence. The causal chain that triggers danger is identified in the causal structure graph. The causal chain that triggers danger is a path of potential risk events, including: The sudden stop following causal chain is formed by the high-speed approach phenomenon of the target before the sudden change of the state of the target in front. The high-speed approach phenomenon is that the target is in a rear position in time, the speed is higher than that of the target in front in multiple consecutive time frames, and the speed difference between the two is greater than the set approach threshold, and the spatial distance decreases continuously in three consecutive image frames. A lateral conflict-type causal chain in which a target makes an avoidance maneuver after a lateral displacement occurs between two adjacent targets in space; The blind zone intrusion causal chain is triggered by the target rapidly entering the high-density area from the edge area. The high-density area is defined as the area in which the number of targets within a unit space is greater than a set density threshold and is located in the non-visible area of ​​the subjective target or the sensor detection blind zone.

7. The high-speed hazard warning method based on machine vision according to claim 1, characterized in that, Step six specifically involves: A risk score is performed on each hazard-triggered causal chain in the causal structure diagram. The risk scoring steps include: Obtain the set of directed edges that constitute each causal chain of danger triggering, count the edge weight of each directed edge and the path length of each causal chain of danger triggering, and calculate the average edge weight. The risk score for each causal chain is calculated by combining the path length and average edge weight based on preset weighting coefficients. Based on the comparison between the risk score and a preset set of risk score thresholds, the risk is divided into different risk levels, including: Warning level: The risk score is between the low and medium thresholds, indicating that there is potential risk. Warning level: The risk score is between the medium and high thresholds, indicating that there is a foreseeable risk. Intervention level: The risk score exceeds the high threshold, indicating an urgent risk requiring immediate response. The system maps risk levels to preset response strategy logic and outputs an early warning response command corresponding to the current risk level. The early warning response command includes: When the risk level is warning, output a visual interface alert or voice reminder message; When the risk level is warning level, an auxiliary behavior guidance instruction is output, which includes movement path adjustment or target center of gravity shift guidance. When the risk level is intervention-level, a mandatory intervention command is output, which includes an emergency stop or a preset avoidance behavior.

8. A high-speed hazard warning system based on machine vision, executing the high-speed hazard warning method based on machine vision as described in any one of claims 1 to 7, characterized in that, Includes the following modules: The data acquisition and preprocessing module is used to acquire and preprocess multi-source visual video data to generate multi-source visual video data with a unified structure. The superpixel region aggregation module is used to perform multi-scale superpixel region aggregation processing on image frames of multi-source visual video data to generate a set of structured visual image blocks. The spatiotemporal feature extraction module is used to input a set of structured visual image blocks into a spatiotemporal graph convolutional encoder, extract spatial context features and temporal series features from consecutive image frames, identify dynamic target categories, and output a set of target feature vectors. The spatiotemporal interaction graph construction module is used to construct a multi-target spatiotemporal interaction graph based on the target feature vector set. Different target feature vectors are used as graph nodes. The module calculates the spatial distance, velocity difference and directional angle difference between nodes. It analyzes the geodesic path between nodes in the Riemannian manifold space and calculates the straight-line distance deviation. It generates the edge weights between nodes according to the preset weight combination. The causal structure learning module is used to perform improved NOTEARS causal analysis on multi-objective spatiotemporal interaction graphs. It establishes causal connection hypotheses for each graph node pair, generates causal connection candidates based on temporal sequence, spatial proximity trend and consistency of state changes, and generates a causal structure graph through loop detection and edge weight priority elimination. The identified dangers trigger causal chains. The risk assessment and early warning response module is used to score the risk of each hazard-triggered causal chain in the causal structure graph, calculate the risk score value by weighting the path length and average edge weight, and classify the risk into three levels: prompt level, warning level and intervention level according to the risk score threshold set, and output the corresponding early warning response instructions.

Images