High-speed danger warning method and system based on machine vision

Abstract

The application discloses a high-speed danger early warning method and system based on machine vision, comprising the following steps: step one, collecting and preprocessing multi-source visual video data; step two, adopting multi-scale superpixel region aggregation to divide the image frames of the multi-source visual video data into regions; step three, extracting spatial context features and time sequence features in continuous image frames; step four, constructing a multi-target space-time interaction graph according to a target feature vector set; step five, using an improved NOTEARS algorithm to perform causal analysis on the multi-target space-time interaction graph, and generating a causal structure graph containing a danger trigger causal chain; and step six, performing risk scoring on each danger trigger causal chain, determining a risk grade division, and outputting corresponding early warning response instructions. The application combines superpixel region aggregation and the improved NOTEARS algorithm, and realizes intelligent risk early warning in a high-speed scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent perception and danger early warning, and in particular to a high-speed danger early warning method and system based on machine vision. BACKGROUND

[0002] In a high-speed motion environment, especially in a complex urban traffic, a highway intersection area or an intelligent vehicle system, rapid identification and early warning of potential dangers have always been a key research direction in the fields of intelligent driving, traffic safety and automatic inspection. Existing danger identification methods based on machine vision mainly rely on image information from a single perspective for target detection and motion trend prediction, such as using a front-view camera or simply fusing multiple images to extract target trajectory and behavior features through convolutional neural networks (CNN), long short-term memory networks (LSTM) and the like. However, such methods often have problems of response delay, high misidentification rate and serious missed detection in the face of scenes where multiple targets frequently interact, blind areas are severely blocked and sudden events are unpredictable in high-speed driving state, making it difficult to model and deduce potential danger relationships among multiple targets.

[0003] In addition, existing technologies usually use risk assessment mechanisms based on rules or shallow statistics, which cannot deeply mine potential causal relationships between different targets in the time series evolution process, resulting in an inability to form effective cognition of the triggering path of the risk. For example, some methods only judge the risk level based on the distance or speed difference between targets, lack the modeling ability of target state evolution trend, behavior correlation and time series evolution law, and cannot identify hidden danger chains. In the scene where multiple targets exist simultaneously and high-speed intersection motion state is complex, the formation of danger often has causal conduction characteristics, such as an emergency lane change of a target triggering a chain reaction of rear vehicles, or a sudden cut-in of a target in the edge area causing a target in the center area to avoid not in time. These causal chain evolution processes are difficult to accurately capture and quantify in existing systems.

[0004] Therefore, how to provide a high-speed danger early warning method and system based on machine vision is a problem that those skilled in the art need to solve. SUMMARY

[0005] One object of the present application is to provide a high-speed danger early warning method and system based on machine vision. The present application fuses multi-source visual information and an improved NOTEARS algorithm to construct a multi-target space-time interaction graph, realizes identification and quantitative analysis of potential danger triggering chains in a high-speed motion scene, and outputs graded early warning and intervention instructions through a risk scoring and grade division mechanism to improve risk perception accuracy and response timeliness in a complex dynamic environment, and realizes intelligent danger early warning and safety decision support in a high-speed scene.

[0006] The machine vision-based high-speed danger early warning method according to the embodiment of the present application comprises the following steps:

[0007] Step one: collect and pre-process multi-source visual video data to generate multi-source visual video data with unified structure;

[0008] Step two: divide the image frames of the multi-source visual video data into regions by using multi-scale superpixel region aggregation to obtain a set of structured visual image blocks;

[0009] Step three: input the set of structured visual image blocks into a spatio-temporal graph convolutional encoder to extract spatial context features and time series features in the continuous image frames and generate a set of target feature vectors;

[0010] Step four: construct a multi-target spatio-temporal interaction graph according to the set of target feature vectors;

[0011] Step five: perform causal analysis on the multi-target spatio-temporal interaction graph by using an improved NOTEARS algorithm to generate a causal structure graph containing danger trigger causal chains;

[0012] Step six: perform risk scoring on each danger trigger causal chain and determine risk level division to output corresponding early warning response instructions.

[0013] Optionally, the multi-source visual video data comprises a sequence of front-view images collected by a front-facing camera in the direction of travel, a sequence of images of the blind area on both sides collected by side-facing cameras, and an overall dynamic environment image collected by a drone from an aerial perspective; the pre-processing comprises performing grayscale normalization, brightness standardization processing, motion blur repair, edge enhancement, and time synchronization processing on different types of video images respectively.

[0014] Optionally, the step two specifically comprises:

[0015] performing initial superpixel segmentation on each image frame of the multi-source visual video data, wherein the initial superpixel segmentation step comprises:

[0016] calculating the two-dimensional spatial position of each pixel point in the image coordinate system;

[0017] dividing the image frame into regular grid regions with a fixed step length as initial clustering centers;

[0018] extracting color feature values of each pixel point within a fixed radius range around each initial clustering center, wherein the color feature values comprise brightness values of three channels of red, green, and blue;

[0019] For each pixel point in the image frame, simultaneously calculate the color feature value difference and the spatial position distance with each initial clustering center, and fuse them into a comprehensive distance index through a preset weighting ratio, the spatial position distance being the Euclidean space distance between the pixel point and the initial clustering center;

[0020] Classify each pixel point to the initial clustering center with the minimum comprehensive distance to generate a corresponding superpixel unit;

[0021] Set three groups of scale parameters for the superpixel unit, and respectively recalculate the average color feature value, center coordinate and boundary texture gradient of each superpixel unit at each scale;

[0022] At the current scale, traverse all adjacent superpixel units to perform a region merging operation, the region merging operation being to merge superpixel units with a color feature value difference lower than a set threshold, a center coordinate smaller than a specified pixel range and a texture direction consistency between boundaries higher than a preset ratio into a new superpixel unit;

[0023] Respectively perform fusion processing on the superpixel units aggregated at the three groups of scales to obtain a structured visual image block set.

[0024] Optionally, the step three specifically includes:

[0025] Construct the structured visual image block set into a frame-level graph sequence in chronological order;

[0026] Input the frame-level graph sequence into a graph convolution encoding module of a spatio-temporal graph convolution encoder, and extract spatial context features of each image block in the current image frame through graph convolution operation within each image frame, the spatial context features including texture, color, edge information of the image block and semantic association relationship between adjacent image blocks;

[0027] Between adjacent image frames, model state changes of the same image block in consecutive image frames through a temporal convolution unit to extract motion trend and change amplitude of the image block in the time dimension, and obtain time sequence features;

[0028] Fuse the spatial context features and the time sequence features to form a spatio-temporal joint feature representation of each image block in the sequence of consecutive image frames, and integrate multi-image frame information through a feature aggregation mechanism to generate a spatio-temporal context enhanced image block feature vector;

[0029] Input the image block feature vector into a target recognition submodule, identify the dynamic target category corresponding to each image block based on a multi-class classifier, the dynamic target category including a vehicle, a pedestrian and an obstacle;

[0030] A target feature vector corresponding to the identified target category image block is extracted, and a target feature vector set is generated, the target feature vector including a center position coordinate of the target category image block in an image coordinate system, a motion speed in a time dimension, a motion direction, and an appearance feature representation.

[0031] Optionally, the step four is specifically:

[0032] According to the target feature vector set, a multi-target space-time interaction graph is constructed, and the construction step is:

[0033] Different target feature vectors are used as graph nodes of the multi-target space-time interaction graph.

[0034] All graph nodes are paired and combined, the spatial position distance, the speed difference, and the motion direction angle difference between each pair of graph nodes are calculated, and a feature difference vector of the pair of graph nodes is obtained.

[0035] The feature difference vector is mapped to a Riemannian manifold space.

[0036] A geodesic path in the Riemannian manifold space from one graph node to another graph node is analyzed, and a deviation of a straight line distance between the two nodes is compared.

[0037] The measured spatial position distance, the speed difference, the motion direction angle difference, and the straight line distance deviation are weighted and combined according to a preset weight to obtain an edge weight between the pair of graph nodes.

[0038] Optionally, the improved NOTEARS algorithm is specifically:

[0039] For each pair of graph nodes in the multi-target space-time interaction graph, a connection hypothesis of a causal relationship is sequentially established, and the establishment step of the connection hypothesis is:

[0040] For each pair of graph nodes, a corresponding image frame timestamp is obtained, a time sequence relationship is judged, and whether a time causal order is satisfied is determined.

[0041] Center position coordinates of the graph nodes in consecutive image frames are extracted at a fixed time interval, and a spatial distance is calculated.

[0042] If a graph node is located before another graph node in time, and the spatial distance between the two nodes continuously decreases in multiple time frames, it is judged that there is a spatial proximity causal trend.

[0043] The speed and the motion direction of the graph nodes are analyzed, and a state change direction is judged.

[0044] establishing a causal connection hypothesis for the pair of nodes that meet the time causal sequence and simultaneously meet the spatial proximity causal trend and the consistent state change direction in three or more continuous time frames, and recording the starting node and the target node;

[0045] Based on the connection hypothesis, the state change sequence of each pair of nodes in different time frames is analyzed, and the pair of nodes that show a fixed state change relationship in multiple time frames and have the same direction of state change in at least three continuous time windows are selected as causal connection candidates;

[0046] Loop detection is performed on all causal connection candidates, and if a closed path is formed by a group of causal connections, the connection edges are removed according to the time sequence and the size of the edge weight to generate a causal structure graph, and the removal step is:

[0047] The connection edge with the node occurring earlier in time is retained as the starting point;

[0048] If the time is equivalent, the connection edge with the higher edge weight is retained;

[0049] If the edge weight is equivalent, one connection edge is randomly removed until there is no closed path;

[0050] The directed edge of the causal structure graph represents that a target state has a triggering or guiding effect on another target state, the direction of the directed edge represents the causal sequence, and the weight of the directed edge represents the degree of influence;

[0051] A dangerous trigger causal chain is identified in the causal structure graph, and the dangerous trigger causal chain is a potential risk event path, including:

[0052] An emergency stop follow-up type causal chain formed by the target appearing in a high-speed approach phenomenon before the target state mutation in the front, the high-speed approach phenomenon being that the target in the rear position has a speed higher than the target in the front in multiple continuous time frames, and the speed difference between the two is greater than a set approach threshold, and the spatial distance is continuously reduced in three continuous image frames;

[0053] A lateral conflict type causal chain in which the target performs an avoidance action after a lateral displacement of a spatially adjacent target;

[0054] A blind area intrusion type causal chain triggered by the target quickly entering a high-density area from an edge area, the high-density area being an area in which the number of targets in a unit space is greater than a set density threshold, and being in a non-visible area of the subjective target or a sensor detection blind area.

[0055] Optionally, the step six is specifically:

[0056] Risk scoring is performed on each dangerous trigger causal chain in the causal structure graph, and the risk scoring step includes:

[0057] obtain a set of directed edges constituting each dangerous trigger causal chain, count the edge weight of each directed edge and the path length of each dangerous trigger causal chain, and calculate the average edge weight;

[0058] combine the path length and the average edge weight according to a preset weighting coefficient, and calculate the risk score value of each causal chain;

[0059] According to the risk score value and the preset risk score threshold set, different risk levels are divided, including:

[0060] The prompt level, the risk score value is between the low threshold and the middle threshold, indicating that there is potential risk;

[0061] The warning level, the risk score value is between the middle threshold and the high threshold, indicating that there is foreseeable risk;

[0062] The intervention level, the risk score value exceeds the high threshold, indicating that there is an urgent immediate response risk;

[0063] The risk level is corresponded to the preset response strategy logic, and the early warning response instruction corresponding to the current risk level is output, and the early warning response instruction includes:

[0064] When it is a prompt level risk, a visual interface warning or voice reminder information is output;

[0065] When it is a warning level risk, an auxiliary behavior guidance instruction is output, and the auxiliary behavior guidance instruction includes a motion path adjustment or a target gravity center offset guidance;

[0066] When it is an intervention level risk, a forced intervention instruction is output, and the forced intervention instruction includes an emergency stop or a preset avoidance behavior.

[0067] The high-speed danger early warning system based on machine vision according to the embodiment of the application includes the following modules:

[0068] The data acquisition and preprocessing module is used for acquiring and preprocessing multi-source visual video data, and generating multi-source visual video data with a unified structure;

[0069] The superpixel region aggregation module is used for performing multi-scale superpixel region aggregation processing on the image frames of the multi-source visual video data, and generating a set of structured visual image blocks;

[0070] The spatiotemporal feature extraction module is used for inputting the set of structured visual image blocks into a spatiotemporal graph convolutional encoder, extracting spatial context features and time sequence features in continuous image frames, identifying dynamic target categories, and outputting a set of target feature vectors;

[0071] The spatio-temporal interaction graph construction module is configured to construct a multi-target spatio-temporal interaction graph according to the target feature vector set, take different target feature vectors as graph nodes, calculate the spatial position distance, speed difference and direction angle difference between nodes, analyze the geodesic path between nodes in the Riemannian manifold space and calculate the straight line distance deviation, and generate the edge weight between nodes by combining according to a preset weight;

[0072] The causal structure learning module is configured to perform improved NOTEARS causal analysis on the multi-target spatio-temporal interaction graph, establish a causal connection hypothesis for each pair of graph nodes, generate a causal connection candidate according to the time sequence, spatial proximity trend and state change consistency, and generate a causal structure graph through closed loop detection and edge weight priority elimination, and identify a dangerous trigger causal chain;

[0073] The risk assessment and early warning response module is configured to perform risk scoring on each dangerous trigger causal chain in the causal structure graph, calculate a weighted combination of the path length and the average edge weight to obtain a risk score value, divide the risk into three levels of prompt level, warning level and intervention level according to a risk score threshold set, and output corresponding early warning response instructions.

[0074] The beneficial effects of the present application are:

[0075] The present application realizes in-depth modeling and analysis of potential dangerous relationships between multiple targets in a high-speed motion scene by constructing an interaction graph that integrates the spatio-temporal relationships of multiple targets and introducing an improved NOTEARS causal structure recognition method. Compared with the prior art which only relies on single-target trajectory prediction or static risk assessment, the present method can accurately identify the state evolution law of multiple targets in consecutive time frames and reveal the formation path of the risk in the form of a causal chain, thereby realizing dynamic perception and prediction of typical risk patterns such as sudden stop pursuit, lateral conflict and blind area intrusion. At the same time, combined with the risk scoring mechanism of weight fusion, the present method can quantitatively evaluate different causal chains and output hierarchical early warning and intervention instructions according to the risk level, so that the system can flexibly match the corresponding response strategy when facing sudden or gradual risks, greatly improving the accuracy, timeliness and safety guarantee capability of the danger warning, and breaking through the technical bottleneck of the traditional visual system in identifying causal risks in complex dynamic scenes. BRIEF DESCRIPTION OF DRAWINGS

[0076] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, illustrate the present application together with the embodiments thereof, and explain the present application, but do not constitute a limitation of the present application. In the drawings:

[0077] Figure 1 The overall flowchart of the high-speed danger early warning method based on machine vision proposed by the present application;

[0078] Figure 2A structural schematic diagram of a high-speed danger early warning system based on machine vision is provided in the present application. DETAILED DESCRIPTION

[0079] The present application will now be further described in detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams and only illustrate the basic structure of the present application in a schematic manner, and thus only show the components related to the present application.

[0080] REFERENCE Figure 1 A high-speed danger early warning method based on machine vision, comprising:

[0081] Step one: collect and pre-process multi-source visual video data to generate multi-source visual video data with uniform structure;

[0082] Step two: divide the image frames of the multi-source visual video data into regions using multi-scale superpixel region aggregation to obtain a set of structured visual image blocks;

[0083] Step three: input the set of structured visual image blocks into a spatio-temporal graph convolutional encoder to extract spatial context features and temporal sequence features within consecutive image frames and generate a set of target feature vectors;

[0084] Step four: construct a multi-target spatio-temporal interaction graph based on the set of target feature vectors;

[0085] Step five: use an improved NOTEARS algorithm to perform causal analysis on the multi-target spatio-temporal interaction graph to generate a causal structure graph containing danger trigger causal chains;

[0086] Step six: score each danger trigger causal chain and determine the risk level division to output corresponding early warning response instructions.

[0087] This step can fuse multi-source visual data and perform multi-scale image analysis to construct an interaction graph structure with spatio-temporal semantics, and use an improved NOTEARS algorithm to realize efficient causal reasoning, identify potential danger causal chains, and then realize quantitative scoring and level response of risk events, effectively improving the perception accuracy and response timeliness of the early warning system for sudden risks, and is suitable for real-time intelligent early warning scenarios in complex dynamic environments.

[0088] In this embodiment, the multi-source visual video data includes a main perspective image sequence collected by a front-facing camera in the direction of travel, an image sequence of the blind area range on both sides collected by a lateral camera, and an overall dynamic environment image collected by a drone from an aerial perspective; the pre-processing includes performing grayscale normalization, brightness standardization processing, motion blur repair, edge enhancement, and time synchronization processing on different types of video images respectively.

[0089] The step comprehensively covers the front, blind area and overall environment dynamics by introducing multi-source visual video data from front, side and top angles to realize higher dimensional perception ability; combining multi-dimensional preprocessing steps such as gray normalization, brightness standardization, blur repair and time synchronization, the consistency of image quality and the time sequence alignment accuracy are effectively improved, which provides a more stable and reliable basis for subsequent image region division and spatio-temporal modeling, and helps to enhance the robustness and perception accuracy of the overall early warning system.

[0090] In the embodiment, the step two is specifically:

[0091] An initial superpixel segmentation is performed on each image frame of the multi-source visual video data, and the initial superpixel segmentation step is:

[0092] The two-dimensional space position of each pixel point in the image frame in the image coordinate system is calculated;

[0093] The image frame is divided into regular grid regions with a fixed step length as initial clustering centers;

[0094] In a fixed radius range near each initial clustering center, the color feature value of each pixel point is extracted, and the color feature value includes the brightness value of the red, green and blue channels;

[0095] For each pixel point in the image frame, the color feature value difference and the spatial position distance between the pixel point and each initial clustering center are calculated simultaneously, and the two are fused into a comprehensive distance index through a preset weighting ratio, and the spatial position distance is the Euclidean space distance between the pixel point and the initial clustering center;

[0096] Each pixel point is classified into the initial clustering center with the smallest comprehensive distance to generate a corresponding superpixel unit;

[0097] Three groups of scale parameters are set for the superpixel unit, and the average color feature value, center coordinate and boundary texture gradient of each superpixel unit are recalculated at each scale;

[0098] At the current scale, all adjacent superpixel units are traversed to perform a region merging operation, and the region merging operation is to merge superpixel units with a color feature value difference below a set threshold, a center coordinate less than a specified pixel range and a texture direction consistency between boundaries higher than a preset ratio into new superpixel units;

[0099] The superpixel units aggregated at the three groups of scales are respectively subjected to fusion processing to obtain a structured visual image block set.

[0100] The step constructs a structured visual image block with spatial consistency, color similarity and boundary texture constraints on the image frame by introducing a multi-scale superpixel region aggregation mechanism, significantly enhancing the accuracy of region division and the structure of image content expression. The fusion of the merging strategy at multiple scales can effectively eliminate visual noise and redundant details, while preserving key visual features, improving the robustness of image segmentation and the adaptability to complex scenes, providing a clear and compact image region basis for subsequent target feature extraction and causal modeling.

[0101] In the embodiment, the step three is specifically:

[0102] The structured visual image block set is constructed into a frame-level graph sequence in chronological order;

[0103] The frame-level graph sequence is input into the graph convolution coding module of the spatio-temporal graph convolution encoder. In each image frame, the spatial context features of each image block in the current image frame are extracted by graph convolution operation, including texture, color, edge information and semantic association relationship between adjacent image blocks.

[0104] Between adjacent image frames, the state changes of the same image block in consecutive image frames are modeled by using a time convolution unit, and the motion trend and change amplitude of the image block in the time dimension are extracted to obtain the time sequence features.

[0105] The spatial context features and the time sequence features are fused to form the spatio-temporal joint feature representation of each image block in the sequence of consecutive image frames, and the multi-image frame information is integrated through a feature aggregation mechanism to generate the image block feature vector enhanced by spatio-temporal context.

[0106] The image block feature vector is input into the target recognition sub-module, and based on a multi-class classifier, the dynamic target category corresponding to each image block is identified, including vehicles, pedestrians and obstacles.

[0107] The target feature vector corresponding to the identified target category image block is extracted, and a target feature vector set is generated, including the center position coordinates of the target category image block in the image coordinate system, the motion speed, the motion direction and the appearance feature representation in the time dimension.

[0108] This step utilizes a spatiotemporal graph convolutional encoder to jointly model image patches in both spatial and temporal dimensions, achieving accurate target identification and state representation in complex dynamic scenes. The graph convolution module effectively captures the spatial contextual semantic relationships within image frames, while the temporal convolution unit characterizes the dynamic changes of the target. The resulting spatiotemporal joint feature vector enhances the understanding of moving target behavior. A multi-class classifier is then used to identify dynamic target categories and generate a set of target feature vectors containing position, velocity, orientation, and appearance, providing high-quality input data for subsequent causal mapping and risk prediction.

[0109] In this embodiment, step four specifically includes:

[0110] Based on the target feature vector set, a multi-target spatiotemporal interaction graph is constructed. The construction steps are as follows:

[0111] Different target feature vectors are used as graph nodes in a multi-target spatiotemporal interaction graph;

[0112] Pair up all graph nodes and calculate the spatial distance, velocity difference, and motion direction angle difference between each pair of graph nodes to obtain the feature difference vector of the graph node pair.

[0113] Map the feature difference vectors to the Riemannian manifold space;

[0114] Analyze the geodesic path from one graph node to another in the Riemannian manifold space, and compare the deviation of the geodesic path from the straight-line distance between the two nodes;

[0115] The measured spatial distance, velocity difference, motion direction angle difference, and straight-line distance deviation are weighted and combined according to preset weights to obtain the edge weights between graph node pairs.

[0116] This step constructs a multi-target spatiotemporal interaction graph based on target feature vectors and maps the feature difference vectors between nodes to a Riemannian manifold space, effectively characterizing the nonlinear spatiotemporal interaction relationships between moving targets. By analyzing the deviation between geodesic paths and straight-line distances, the ability to quantify the coupling degree of target behavior in complex dynamic environments is further enhanced. Edge weights are calculated using multi-dimensional indicators such as spatial position, velocity difference, and directional angle difference, providing a more accurate node association measurement basis for subsequent causal structure identification, thereby improving the response accuracy of the early warning system.

[0117] In this embodiment, the improved NOTEARS algorithm is specifically as follows:

[0118] For each pair of graph nodes in a multi-objective spatiotemporal interaction graph, a causal connection hypothesis is established sequentially. The steps for establishing the connection hypothesis are as follows:

[0119] For each pair of graph nodes, the corresponding image frame timestamps are obtained respectively, the time sequence is judged, and whether the time causal sequence is met is determined;

[0120] The center position coordinates of the graph nodes in the continuous image frames are extracted at fixed time intervals, and the spatial distance is calculated;

[0121] If a graph node is located before another graph node in time, and the spatial distance between them continuously decreases in multiple time frames, it is judged that there is a spatial proximity causal trend;

[0122] The speed and motion direction of the graph nodes are analyzed, and the state change direction is judged;

[0123] For the pair of graph nodes that meet the time causal sequence and simultaneously meet the spatial proximity causal trend and the consistency of the state change direction in three or more continuous time frames, a causal connection hypothesis is established, and the starting node and the target node are recorded;

[0124] Based on the connection hypothesis, the state change sequence of each pair of graph nodes in different time frames is analyzed, and the pair of graph nodes that presents a fixed sequence of state changes in multiple time frames and has the same direction of state change in at least three continuous time windows is selected as a causal connection candidate;

[0125] All causal connection candidates are closed loop detected, if a group of causal connections forms a closed path, the connection edges are removed according to the time sequence and the size of the edge weight, and a causal structure graph is generated, the removing step is:

[0126] The connection edge with the node occurring earlier in time is retained as the starting point;

[0127] If the time is equivalent, the connection edge with the higher edge weight is retained;

[0128] If the edge weight is equivalent, one connection edge is randomly removed until there is no closed path;

[0129] The directed edge of the causal structure graph represents that a target state has a triggering or guiding effect on another target state, the direction of the directed edge represents the causal sequence, and the weight of the directed edge represents the influence degree;

[0130] A dangerous trigger causal chain is identified in the causal structure graph, the dangerous trigger causal chain is a potential risk event path, including:

[0131] An emergency stop following type causal chain formed by the target appearing in high speed approaching phenomenon before the target state mutation in front, the high speed approaching phenomenon is that the target in the rear position in time has a speed higher than the target in front in multiple continuous time frames, and the speed difference between them is greater than a set approaching threshold, and the spatial distance continuously decreases in three continuous image frames;

[0132] Target appears to avoid action in lateral collision type causal chain after lateral displacement of spatially adjacent target;

[0133] Blind area intrusion type causal chain triggered by target quickly entering high-density area from edge area, the high-density area being the number of targets within a unit space range being greater than a set density threshold, and being in a non-visual area of a subjective target or a sensor detection blind area.

[0134] This step improves the NOTEARS algorithm by introducing temporal sequence judgment, dynamic spatial distance change and state consistency analysis, establishes causal connection hypotheses under multiple conditions, and significantly improves the accuracy and robustness of causal relationship identification. Through multi-frame continuity verification and closed loop elimination strategy, an acyclic causal structure graph is constructed to ensure clear causal chain logic. Further combining edge weight and temporal order to identify three typical dangerous trigger causal chains: sudden stop following, lateral conflict and blind area intrusion, providing a highly reliable causal graph basis for subsequent risk scoring and response strategy, and improving the practicality and intelligence of the early warning system in high-speed dynamic scenarios.

[0135] In this embodiment, step six is specifically:

[0136] Risk scoring is performed on each dangerous trigger causal chain in the causal structure graph, and the risk scoring step includes:

[0137] Obtain the set of directed edges constituting each dangerous trigger causal chain, count the edge weight of each directed edge and the path length of each dangerous trigger causal chain, and calculate the average edge weight;

[0138] According to the preset weighting coefficient, the path length and the average edge weight are combined to calculate the risk score value of each causal chain;

[0139] According to the risk score value and the preset risk score threshold set, different risk levels are divided, and the risk levels include:

[0140] Prompt level, risk score value between low threshold and medium threshold, indicating potential risk;

[0141] Warning level, risk score value between medium threshold and high threshold, indicating foreseeable risk;

[0142] Intervention level, risk score value exceeding high threshold, indicating imminent immediate response risk;

[0143] The risk level is corresponded to the preset response strategy logic, and the early warning response instruction corresponding to the current risk level is output, and the early warning response instruction includes:

[0144] When it is a prompt-level risk, a visual interface warning or a voice reminder information is outputted;

[0145] When it is a warning-level risk, an auxiliary behavior guidance instruction is outputted, the auxiliary behavior guidance instruction including a motion path adjustment or a target gravity center shift guidance;

[0146] When it is an intervention-level risk, a forced intervention instruction is outputted, the forced intervention instruction including an emergency stop or a preset avoidance behavior.

[0147] This step realizes accurate grading determination of potential risks by quantitatively evaluating the path length and edge weight of the risk trigger causal chain, and constructing a risk scoring mechanism based on weighted coefficient fusion. According to the scoring results, three risk levels of prompt level, warning level and intervention level are divided, and corresponding response strategies are matched, from mild reminder to forced intervention, with good hierarchical early warning and intelligent response ability. This mechanism improves the real-time and accuracy of risk processing in complex dynamic scenarios, effectively enhances the intelligent level and safety intervention efficiency of the early warning system.

[0148] Reference Figure 2 , the high-speed danger early warning system based on machine vision includes the following modules:

[0149] The data acquisition and preprocessing module is used for acquiring and preprocessing multi-source visual video data to generate multi-source visual video data with a unified structure.

[0150] The superpixel region aggregation module is used for performing multi-scale superpixel region aggregation processing on the image frames of the multi-source visual video data to generate a structured visual image block set.

[0151] The spatio-temporal feature extraction module is used for inputting the structured visual image block set into a spatio-temporal graph convolutional encoder to extract spatial context features and time sequence features in continuous image frames, identify dynamic target categories, and output a target feature vector set.

[0152] The spatio-temporal interaction graph construction module is used for constructing a multi-target spatio-temporal interaction graph according to the target feature vector set, taking different target feature vectors as graph nodes, calculating the spatial position distance, speed difference and direction angle difference between nodes, analyzing the geodesic path between nodes in the Riemannian manifold space and calculating the straight line distance deviation, and combining the edge weights between nodes according to the preset weights.

[0153] The causal structure learning module is used for performing improved NOTEARS causal analysis on the multi-target spatio-temporal interaction graph, establishing causal connection hypotheses for each pair of graph nodes, generating causal connection candidates according to the time sequence, spatial proximity trend and state change consistency, and generating a causal structure graph through closed-loop detection and edge weight priority elimination, and identifying a risk trigger causal chain.

[0154] The risk assessment and early warning response module is used for risk scoring of each risk trigger causal chain in the causal structure diagram, calculating a weighted combination of path length and average edge weight to obtain a risk score value, and dividing the risk into three levels of prompt level, warning level and intervention level according to a risk score threshold set, and outputting corresponding early warning response instructions.

[0155] This step realizes the whole-process closed-loop processing from multi-source visual data acquisition to structured image processing, spatio-temporal feature extraction, causal relationship mining and hierarchical response control by constructing a modular high-speed risk early warning system. The data flow between the functional modules is efficient and the logic is clear, which guarantees the multi-angle modeling and real-time response capability of the system to potential risks in high-speed scenes. By integrating the improved NOTEARS causal analysis and Riemannian manifold mapping mechanism, the recognition accuracy of the system for complex spatio-temporal interaction relationships is improved, and the accuracy of risk prediction and the intelligence level of dynamic intervention are effectively enhanced.

[0156] Embodiment 1

[0157] In order to verify the feasibility of the application in implementation, the application is applied to a high-speed interchange experimental section with a total length of about 15 kilometers, complex lanes, small turning radius and overlapping blind areas. The experimental section is arranged with a multi-source visual acquisition system, including a front-view camera, double-blind area cameras and a top-view camera device configured on a UAV platform. All camera modules collect image data at a frequency of 30 frames per second, and the video resolution is 1920x1080 pixels.

[0158] The system performs gray scale normalization, brightness standardization, edge enhancement and blur repair on each frame of image in the high-speed driving state, completes preliminary clarification and structure unification. Then a multi-scale superpixel region aggregation algorithm is used to perform spatial region segmentation on each frame of image, and combined with region merging strategies under three scales, about 85 stable visual image blocks are generated on average per frame. These image blocks are input into the improved spatio-temporal graph convolutional encoder in time sequence, to extract spatial context features such as vehicle-to-vehicle distance, texture features, occlusion state, etc., and to extract time series features such as speed change trend, direction angle offset, etc. The model automatically identifies dynamic obstacles on the travel path, forming a target feature vector including multiple categories of targets such as vehicles, pedestrians and roadblocks.

[0159] Based on the above feature vector, a multi-target space-time interaction graph is constructed, and the motion relationship between different targets in the high-speed interaction environment is analyzed. By calculating the spatial distance, velocity difference and motion direction angle difference between nodes, and mapping them to the Riemannian manifold space, a more realistic dynamic approach relationship is measured, thereby giving the graph edge a reasonable interaction weight. Then, the system uses the improved NOTEARS causal inference algorithm to identify multiple dangerous trigger causal chains that may cause accidents in the graph. The system detects a space-time causal structure in a group of targets that has continuous rear-end acceleration increase, sudden state change of the front vehicle, and fast side-moving targets on both sides, and automatically classifies it as a "sudden stop following type" and "lateral conflict type" compound causal chain.

[0160] To quantify the risk level, the system calculates the weighted combination risk score of the edge weight average value and the path length for each causal chain. The trigger warning response is prompted when the score value is higher than the set threshold: the visual interface is prompted for warning, the path adjustment instruction is started for alert, and the emergency stop command is directly triggered for intervention. A total of 672 high-dynamic driving processes are collected in the experimental stage, 63 of which are artificially annotated accident risk cases, and 61 high-risk causal chains are detected by the system. The following is a performance comparison of the present invention method and three existing mainstream warning systems in the experiment.

[0161] Table 1: Performance comparison of multiple methods in high-speed dynamic warning

[0162]

[0163] From the data in Table 1, it can be seen that the present invention method is significantly better than existing methods in terms of target recognition accuracy, warning response delay, multi-target dangerous chain capture ability, blind area target recognition ability, system false alarm rate and emergency intervention trigger accuracy. Especially in constructing an interaction model between multiple targets, deducing potential causal chains and quantifying risks, it has significant innovations. In a typical high-speed blind area intrusion scene, the traditional YOLO-based detection system misses a group of targets entering from the blind area due to unclear edge detection, while the present invention method identifies the intrusion targets through the combination of overhead vision blocks and side-view blocks, and deduces a blind area intrusion type causal chain based on the speed difference and direction mutation, issuing a warning in advance before the target and the invaded object intersect, successfully avoiding potential risks. This embodiment verifies the accuracy and response efficiency of the present invention system in handling dangerous event perception in high-speed complex visual environments, and has wide engineering promotion prospects.

[0164] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can make equivalent substitutions or changes within the technical scope disclosed by the present invention and according to the technical solutions and inventive concepts of the present invention, which should be covered within the protection scope of the present invention.

Claims

1. A high speed hazard warning method based on machine vision, characterized in that, The method comprises the following steps: Step 1: Collect and pre-process multi-source visual video data to generate multi-source visual video data with a unified structure; Step 2: Divide the image frames of the multi-source visual video data into regions by using multi-scale superpixel region aggregation to obtain a set of structured visual image blocks, wherein the division of the image frames of the multi-source visual video data into regions by using multi-scale superpixel region aggregation comprises the following steps: Perform initial superpixel segmentation on each image frame of the multi-source visual video data, wherein the initial superpixel segmentation comprises the following steps: Calculate the two-dimensional spatial position of each pixel in the image coordinate system; Divide the image frame into regular grid regions with a fixed step size as initial clustering centers; Extract the color feature values of each pixel within a fixed radius range around each initial clustering center, wherein the color feature values include the brightness values of the red, green, and blue channels; For each pixel in the image frame, simultaneously calculate the color feature value difference and the spatial distance between the pixel and each initial clustering center, and fuse them into a comprehensive distance index by a preset weighting ratio, wherein the spatial distance is the Euclidean space distance between the pixel and the initial clustering center; Classify each pixel into the initial clustering center with the smallest comprehensive distance to generate a corresponding superpixel unit; Set three groups of scale parameters for the superpixel units, and respectively recalculate the average color feature values, center coordinates, and boundary texture gradients of the superpixel units at each scale; At the current scale, traverse all adjacent superpixel units to perform a region merging operation, wherein the region merging operation comprises merging superpixel units with a color feature value difference lower than a set threshold, center coordinates smaller than a specified pixel range, and a texture direction consistency between boundaries higher than a preset ratio into a new superpixel unit; Perform fusion processing on the superpixel units aggregated at the three groups of scales to obtain a set of structured visual image blocks; Step 3: Input the set of structured visual image blocks into a spatio-temporal graph convolutional encoder to extract spatial context features and temporal sequence features within consecutive image frames and generate a set of target feature vectors; Step 4: Construct a multi-target spatio-temporal interaction graph based on the set of target feature vectors; Step 5: Perform causal analysis on the multi-target spatio-temporal interaction graph using an improved NOTEARS algorithm to generate a causal structure graph containing dangerous trigger causal chains, wherein the causal analysis on the multi-target spatio-temporal interaction graph using the improved NOTEARS algorithm comprises the following steps: For each pair of graph nodes in the multi-target spatio-temporal interaction graph, sequentially establish a connection hypothesis that exists a causal relationship, wherein the establishment of the connection hypothesis comprises the following steps: For each pair of graph nodes, respectively obtain the corresponding image frame timestamps to determine whether the time causal order is satisfied; Extract the center position coordinates of the graph nodes in consecutive image frames at a fixed time interval to calculate the spatial distance; If a graph node is located before another graph node in time and the spatial distance between them continuously decreases in multiple time frames, it is determined that there is a spatial proximity causal trend; Analyze the speed and motion direction of the graph nodes to determine the direction of state change; For the node pairs meeting the time causal sequence and simultaneously meeting the spatial proximity causal trend and consistent state change direction in three or more consecutive time frames, a causal connection hypothesis is established, and the starting node and the target node are recorded; Based on the connection hypothesis, the state change sequence of each node pair in different time frames is analyzed, and the node pairs that show a fixed state change relationship in multiple time frames and have the same direction state change in at least three consecutive time windows are selected as causal connection candidates; For all causal connection candidates, closed loop detection is performed, if a group of causal connections forms a closed path, then the connection edges are removed according to the time sequence and the size of the edge weight, and a causal structure graph is generated, the removal step is: Retain the connection edge with the node occurring earlier in time as the starting point; If the time is equivalent, the connection edge with the higher edge weight is retained; If the edge weight is equivalent, one connection edge is randomly removed until there is no closed path; The directed edges of the causal structure graph represent that a target state has a triggering or guiding effect on another target state, the direction of the directed edge represents the causal sequence, and the weight of the directed edge represents the influence degree; In the causal structure graph, a dangerous trigger causal chain is identified, which is a potential risk event path, including: An emergency stop follow-up type causal chain formed by a target appearing high-speed proximity phenomenon before a sudden change in the front target state, the high-speed proximity phenomenon is that the target in the rear position has a higher speed than the front target in multiple consecutive time frames, and the speed difference between them is greater than a set proximity threshold, and the spatial distance is continuously decreasing in three consecutive image frames; A lateral conflict type causal chain in which a target appears to avoid action after a spatially adjacent target has a lateral displacement; A blind area intrusion type causal chain triggered by a target quickly entering a high-density area from an edge area, the high-density area is an area where the number of targets in a unit space is greater than a set density threshold, and is in the non-visible area of the subjective target or the sensor detection blind area; Step six: risk scoring and risk level division are performed on each dangerous trigger causal chain, and the corresponding early warning response instruction is output. 2.The machine vision based high speed hazard warning method of claim 1, wherein, The multi-source visual video data includes a front-view camera collected main-view image sequence in the traveling direction, a side camera collected image sequence in the blind area range on both sides, and a UAV collected overall dynamic environment image from the aerial perspective; the preprocessing includes respectively performing gray scale normalization, brightness standardization processing, motion blur repair, edge enhancement and time synchronization processing on different types of video images. 3.The machine vision based high speed hazard warning method of claim 1, wherein, The step three is specifically: The structured visual image block set is constructed into a frame-level graph sequence in time sequence; The frame-level graph sequence is input into the graph convolution coding module of the spatio-temporal graph convolution encoder, and the spatial context features of each image block in the current image frame are extracted through graph convolution operation, including texture, color, edge information and semantic association relationship between adjacent image blocks; In the adjacent image frames, the state changes of the same image block in the continuous image frames are modeled by using a time convolution unit, the motion trend and change amplitude of the image block in the time dimension are extracted, and a time sequence feature is obtained; The spatial context feature and the time sequence feature are fused to form a spatio-temporal joint feature representation of each image block in a sequence of continuous image frames, and a feature aggregation mechanism is used to integrate the information of multiple image frames to generate an image block feature vector enhanced by spatio-temporal context; The image block feature vector is input into a target recognition submodule, and a multi-class classifier is used to identify the dynamic target category corresponding to each image block, wherein the dynamic target category includes vehicles, pedestrians and obstacles; The target feature vector corresponding to the identified target category image block is extracted, and a target feature vector set is generated, wherein the target feature vector includes the center position coordinates of the target category image block in the image coordinate system, the motion speed in the time dimension, the motion direction and the appearance feature representation. 4.The machine vision based high speed hazard warning method of claim 1, wherein, The step four is specifically: According to the target feature vector set, a multi-target spatio-temporal interaction graph is constructed, and the construction step is: Different target feature vectors are used as graph nodes of the multi-target spatio-temporal interaction graph; All graph nodes are paired and combined, the spatial position distance, speed difference and motion direction angle difference between each pair of graph nodes are calculated, and a feature difference vector of the pair of graph nodes is obtained; The feature difference vector is mapped to the Riemann manifold space; The geodesic path from one graph node to another graph node in the Riemann manifold space is analyzed, and the deviation of the geodesic path from the straight line distance between the two nodes is compared; The measured spatial position distance, speed difference, motion direction angle difference and straight line distance deviation are combined by weighting according to the preset weight, and the edge weight between the pair of graph nodes is obtained. 5.The machine vision based high speed hazard warning method of claim 1, wherein, The step six is specifically: Risk scores are calculated for each dangerous trigger causal chain in the causal structure graph, and the risk score calculation step includes: Obtain the directed edge set constituting each dangerous trigger causal chain, count the edge weight of each directed edge and the path length of each dangerous trigger causal chain, and calculate the average edge weight; According to the preset weighting coefficient, the path length and the average edge weight are combined to calculate the risk score value of each causal chain; According to the risk score value and the preset risk score threshold set, different risk levels are divided, and the risk levels include: Prompt level, the risk score value is between the low threshold and the medium threshold, indicating that there is a potential risk; Warning level, the risk score value is between the medium threshold and the high threshold, indicating that there is a predictable risk; Intervention level, the risk score value exceeds the high threshold, indicating that there is an immediate response risk; The risk level is corresponded to the preset response strategy logic, and the early warning response instruction corresponding to the current risk level is output, and the early warning response instruction includes: When it is a prompt level risk, visual interface warning or voice reminder information is output; When it is a warning level risk, an auxiliary behavior guidance instruction is output, and the auxiliary behavior guidance instruction includes a motion path adjustment or a target gravity center shift guide; When it is an intervention level risk, a forced intervention instruction is output, and the forced intervention instruction includes an emergency stop or a preset avoidance behavior.

6. A machine vision-based high-speed hazard warning system, which performs the machine vision-based high-speed hazard warning method according to any one of claims 1 to 5, characterized in that, The following modules are included: The data acquisition and preprocessing module is configured to acquire and preprocess multi-source visual video data to generate multi-source visual video data with a unified structure. The superpixel region aggregation module is configured to perform multi-scale superpixel region aggregation processing on image frames of the multi-source visual video data to generate a set of structured visual image blocks. The spatio-temporal feature extraction module is configured to input the set of structured visual image blocks into a spatio-temporal graph convolutional encoder, extract spatial context features and time series features in continuous image frames, identify dynamic target categories, and output a set of target feature vectors. The spatio-temporal interaction graph construction module is configured to construct a multi-target spatio-temporal interaction graph according to the set of target feature vectors, take different target feature vectors as graph nodes, calculate spatial position distance, velocity difference, and direction angle difference between nodes, analyze geodesic path and calculate straight line distance deviation between nodes in the Riemannian manifold space, and combine the node edge weights according to a preset weight to generate node edge weights. The causal structure learning module is configured to perform improved NOTEARS causal analysis on the multi-target spatio-temporal interaction graph, establish causal connection hypotheses for each pair of graph nodes, generate causal connection candidates according to time sequence, spatial proximity trend, and state change consistency, and generate a causal structure graph through closed-loop detection and edge weight priority elimination, and identify a dangerous trigger causal chain. The risk assessment and early warning response module is configured to score each dangerous trigger causal chain in the causal structure graph, calculate a weighted combination of path length and average edge weight to obtain a risk score value, divide the risk into three levels of prompt level, warning level, and intervention level according to a set of risk score thresholds, and output corresponding early warning response instructions.

Images