A method, system and storage medium for security visualization display

Abstract

This invention relates to the field of security visualization technology, and more particularly to a method, system, and storage medium for security visualization display. The method includes the following steps: acquiring data streams collected by security devices and compressing them into a unified set of acquisition frames; identifying the road boundary point set within the unified set of acquisition frames and detecting dynamic and static targets at various times within the unified set of acquisition frames; semantically labeling the changes in the motion state of dynamic targets at various times within the unified set of acquisition frames to obtain a road boundary semantic distribution map; identifying and predicting the changes in the motion state of static targets based on the road boundary semantic distribution map; predicting the overlap time point of the target motion state based on the changes in the motion state of static and dynamic targets, and transmitting this overlap time point to the communication device of the static target via the Internet of Things to perform a voice prompt task. This invention helps improve the display accuracy and response efficiency of security visualization.

Description

Technical Field

[0001] This invention relates to the field of security visualization technology, and in particular to a method, system and storage medium for security visualization display. Background Technology

[0002] In existing technologies, security equipment primarily relies on surveillance cameras to capture images of key areas, and then uses a backend system to store, play back, and allow manual review of these images. Some systems have incorporated auxiliary functions such as image recognition and abnormal behavior analysis to enhance their responsiveness to critical events.

[0003] However, existing security systems still have significant limitations in terms of display and interaction. The high information density of video footage, coupled with a lack of structured abstraction methods, prevents personnel from quickly obtaining the spatial location and event attributes of key incidents or potential hazards.

[0004] In addition, most current security visualization methods rely on two-dimensional images or simple interface overlays, lacking the ability to understand the structural semantics of the target area, making it difficult to meet the comprehensive judgment needs for passage safety in complex scenarios. Summary of the Invention

[0005] Therefore, it is necessary for the present invention to provide a method, system and storage medium for security visualization display, in order to solve at least one of the above-mentioned technical problems.

[0006] To achieve the above objectives, a method for visualizing security displays includes the following steps:

[0007] Step S1: Acquire the data stream collected by the security device and compress the data stream to obtain a unified collection frame set;

[0008] Step S2: Identify the road boundary point set in the unified acquisition frame set, and use the road boundary point set as the principal axis of the road coordinate system to detect dynamic and static targets at each moment in the unified acquisition frame set;

[0009] Step S3: Based on the changes in the motion state of dynamic targets at various times in the unified collection of frames, distinguish the categories of dynamic targets, and perform semantic annotation on the categories of dynamic targets to obtain a semantic distribution map of road boundaries;

[0010] Step S4: Bind the corresponding image slices and timestamps to each node in the road boundary semantic distribution map to obtain the road boundary dynamic distribution map; analyze the interaction between dynamic and static targets based on the road boundary dynamic distribution map, and identify and predict the changes in the motion state of static targets;

[0011] Step S5: Based on the changes in the motion state of the static target and the changes in the motion state of the dynamic target, predict the overlap time point of the target's motion state, and transmit the overlap time point to the communication device of the static target through the Internet of Things to perform the voice prompt task.

[0012] Optionally, step S1 includes:

[0013] Step S11: Obtain the security equipment data stream and auxiliary status data through the security equipment deployed in the security area, wherein the security equipment data stream includes RGB visible light image sequence, depth map sequence and infrared thermal imaging data;

[0014] Step S12: Perform device timestamp alignment processing on the data stream collected by the security equipment, and perform frame reconstruction interpolation on the timestamp alignment processing result based on the camera time series in the device auxiliary status data to obtain a synchronized image frame sequence;

[0015] Step S13: Based on the pose parameters of each acquisition device and the lens intrinsic parameters in the auxiliary status data of the equipment, perform spatial viewpoint transformation processing on the synchronized image frame sequence to generate a viewpoint normalized image set.

[0016] Step S14: Perform inter-frame difference feature extraction on the viewpoint normalized image set, remove redundant frames without target motion changes, and effectively sample and compress the remaining frames to generate a high-confidence frame sequence.

[0017] Step S15: The RGB visible light image, depth map, infrared thermal image and timestamp in the high confidence frame sequence are stitched and encapsulated to obtain a unified acquisition frame set.

[0018] Optionally, step S14 includes:

[0019] Step S141: Perform two-frame sliding pairing of the viewpoint normalized image set in chronological order, and perform pixel difference calculation on each frame pair of the sliding pairing result to obtain the inter-frame difference map.

[0020] Step S142: Perform spatial connected component extraction on the inter-frame difference map, remove regions with a connected component area of ​​less than 20 pixels, and construct an inter-frame motion activation mask based on the maximum bounding box size and density distribution of the remaining regions.

[0021] Step S143: Calculate the proportion of activated pixels in the inter-frame motion activation mask of each frame. If the proportion of activated pixels is less than 2% of the preset proportion threshold, the frame is marked as a redundant frame and discarded; the remaining frames are retained as a candidate frame set.

[0022] Step S144: Set the time window to 2s, perform density distribution analysis on the frame time series in the candidate frame set, and select the frame with the largest average motion activation area from the density distribution analysis results in each time window as the representative frame to obtain the representative frame set.

[0023] Step S145: Perform a confidence assessment on the representative frame set, and retain the frames with a confidence assessment result greater than 0.75 to obtain a high-confidence frame sequence.

[0024] Optionally, identifying the set of road boundary points in the unified acquisition frame set in step S2 includes:

[0025] Edge detection is performed on the RGB visible light image, depth map, and infrared thermal image corresponding to each frame in the unified acquisition frame set, and the edge detection results are weighted and fused to obtain a fused edge response map.

[0026] Extract the set of main line segments from the fused edge response map, classify the set of main line segments according to their angular distribution, and select continuous line segments in the horizontal direction with a length exceeding 20% ​​of the image width as candidate road boundary line segments;

[0027] Calculate the boundary symmetry of the left and right region line segment pairs in the candidate road boundary line segments, and retain the line segment pairs with a boundary symmetry greater than 0.85 to obtain the boundary line segment pairs;

[0028] The boundary line segment pairs are reverse-mapped back to each view frame in the unified acquisition frame set. The boundary position reprojection error is evaluated on the mapping result. Boundary line segment pairs with projection errors less than 4 pixels in each view frame are retained to obtain the verified boundary line segment pairs.

[0029] The verification boundary line segments are fitted with curves to obtain the fitted curve; equidistant interpolation sampling is performed on both sides of the fitted curve to obtain the road boundary point set.

[0030] Optionally, the detection of dynamic and static targets at various times in the unified acquisition frame set in step S2 includes:

[0031] The road boundary point set is divided into two boundary points, resulting in the left boundary point set and the right boundary point set;

[0032] Spline curve fitting is performed on the left boundary point set and the right boundary point set respectively to obtain the left continuous boundary curve and the right continuous boundary curve;

[0033] For each set of continuous boundary curves on the left and right sides, calculate the centerline points at the corresponding positions, and calculate the tangent vector and normal vector of the centerline points to construct the road coordinate system.

[0034] The spatial positions of the bounding boxes at each time point in the unified collection of frames are projected onto the corresponding road coordinate system to obtain the spatial position data of the target to be analyzed.

[0035] Cross-frame correlation is performed on the spatial location data of the target to be analyzed at each time point. Targets that appear in less than 3 frames are marked as misjudged targets and removed to obtain the target trajectory set.

[0036] Calculate the center point motion vector of each target in the target trajectory set between consecutive frames, and determine the target motion state based on the center point motion vector to obtain the motion state labeled target set;

[0037] The motion state marker target set is overlaid with the road coordinate system, and only targets within the boundary area are retained. The retained targets are divided into dynamic targets and static targets according to their motion state. The boundary area is the region enclosed by the continuous boundary curves on the left and right.

[0038] Optionally, step S3 includes:

[0039] Step S31: Set a sliding time window, segment the target trajectory corresponding to the dynamic target in the target trajectory set to obtain a short-time trajectory segment; calculate the motion vector sequence, velocity change rate, acceleration characteristics and direction offset angle of the short-time trajectory segment within the continuous time window to obtain the motion trajectory feature set;

[0040] Step S32: Embed each trajectory behavior in the motion trajectory feature set to obtain a trajectory behavior embedding vector set;

[0041] Step S33: Remove outlier trajectory behaviors from the trajectory behavior embedding vector set, and use a predefined target category dictionary to label the remaining trajectory behaviors with category labels to obtain the category of the dynamic target;

[0042] Step S34: Label each target category in the category of dynamic targets, and combine it with its timestamp and spatial coordinates projected in the road coordinate system to perform spatial node encoding, thereby obtaining a spatial node encoding set;

[0043] Step S35: Perform inter-frame semantic continuity filling on the time axis of the spatial node encoding set, and at the same time construct the motion boundaries of various targets on the spatial axis, thereby constructing a road semantic partition map;

[0044] Step S36: Project the road semantic partition map onto a standard raster map established in the road coordinate system to construct a road boundary semantic distribution map.

[0045] Optionally, step S4 includes:

[0046] Step S41: Obtain the two-dimensional coordinates of each semantic node in the road coordinate system based on the semantic distribution map of the road boundary, and bind the image frame and timestamp in the high-confidence frame sequence that is closest to the two-dimensional coordinate point to obtain the semantic image binding node set;

[0047] Step S42: Sort all nodes in the semantic image binding node set by timestamp and aggregate them into frame segments by sliding time window to obtain a dynamic distribution map of road boundaries;

[0048] Step S43: Calculate the nearest neighbor spatial distance between dynamic and static targets in the dynamic distribution map of road boundaries, predict the nearest neighbor spatial distance trend of adjacent frames, construct the interaction map between dynamic and static targets, and obtain the target interaction map.

[0049] Step S44: Select static target nodes affected by dynamic targets from the target interaction graph, perform change tendency analysis, and obtain the static target motion probability set;

[0050] Step S45: For static targets whose motion probability concentration changes more than 0.7, perform continuous time window confidence verification. If the change probability is higher than 0.7 for two consecutive time windows, then mark the static target as a motion trend target, thereby obtaining the change of the static target's motion state.

[0051] Optionally, the predicted target motion state overlap time points in step S5 include:

[0052] Based on a spatial threshold of 1.5m and a temporal overlap window of 3s, interactive pairs of candidate dynamic targets and static targets are constructed from the changes in the motion state of static targets and the dynamic distribution map of road boundaries, resulting in a set of candidate target pairs.

[0053] Perform constant-rate-mutation prediction on the static targets in the candidate target pair set to obtain the static target state prediction trajectory set;

[0054] Multi-time-step trajectory extrapolation is performed on the dynamic targets in the candidate target set to obtain the dynamic target state prediction trajectory set;

[0055] Based on the static target state prediction trajectory set and the dynamic target state prediction trajectory set of each group, calculate the Euclidean distance between the two prediction trajectories at each moment, mark the set of all time points where the Euclidean distance is less than 0.8m, and obtain the set of prediction conflict time points.

[0056] The motion overlap risk is determined for the set of predicted conflict time points. If the predicted conflict time point between two predicted trajectories in this set exceeds 0.5s, it is determined to be a motion overlap risk time point, and the potential motion conflict time segment set is obtained.

[0057] The time point with the smallest spatial distance in the set of potential motion conflict time segments is taken as the predicted overlapping time point to obtain the overlapping time point.

[0058] This invention introduces multi-level processing methods in security scenarios, including structured semantic modeling, dynamic trajectory analysis, target interaction recognition, and proactive prompting feedback. It overcomes the technical bottlenecks of existing systems that rely on intuitive image observation, suffer from delayed response, and passive interaction, significantly improving display accuracy, judgment depth, response efficiency, and intervention capabilities. In the initial stage of video processing, a motion activation mask is constructed through inter-frame pixel activation analysis, with a 2% redundancy threshold set. This effectively filters out background images with insufficient motion, significantly compresses redundant information interference, and improves system processing efficiency and resource scheduling capabilities. Simultaneously, within a 2-second time window, a representative frame with the largest average activation area is selected, and a confidence threshold of 0.75 is set for the retained frames. This ensures that displayed information is always concentrated on the most dynamic and significant moments, enhancing the focus of event perception and facilitating operators to quickly locate abnormal areas in large scenes. In terms of spatial structure modeling, a fused edge response map is formed through multimodal image fusion (RGB, infrared, depth) to avoid potential lighting, occlusion, or texture interference issues that may exist with a single image source. Furthermore, by jointly constraining boundary symmetry (threshold set to 0.85) and reprojection error (less than 4 pixels), the extraction of road boundaries ensures both geometric symmetry and multi-view consistency, fundamentally improving the spatial representation capability of security scenarios. Subsequently, semantic nodes constructed based on road boundaries and their binding with high-confidence frames enable spatial semantic mapping and visualization alignment of the originally complex and unstructured dynamic scenes. Compared to the image coordinate system or world coordinate system commonly used in existing security systems, the road coordinate system has significant structural advantages and improved practical performance. In traditional security systems, image coordinate systems rely solely on pixel-level positions for target labeling, lacking an understanding of the actual road structure. This makes it difficult to abstractly represent spatial relationships of targets, especially when camera perspectives change or regional structures are complex, as target behavior paths and traffic logic cannot be effectively modeled. While world coordinate systems can provide some spatial positioning capabilities, their construction depends on high-precision extrinsic parameter calibration and environmental modeling, making adaptation difficult. Furthermore, they lack a close connection with the semantic structure of roads, making it difficult to support constraint judgments and rule reasoning for dynamic target behavior within actual traffic structures. The road coordinate system of this invention automatically generates a centerline coordinate system consistent with the actual road direction and possessing physical semantic meaning through steps such as extracting road boundary point sets, fitting continuous boundary curves, and constructing centerlines and their direction vectors. It possesses real-time construction capabilities while preserving road geometric attributes and directional logic, serving as a unified spatial reference benchmark for position reduction, motion correlation, and conflict analysis of multiple types of targets. Its structure not only boasts strong adaptability and high semantic relevance, but also significantly enhances the intelligence level of security visualization systems in target recognition, behavior prediction, and event early warning, overcoming the limitations of existing coordinate systems in terms of insufficient expressiveness and weak interactive logic in complex security scenarios. It effectively compensates for the shortcomings of existing security visualization methods in spatial structure expression and target semantic organization.By constructing continuous boundary curves on the left and right sides based on boundary points, and combining the tangent and normal vector information of their corresponding centerlines, a road coordinate reference system with directionality and hierarchical structure is formed, enabling spatial positioning and semantic classification of targets in the monitoring screen. This coordinate system not only improves the structuring of monitoring data, but also, through standardized mapping of target positions, allows dynamic targets and static obstacles to be intuitively displayed and dynamically tracked within a unified spatial framework, significantly reducing the cognitive burden of manual identification of complex incidents. Furthermore, by using this road coordinate system to predict target behavior paths and detect conflicts, potential traffic risks can be identified in advance, providing accurate security decision-making basis for high-density traffic areas or emergency passage scenarios. At the behavioral analysis level, short-term velocity vectors, direction angles, and acceleration features are extracted from the trajectories of dynamic targets. By setting conditions such as mean velocity > 0.1 m / s and standard deviation < 0.05, the system can identify objects with potential moving intentions from low-speed or stationary targets. For trajectory prediction, static targets are constructed by combining a disturbance trajectory branch with a disturbance angle of ±15° and a velocity disturbance of ±25%. This allows the prediction model to not only characterize normal stable behavior but also accommodate the feasibility of abrupt changes, significantly reducing the misjudgment rate for unpredictable factors. For dynamic targets, the model extracts joint features of global behavior and spatial neighbors to achieve more reasonable and interactive reasoning about future trajectories. The key lies in the construction of the interaction graph and the identification of motion overlap risks. By setting a spatial filtering distance of 1.5m, a danger judgment distance of 0.8m, and a minimum conflict duration of 0.5s, the model achieves the identification of real conflict events between dynamic and static targets, avoiding false alarms triggered by accidental proximity and ensuring the practicality and accuracy of the alerts. Furthermore, static targets with high abrupt change probabilities undergo confidence verification over two consecutive time windows. Only targets exhibiting continuous and stable trends are marked as trend targets, improving the system's noise resistance and the stability of behavior judgment. Ultimately, the determined conflict time point is transmitted to the target terminal via 5G communication, and its voice prompt module proactively issues commands such as "Please be aware and avoid the area." Compared to traditional systems that rely solely on monitoring images and backend viewing, this achieves a closed-loop integration of intelligent information recognition, target localization, and localized intervention. Especially in pedestrian-vehicle mixed scenarios, such intelligent terminals can be deployed on non-motorized vehicles, smart bracelets, or fixed facilities to achieve target-level real-time alerts and distributed security linkage. Its beneficial effects lie not only in enhancing the target's self-perception and self-avoidance capabilities but also in truly expanding security equipment from "passive data collection" to "active guidance," integrating the triple values ​​of traffic safety judgment, spatial dynamic understanding, and individual behavior early warning.

[0059] Optionally, this application also provides a security visualization display system for performing the security visualization display method described above, the security visualization display system comprising:

[0060] The data stream compression module is used to acquire and compress the data stream collected by the security equipment to obtain a unified set of acquisition frames.

[0061] The target classification module is used to identify the road boundary point set in the unified acquisition frame set, and to detect dynamic and static targets at each moment in the unified acquisition frame set, using the road boundary point set as the main axis of the road coordinate system.

[0062] The dynamic target category differentiation module is used to differentiate the categories of dynamic targets based on the changes in the motion state of dynamic targets at various times in the unified collection of frames, and to perform semantic annotation on the categories of dynamic targets to obtain a semantic distribution map of road boundaries.

[0063] The static target motion prediction module is used to bind corresponding image slices and timestamps to each node in the road boundary semantic distribution map to obtain the road boundary dynamic distribution map; based on the road boundary dynamic distribution map, it analyzes the interaction between dynamic and static targets, and identifies and predicts the changes in the motion state of static targets.

[0064] The target collision warning module is used to predict the overlap time point of the target motion state based on the changes in the motion state of static targets and the changes in the motion state of dynamic targets, and transmits the overlap time point to the communication device of the static target through the Internet of Things to perform voice prompt tasks.

[0065] The present invention relates to a security visualization display system. This system can implement any of the security visualization display methods of the present invention. It serves as a medium for coordinating the operation and signal transmission between various modules to complete the security visualization display method. The internal modules of the system cooperate with each other, thereby improving the display accuracy and response efficiency of security visualization.

[0066] Optionally, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the security visualization display method described above. Attached Figure Description

[0067] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0068] Figure 1 This is a flowchart illustrating the steps of applying the present invention to the security visualization display method;

[0069] Figure 2 This is a detailed flowchart of step S1 in the present invention;

[0070] Figure 3 This is a schematic diagram of the road coordinate system in step S2 of the present invention;

[0071] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0072] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0073] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0074] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0075] To achieve the above objectives, please refer to Figures 1 to 3 This invention provides a method for visualizing security applications, the method comprising the following steps:

[0076] Step S1: Acquire the data stream collected by the security device and compress the data stream to obtain a unified collection frame set;

[0077] In this embodiment, five sets of security equipment are deployed at the main entrance of a residential community. Each set includes an RGB high-definition camera, a TOF depth sensor, and an infrared thermal imaging unit, and data is synchronized through a unified edge acquisition node. During the acquisition process, 30 frames each of RGB image, depth map, and infrared image are acquired synchronously per second, while the device's attitude angle and lens parameters are recorded. First, the acquired raw image stream is initially synchronized using the camera's local timestamp, with an error tolerance range of ±20ms. For image frames with time differences, spatial alignment is performed based on the attitude angle change trend, and missing frames are reconstructed through linear interpolation. After time synchronization is completed, the camera's intrinsic parameter matrix and device attitude are used to jointly perform unified viewpoint transformation processing, standardizing all image frames to a forward viewpoint. Subsequently, the normalized image frame pairs are subjected to pairwise differencing in a time-sliding window manner, calculating the brightness, depth, and thermal radiation changes of each pixel in adjacent frames, and static frames are eliminated based on the area of ​​connected regions and the proportion of active pixels. In the remaining frames, one representative frame with the most active motion is selected every 2-second time window, and frames with a confidence score greater than 0.75 are retained through motion region confidence evaluation. Finally, the RGB image, depth map, heat map and their timestamps are stitched together into a unified data frame structure to form a unified acquisition frame set.

[0078] Step S2: Identify the road boundary point set in the unified acquisition frame set, and use the road boundary point set as the principal axis of the road coordinate system to detect dynamic and static targets at each moment in the unified acquisition frame set;

[0079] In this embodiment, the obtained unified acquisition frame set is first processed using edge gradient-based image processing techniques to identify possible boundary line segments for each frame. RGB images use Sobel edge extraction, depth maps use margin gradient enhancement, and infrared images use thermal intensity change rate mapping. The edge response maps of the three images are then fused with a weight ratio of 3:2:1 to generate a fused edge map. From this map, a set of continuous line segments with approximately horizontal orientations and lengths exceeding 20% ​​of the image width is extracted. The boundary symmetry of each pair of left and right line segments is calculated, and only line segment combinations with a symmetry greater than 0.85 are retained. Subsequently, this is projected onto multi-view images, and its reprojection error is evaluated. Line segment pairs with errors less than 4 pixels across all view frames are retained. Quadratic curve fitting is performed on the validated line segment pairs to obtain the boundary curves on both sides of the road. Based on these curves, uniform interpolation is performed on the road area to generate a road boundary point set. When constructing the road coordinate system, cubic spline fitting is performed on the left and right boundary points respectively, and the tangent vector and normal vector of each center point are extracted to establish a spatial reference system. The center point of the target detection box in each frame is projected onto the road coordinate system, and the target center point sequence of all frames is correlated. A minimum of 3 consecutive frames is set, and only consecutively appearing targets are retained. The inter-frame motion vector magnitude is calculated for these targets; those with a motion speed greater than 0.05 m / s are marked as dynamic targets, and the rest as static targets. Only targets located within the fitted boundary area are retained for classification.

[0080] Step S3: Based on the changes in the motion state of dynamic targets at various times in the unified collection of frames, distinguish the categories of dynamic targets, and perform semantic annotation on the categories of dynamic targets to obtain a semantic distribution map of road boundaries;

[0081] In this embodiment, within a unified collection of frames, a sliding time window of 1.5 seconds is set for the trajectory data identified as dynamic targets, and short-term trajectory segments of each dynamic target are extracted. For each trajectory segment, the angle of change in motion direction, velocity increment, and acceleration change amplitude between frames are extracted, and the direction offset rate is calculated to form a motion trajectory feature vector. This vector is mapped to an embedded vector structure with a dimension of 64, representing the differences in speed, direction, and temporal continuity among different behavior patterns. After removing abnormal behavior trajectories using outlier detection, the remaining vectors are matched with a pre-constructed target behavior category dictionary, which includes eight target behavior categories such as "fast pedestrians," "slow pedestrians," "scooters," and "small electric vehicles." The identified category labels are bound to their spatial projection coordinates and timestamps to form spatial semantic nodes, and inter-frame filling operations are performed on the time axis to ensure semantic continuity. In the spatial dimension, the boundary envelope of the region where each type of target appears is extracted to establish the target behavior boundary. A semantic partition map is constructed for the entire road area and projected onto the road coordinate grid to output the final road boundary semantic distribution map. Each grid unit in this distribution map represents the probability density of a certain type of dynamic target and corresponds precisely to road coordinates.

[0082] Step S4: Bind the corresponding image slices and timestamps to each node in the road boundary semantic distribution map to obtain the road boundary dynamic distribution map; analyze the interaction between dynamic and static targets based on the road boundary dynamic distribution map, and identify and predict the changes in the motion state of static targets;

[0083] In this embodiment, the two-dimensional coordinates of each node in the obtained road boundary semantic distribution map are matched with the corresponding image frames in the high-confidence frame sequence, and the nearest timestamp image is taken as the binding image for that semantic node. All binding nodes are arranged in ascending order of timestamp and aggregated into frame segments in a sliding window manner (2 seconds per window) to form a time-series dynamic distribution map of the road boundary. In this distribution map, each frame is traversed, and the position coordinates of dynamic and static targets are extracted respectively, and the spatial distance between nearest neighbors is calculated. By comparing the spatial distance trends of adjacent frames, the presence of proximity behavior is evaluated. All target pairs with proximity trends are constructed into a set of interaction edges to form a target interaction graph. For static target nodes in the interaction graph, the frequency and direction consistency of their interference by dynamic targets are statistically analyzed, and their change probability in the next 3 seconds is calculated. The change probability threshold is set to 0.7. If the probability of a static target exceeds this threshold in two consecutive sliding windows, the target is considered to have a movement trend. Finally, the time period, spatial location, and correspondence with the interaction targets of this batch of moving trend targets are output as the basis for subsequent conflict judgment.

[0084] Step S5: Based on the changes in the motion state of the static target and the changes in the motion state of the dynamic target, predict the overlap time point of the target's motion state, and transmit the overlap time point to the communication device of the static target through the Internet of Things to perform the voice prompt task.

[0085] In this embodiment, after obtaining the motion trend of static targets, it is matched with the predicted trajectory of dynamic targets. A spatial overlap threshold of 1.5 meters and a time window of 3 seconds are set to filter out all target pairs that may interact within the time and spatial range. For each group of static targets, the center position trajectory of the most recent 4 seconds is extracted, and the mean and standard deviation of the velocity vector are calculated. If the mean velocity is >0.1 m / s and the standard deviation is <0.05, it is determined to be a stable low-speed trend target, and a constant-velocity linear trajectory is extrapolated. For targets with a mutation probability higher than 0.7, a trajectory branch containing ±15° angle and ±25% velocity perturbation is constructed. In parallel, dynamic targets use the direction, velocity, and acceleration features in their continuous trajectories, combined with the behavioral features of the five spatial nearest neighbor targets in the trajectory neighborhood, to predict future multi-time-step trajectories. The Euclidean distance between the two types of trajectories is calculated every second within the prediction time window, and all time points below 0.8 meters are filtered out as potential conflict points. If the continuous conflict period exceeds 0.5 seconds, the group of targets is marked as having a risk of overlapping movement. The time point with the smallest spatial distance within the conflict period is selected as the final overlap time point. This time point information is transmitted via 5G IoT connection to the embedded communication terminal or application software of the corresponding static target, triggering the voice module to issue voice prompts such as "Please pay attention and avoid the area," thus achieving proactive security alerts and interventions. The static target can be a motor vehicle with an integrated communication terminal, a non-motorized vehicle with a mobile smart terminal, or a living person wearing a connected smart bracelet with voice prompt functionality. The design of sending voice prompts to static targets is mainly based on the risk that static targets may move or conflict unexpectedly due to the influence of dynamic targets in the environment. Static targets typically include parked motor vehicles, parked non-motorized vehicles, or people wearing smart bracelets. These objects are relatively fixed in the environment but have potential movement trends. By predicting the overlap time point of the trajectories of static and dynamic targets in real time, and proactively transmitting early warning information to the communication devices of static targets via IoT, early intervention and alerts for potential dangers can be achieved. This approach not only enhances proactive safety measures and reduces the probability of collisions or congestion, but also effectively improves the response efficiency and management intelligence of security systems, ensuring the safety and smooth flow of people and vehicles in complex and dynamic environments.

[0086] Optionally, step S1 includes:

[0087] Step S11: Obtain the security equipment data stream and auxiliary status data through the security equipment deployed in the security area, wherein the security equipment data stream includes RGB visible light image sequence, depth map sequence and infrared thermal imaging data;

[0088] In this embodiment, four sets of multimodal security monitoring equipment are deployed at the main entrance area of ​​a residential community. Each set includes a 2-megapixel color video camera (RGB visible light), a 640×480 resolution TOF depth-sensing camera, and an infrared thermal imager with a thermoelectric sensor array. These devices are arranged in a triangular staggered layout, covering an intersection area ranging from 3 to 12 meters, with an installation height of 3.2 meters and a downward tilt of 15°. Each set of equipment is equipped with a local edge computing unit for acquiring image sequences and auxiliary status data. The auxiliary status data includes the acquisition timestamp of each frame (accuracy 0.001s), device number, current attitude angle (pitch angle, yaw angle, roll angle), and intrinsic parameter information such as lens focal length and aperture value. The acquisition frequency is uniformly set to 30 frames / second, and all acquired data is transmitted in real time to the edge server in the area for processing via the network port.

[0089] Step S12: Perform device timestamp alignment processing on the data stream collected by the security equipment, and perform frame reconstruction interpolation on the timestamp alignment processing result based on the camera time series in the device auxiliary status data to obtain a synchronized image frame sequence;

[0090] In this embodiment, the image sequences uploaded by each device in the data stream acquired by the security equipment are initially time-aligned according to the acquisition timestamp. The frame synchronization tolerance range is set to ±10ms. If the maximum time difference between different modal image frames exceeds this range, the frame is determined to be invalid and discarded directly. Based on this, the frame sequence is judged to have frame loss or timing gaps according to the camera time series provided by the device. For time points where frames can be interpolated, interpolated frames are synthesized using a weighted reconstruction method based on the pose difference and image brightness gradient between the two frames, ensuring that the interpolated frames are consistent with the original frames in terms of spatial structure and light intensity distribution. All processed frames are renumbered with the same frame number and given a unified time tag, ultimately forming an image sequence of 30 frames per second with three channels synchronized, which serves as the basis for subsequent analysis.

[0091] Step S13: Based on the pose parameters of each acquisition device and the lens intrinsic parameters in the auxiliary status data of the equipment, perform spatial viewpoint transformation processing on the synchronized image frame sequence to generate a viewpoint normalized image set.

[0092] In this embodiment, the corresponding camera pose and lens parameters are extracted from the obtained synchronized image frame sequence. For each frame, the camera extrinsic parameter matrix is ​​constructed based on the Euler angles and 3D displacement values ​​provided by the capturing device. Simultaneously, an intrinsic parameter matrix is ​​constructed by combining the lens focal length, principal point position, and radial distortion coefficient. Using these parameters, each image is back-projected onto a unified world coordinate reference plane and uniformly transformed to a set top-down viewpoint. The goal is to ensure that the principal view axis of all images is perpendicular to the road surface, and that the horizontal projection is not offset. For infrared images and depth maps, linear interpolation is used to fill in voids, and spatial resampling is used to maintain the same resolution (uniformly 720×1280 pixels) for each modality image after transformation. The resulting viewpoint-normalized image set possesses spatial consistency and is suitable for subsequent cross-modal joint analysis.

[0093] Step S14: Perform inter-frame difference feature extraction on the viewpoint normalized image set, remove redundant frames without target motion changes, and effectively sample and compress the remaining frames to generate a high-confidence frame sequence.

[0094] In this embodiment, inter-frame processing is performed on the viewpoint-normalized image set. First, the changes in image frames are analyzed segment by segment using a time sliding window (window width 1 second, step size 0.5 seconds). Within each time period, pairwise difference operations are performed on consecutive image frames to calculate the brightness gradient change in the RGB image, the area of ​​the geometric abrupt change region in the depth image, and the temperature hotspot movement trajectory in the infrared image. A threshold is set: when the difference response value (the maximum value of the three channels after normalization) of an image frame is less than 0.05, and it remains below this threshold for more than 5 consecutive frames, the frame is marked as a frame with no significant change and is discarded. Among the remaining image frames, the frame with the most active motion features in each sliding window is selected as the representative frame, and contour stability analysis is performed on the target region of the frame. Only frames with a motion region edge integrity greater than 80% and a target area greater than 400 pixels are retained. The image frames retained after compressed sampling constitute a high-confidence frame sequence, and their corresponding modal images are stored respectively.

[0095] Step S15: The RGB visible light image, depth map, infrared thermal image and timestamp in the high confidence frame sequence are stitched and encapsulated to obtain a unified acquisition frame set.

[0096] In this embodiment, the RGB image, depth map, and infrared image of the obtained high-confidence frame sequence are extracted frame by frame according to the timestamp. The images are then uniformly sized to 720×1280 pixels while maintaining consistent image alignment. Subsequently, the three-modal images in each frame are encapsulated in a four-channel manner according to the "RGB-DEP-IR-TS" format. The RGB image and depth map are encoded using 8-bit grayscale compression, the infrared image maintains 16-bit thermal radiometric precision, and the timestamp is appended as metadata to the header of the frame file. Finally, the images are encapsulated into a unified image frame structure unit, each occupying approximately 1.2MB of memory, and possessing the characteristics of temporal consistency, viewpoint uniformity, and motion saliency. All frame structures are arranged in chronological order to form a unified acquisition frame set, serving as the basic data input source for subsequent boundary recognition and target tracking.

[0097] Optionally, step S14 includes:

[0098] Step S141: Perform two-frame sliding pairing of the viewpoint normalized image set in chronological order, and perform pixel difference calculation on each frame pair of the sliding pairing result to obtain the inter-frame difference map.

[0099] In this embodiment, the viewpoint-normalized image set is first strictly sorted according to timestamps to ensure the continuity and temporal consistency of the image frame sequence. Then, a sliding window of two frames is used to progressively traverse the entire sequence, forming frame pairs. For each frame pair, the RGB images of the two frames are first converted to grayscale images. The luminance channel is extracted using a weighted average formula (0.299×R + 0.587×G + 0.114×B) to reduce the influence of color on differences. Next, the luminance difference at corresponding pixel positions is calculated, generating a difference image (same size as the original image, 720×1280 pixels) with a value range of 0 to 255, reflecting pixel-level luminance changes. A difference threshold of 30 is set; pixels with a difference exceeding this threshold are marked as "moving pixels," while the rest are considered stationary. The difference image is stored in 8-bit grayscale format, forming an inter-frame difference map. This difference map is the basis for subsequent motion region extraction, reflecting areas in the image that change significantly over time.

[0100] Step S142: Perform spatial connected component extraction on the inter-frame difference map, remove regions with a connected component area of ​​less than 20 pixels, and construct an inter-frame motion activation mask based on the maximum bounding box size and density distribution of the remaining regions.

[0101] In this embodiment, spatial connectivity analysis is performed on the inter-frame difference map obtained in step S141 to filter motion regions. Specifically, an 8-neighborhood-based connected region detection method is used to cluster all positions with pixel values ​​greater than the threshold (30) to form several spatial connected regions. For each connected region, the total number of pixels and the size of its bounding box are counted. Connected regions with an area less than 20 pixels are removed to avoid misjudgments caused by environmental noise, lighting fluctuations, and device jitter. For the retained connected regions, the aspect ratio and density distribution of the maximum bounding box are further calculated. The density is defined as the number of active pixels in the connected region divided by the area of ​​the bounding box. Using this information, an inter-frame motion activation mask is constructed, which is a binary image with a size of 720×1280, where active region pixels are marked as 1 and non-active region pixels are marked as 0. This mask accurately depicts the spatial range of moving objects in the image and eliminates interference regions.

[0102] Step S143: Calculate the proportion of activated pixels in the inter-frame motion activation mask of each frame. If the proportion of activated pixels is less than 2% of the preset proportion threshold, the frame is marked as a redundant frame and discarded; the remaining frames are retained as a candidate frame set.

[0103] In this embodiment, the proportion of activated pixels is statistically analyzed for the motion activation mask corresponding to each frame. The calculation method is to divide the number of activated pixels by the total number of pixels (921,600). When the proportion is lower than a preset threshold of 2% (i.e., less than 18,432 activated pixels), the frame is determined to lack significant motion, marked as a redundant frame, and excluded from subsequent processing. For example, a frame of an empty road with no vehicles is usually discarded, greatly reducing the amount of data. All frames with an activation pixel proportion ≥ 2% are retained to form a candidate frame set, providing candidate samples for subsequent keyframe screening. This process ensures that processing resources are concentrated on dynamic and significant frames, improving efficiency while reducing false positives.

[0104] Step S144: Set the time window to 2s, perform density distribution analysis on the frame time series in the candidate frame set, and select the frame with the largest average motion activation area from the density distribution analysis results in each time window as the representative frame to obtain the representative frame set.

[0105] In this embodiment, the obtained candidate frame set is divided into multiple non-overlapping time windows based on timestamps, with a window width of 2 seconds. Within each time window, the motion activation area (i.e., the number of activated pixels in the mask) of all candidate frames is counted, and the average activation area is calculated. The frame with the largest activation area within the time window is selected as the representative frame, representing the richest motion information within that time period. For example, if a 2-second window contains 30 frames, the representative frame is the one with the most obvious motion and the most complete target. If the number of frames in a window is less than 5, all frames are retained to avoid losing key dynamics. This method, through temporal density analysis, ensures that the representative frame can fully reflect the dynamic characteristics of the time segment, facilitating subsequent fine processing.

[0106] Step S145: Perform a confidence assessment on the representative frame set, and retain the frames with a confidence assessment result greater than 0.75 to obtain a high-confidence frame sequence.

[0107] In this embodiment, a multi-dimensional confidence assessment is performed on the obtained representative frame set to comprehensively determine the quality and reliability of each frame. The assessment includes image sharpness (using the mean edge intensity gradient, with a threshold of 0.8), spatial coherence of moving regions (shape similarity of activation masks across consecutive frames ≥ 0.75), and spatial consistency of multimodal data (positional deviation between RGB, depth, and infrared images does not exceed 3 pixels). These indicators are fused to generate a comprehensive confidence score; frames exceeding 0.75 are considered high-confidence frames and included in the high-confidence frame sequence. This sequence ensures excellent image quality and sufficient dynamic information, providing a solid data foundation for target recognition and behavior analysis. The final sequence size is typically reduced by more than 70% compared to the original number of frames, greatly improving subsequent computational efficiency.

[0108] Optionally, identifying the set of road boundary points in the unified acquisition frame set in step S2 includes:

[0109] Edge detection is performed on the RGB visible light image, depth map, and infrared thermal image corresponding to each frame in the unified acquisition frame set, and the edge detection results are weighted and fused to obtain a fused edge response map.

[0110] In this embodiment, edge extraction processing is performed on each frame of the unified acquisition frame set, specifically on the RGB visible light image, depth map, and infrared thermal image. Edge extraction employs a detection method based on gradient magnitude and direction to ensure accurate capture of road edge features. The RGB image is processed based on a three-channel synthesized brightness map, while the depth map and infrared image are edge detected based on depth value gradients and temperature gradients, respectively. The edge results from each channel are fused with corresponding weights: RGB 0.5, depth map 0.3, and infrared thermal image 0.2, to accommodate differences in edge performance under different lighting and environmental conditions. The fused edge response map is saved as a binary image, highlighting the continuity and clarity of road boundaries.

[0111] Extract the set of main line segments from the fused edge response map, classify the set of main line segments according to their angular distribution, and select continuous line segments in the horizontal direction with a length exceeding 20% ​​of the image width as candidate road boundary line segments;

[0112] In this embodiment, a set of main line segments is extracted from the fused edge response map. Main line segments are formed by aggregating continuous edge pixels, with segment length in pixels and an image width standard of 1280 pixels. The angular distribution of all line segments relative to the horizontal axis is calculated, and angle classification is used to categorize the line segments into different angle intervals. Line segments with an angle deviation within ±10 degrees and a length exceeding 256 pixels (20% of the image width) are selected as candidate road boundary line segments. These line segments appear as horizontal and continuous edges in the image, representing potential road boundaries. This step effectively filters out noisy line segments and non-road edges.

[0113] Calculate the boundary symmetry of the left and right region line segment pairs in the candidate road boundary line segments, and retain the line segment pairs with a boundary symmetry greater than 0.85 to obtain the boundary line segment pairs;

[0114] In this embodiment, for candidate road boundary segments, left and right segment pairs are divided, and boundary symmetry is calculated. Boundary symmetry is an index of the similarity between corresponding segment pairs in the horizontal and vertical directions, specifically calculating the matching degree of the length, position, and direction vectors of the left and right segment pairs. Only segment pairs with a boundary symmetry greater than 0.85 are retained to ensure high symmetry and stability of the boundary pairs, thereby enhancing the reliability of the boundary segments. For example, for a set of candidate segment pairs, if the left segment is 512 pixels long, the corresponding right segment is 508 pixels long, and the positional deviation is less than 10 pixels, then the symmetry is high, meeting the retention criteria.

[0115] The boundary line segment pairs are reverse-mapped back to each view frame in the unified acquisition frame set. The boundary position reprojection error is evaluated on the mapping result. Boundary line segment pairs with projection errors less than 4 pixels in each view frame are retained to obtain the verified boundary line segment pairs.

[0116] In this embodiment, the verified boundary line segments are mapped back to image frames from various viewpoints within a unified set of acquired frames. The mapping utilizes the pose parameters and lens intrinsic parameters of the acquisition device from the device's auxiliary status data, transforming the boundary line segments to different viewpoint image coordinate systems through a 3D spatial projection model. For each viewpoint frame, the reprojection error between the mapped line segment and the actual image edge position is calculated. If the reprojection error in all viewpoint frames is less than 4 pixels, the boundary line segment pair is deemed to have passed verification, ensuring consistent boundary line segment performance across multiple viewpoints and enhancing spatial accuracy.

[0117] The verification boundary line segments are fitted with curves to obtain the fitted curve; equidistant interpolation sampling is performed on both sides of the fitted curve to obtain the road boundary point set.

[0118] In this embodiment, the validated boundary line segments are fitted to a curve on the two-dimensional image plane. A spline curve-based smoothing fitting method is used to handle local discontinuities and slight noise at the boundary points, resulting in a continuous and smooth road boundary curve. Along both sides of the fitted curve, equidistant interpolation sampling is performed at fixed intervals of 0.2 meters to generate a high-density road boundary point set. This point set contains the two-dimensional coordinates of each sampling point and its corresponding timestamp. This point set provides a precise foundation for subsequent road coordinate system establishment and target localization, ensuring the spatial accuracy of positioning and analysis.

[0119] Figure 3 This is a schematic diagram of the road coordinate system in step S2 of the present invention; as shown below. Figure 3 As shown, the road coordinate system can be divided into multiple key regions, including but not limited to: the left boundary point set 101, the road coordinate system 102, and the right boundary point set 103. The road coordinate system is a continuous set of coordinate systems; the left and right boundary point sets are named according to their spatial position relative to the road boundary line. Due to the different road conditions and viewing angles of each road, this invention does not restrict the naming of the boundary point sets.

[0120] The left boundary point set 101 is generated by spline fitting of the left boundary point set, which can effectively describe the structural contour of the left side of the road space and provide a spatial basis for boundary constraints and target classification.

[0121] The road coordinate system 102 is formed by extracting the center points and calculating the tangent and normal vectors of the corresponding positions of the left and right boundary curves. This centerline not only reflects the geometric principal axis direction of the road, but also serves as a reference for subsequent spatial location mapping and coordinate system establishment. The tangent vector (T) and normal vector (N) representing the road centerline are clearly marked with their directions at each center point. The tangent vector indicates the main direction of travel of the road, while the normal vector provides a directional reference for constructing a local rectangular coordinate system. Between each pair of continuous boundary curves, a standardized spatial coordinate system can be constructed based on this centerline, providing a structured geometric foundation for subsequent spatial location projection, target state recognition, and interactive analysis.

[0122] The right boundary point set 103 is generated by fitting the right boundary point set, and together with the left boundary curve, it constitutes the physical boundary framework of the road space.

[0123] Optionally, the detection of dynamic and static targets at various times in the unified acquisition frame set in step S2 includes:

[0124] The road boundary point set is divided into two boundary points, resulting in the left boundary point set and the right boundary point set;

[0125] In this embodiment, the road boundary point set is grouped according to its relative position in the road space, thus forming a left boundary point set and a right boundary point set. This operation can be performed by combining the edge change region in the depth map with the reflection features in the infrared imaging for spatial location filtering. The sampling interval of the boundary point set can be set to 0.2 meters to ensure that the accuracy of the boundary curve meets the requirements of spatial modeling.

[0126] Spline curve fitting is performed on the left boundary point set and the right boundary point set respectively to obtain the left continuous boundary curve and the right continuous boundary curve;

[0127] For each set of continuous boundary curves on the left and right sides, calculate the centerline points at the corresponding positions, and calculate the tangent vector and normal vector of the centerline points to construct the road coordinate system.

[0128] In this embodiment, the left and right boundary point sets are fitted separately. Using 20 consecutive boundary points on each side as a group, a continuous boundary curve is generated using cubic interpolation, denoted as the left continuous boundary curve and the right continuous boundary curve, respectively. This curve can effectively reflect the road structure trend and is used for subsequent centerline and coordinate system construction. Based on this, for any set of corresponding left and right boundary curves, the road centerline point sequence is constructed by calculating the equidistant center points between them. At each center point, the tangent direction is obtained by calculating the difference between adjacent points; the corresponding normal direction is obtained by using a vector perpendicular to the tangent direction, thus forming a road coordinate system with the centerline as the principal axis. This road coordinate system is a continuous set of local coordinate sub-units, covering the entire monitored road segment. Figure 3 In the diagram, the road coordinate system is numbered 102, the left boundary curve is numbered 101, and the right boundary curve is numbered 103.

[0129] The spatial positions of the bounding boxes at each time point in the unified collection of frames are projected onto the corresponding road coordinate system to obtain the spatial position data of the target to be analyzed.

[0130] In this embodiment, all target detection bounding boxes in the unified acquisition frame set are projected onto the centerline tangent plane according to the road coordinate system to obtain a standardized position representation of the target relative to the road space. This position representation includes not only the coordinates of the spatial projection point, but also the lateral distance and longitudinal trajectory segment number relative to the road centerline, thus facilitating subsequent target association. The error limit in the projection operation is set to a maximum offset of no more than 0.3 meters to ensure the stability and accuracy of the position mapping.

[0131] Cross-frame correlation is performed on the spatial location data of the target to be analyzed at each time point. Targets that appear in less than 3 frames are marked as misjudged targets and removed to obtain the target trajectory set.

[0132] In this embodiment, to eliminate occasional false positives, cross-frame consistency judgment is performed on target recognition results in consecutive frames. Using three frames as a sliding window, position matching is performed on target detection results near the same location. If a target appears less than twice in the three frames, it is judged as a false positive and removed. The remaining target bounding boxes will form trajectory segments, constituting a target trajectory set. The time interval between trajectory points is set to 0.5 seconds to ensure tracking continuity and maintain real-time performance.

[0133] Calculate the center point motion vector of each target in the target trajectory set between consecutive frames, and determine the target motion state based on the center point motion vector to obtain the motion state labeled target set;

[0134] The motion state marker target set is overlaid with the road coordinate system, and only targets within the boundary area are retained. The retained targets are divided into dynamic targets and static targets according to their motion state. The boundary area is the region enclosed by the continuous boundary curves on the left and right.

[0135] In this embodiment, the center point offset vector of each target trajectory between adjacent consecutive frames is extracted as the target's motion feature. Targets with a motion distance greater than 0.2 m / s are marked as dynamic targets, while those less than 0.2 m / s are treated as stationary targets. All target motion state marking results are mapped back to the road coordinate system and overlap analysis is performed with the closed region enclosed by the left and right boundary curves. Overlap analysis refers to determining the inclusion of each target's spatial position (represented as a center point) in the road coordinate system with the road boundary region composed of the left and right fitted curves. This road boundary region is constructed as a closed strip polygon based on the centerline and the lateral projection width of the two sides. The method for determining whether a target is in this region includes: extracting the target's lateral distance coordinate d and longitudinal line segment number s in the road coordinate system, and determining whether d is located between the left and right boundaries of the segment (the boundary point spacing is calculated at 0.2 meters, and the bandwidth error tolerance range is ±0.1 meters). Only targets that meet the condition of "center point falling into this region" are retained for subsequent interactive analysis. Only target trajectories within the area enclosed by the boundary curves are retained. Those classified as dynamic targets are categorized as dynamic, and those classified as stationary targets are categorized as static, for subsequent analysis. This area is the closed road segment space formed by numbers 101 and 103 in the diagram. Targets outside this area may be background noise or off-site targets and must be removed. This process aims to eliminate false detection results caused by background noise, reflection interference, or environmental changes in areas outside the road, ensuring that the analyzed targets are concentrated on functional roads such as fire lanes. The target set constrained by boundary overlap effectively improves the accuracy of dynamic / static target classification and provides reliable spatial structure support for subsequent operations such as constructing interaction pairs and predicting overlap time.

[0136] Optionally, step S3 includes:

[0137] Step S31: Set a sliding time window, segment the target trajectory corresponding to the dynamic target in the target trajectory set to obtain a short-time trajectory segment; calculate the motion vector sequence, velocity change rate, acceleration characteristics and direction offset angle of the short-time trajectory segment within the continuous time window to obtain the motion trajectory feature set;

[0138] In this embodiment, the trajectory data of dynamic targets in the target trajectory set are sorted by timestamp. A sliding time window length of 5 seconds and a window step size of 1 second are set. The trajectory data is divided into multiple short-time trajectory segments using continuous time intervals. For each short-time trajectory segment, the motion vector of the center point of each frame is calculated, specifically the difference vector between the position coordinates of two adjacent frames, in meters per second. Simultaneously, the velocity change rate and acceleration characteristics are calculated based on the motion vector. The velocity change rate is the percentage change in velocity relative to the velocity in consecutive frames, and the acceleration is calculated by the difference in the velocity change rates. Furthermore, the motion direction offset angle of each frame is statistically analyzed to quantify the minute changes in the target's direction of travel, in degrees. These motion parameters constitute the comprehensive motion trajectory feature set of the short-time trajectory segment. The data format is a time-series matrix, containing four dimensions of information: position, velocity, acceleration, and direction.

[0139] Step S32: Embed each trajectory behavior in the motion trajectory feature set to obtain a trajectory behavior embedding vector set;

[0140] In this embodiment, each short-term trajectory behavior in the aforementioned motion trajectory feature set is mapped to a fixed-dimensional behavior description space. This space employs a pre-trained trajectory behavior representation model, whose structure is based on a multi-layer temporal data encoding unit, capable of converting multi-dimensional time-series inputs into 128-length embedding vectors. The input includes the aforementioned motion feature matrix, which, through frame-by-frame encoding and global pooling, outputs the corresponding trajectory behavior embedding vectors. This embedding vector set contains the behavior descriptions of all short-term trajectories, stored as an N×128 matrix, where N is the number of trajectory segments. This representation helps to uniformly describe motion behaviors of different lengths and complexities.

[0141] Step S33: Remove outlier trajectory behaviors from the trajectory behavior embedding vector set, and use a predefined target category dictionary to label the remaining trajectory behaviors with category labels to obtain the category of the dynamic target;

[0142] In this embodiment, outlier behavior filtering is performed on the trajectory behavior embedding vector set. A threshold screening based on the similarity distribution between vectors is used to remove abnormal embeddings with a cosine similarity lower than 0.4 to most vectors; these removed objects typically correspond to abnormal or noisy trajectory segments. The remaining embedding vectors are matched against a predefined target category dictionary, which includes categories such as "pedestrian," "bicycle," and "motor vehicle" and their standard embedding samples. Nearest neighbor matching is used to assign corresponding category labels to the trajectory behaviors. The dynamic target category label sequence generated by this classification process assists in subsequent semantic annotation and analysis, ensuring accurate category division that conforms to actual traffic scenarios.

[0143] Step S34: Label each target category in the category of dynamic targets, and combine it with its timestamp and spatial coordinates projected in the road coordinate system to perform spatial node encoding, thereby obtaining a spatial node encoding set;

[0144] In this embodiment, after the dynamic target category is labeled, a spatial node code is generated for each target trajectory by combining the trajectory timestamp and the spatial coordinates projected onto the road coordinate system. The encoding format is a quadruple (target ID, timestamp, X coordinate, Y coordinate), where X and Y are two-dimensional coordinates in the road coordinate system, in meters. This spatial node code set is stored in tabular form, supporting spatiotemporal query and indexing. This data structure lays the foundation for the subsequent construction of a spatiotemporal semantic graph, ensuring accurate correspondence of spatiotemporal information of dynamic targets and supporting continuous semantic inference.

[0145] Step S35: Perform inter-frame semantic continuity filling on the time axis of the spatial node encoding set, and at the same time construct the motion boundaries of various targets on the spatial axis, thereby constructing a road semantic partition map;

[0146] Step S36: Project the road semantic partition map onto a standard raster map established in the road coordinate system to construct a road boundary semantic distribution map.

[0147] In this embodiment, for the spatial node encoding set, semantic continuity is first performed to fill in missing frames along the time axis. Linear interpolation is used to complete the spatial coordinates and category labels of missing time points to avoid semantic gaps. Along the spatial axis, motion boundaries are constructed based on the distribution of targets of the same category. The target motion range boundary is formed by calculating the minimum envelope curve between spatial nodes. By integrating the semantic relationship between time and space, a multi-level road semantic partition map is formed, with partitions refined to a grid size of 0.5 meters. The hierarchical structure reflects the dynamic target category and motion state. Finally, this semantic partition map is mapped onto a standard two-dimensional raster map (set to 500×500 pixels) established in the road coordinate system to achieve spatial visualization of the semantic information distribution, resulting in a road boundary semantic distribution map, which provides a spatial semantic foundation for subsequent target interaction analysis.

[0148] Optionally, step S4 includes:

[0149] Step S41: Obtain the two-dimensional coordinates of each semantic node in the road coordinate system based on the semantic distribution map of the road boundary, and bind the image frame and timestamp in the high-confidence frame sequence that is closest to the two-dimensional coordinate point to obtain the semantic image binding node set;

[0150] In this embodiment, the two-dimensional coordinates of all semantic nodes are extracted from the road boundary semantic distribution map. The coordinates are based on a pre-established road coordinate system, with units in meters. For each two-dimensional coordinate point, the high-confidence frame sequence is traversed to find the image frame whose timestamp and spatial location are closest to that node. Euclidean distance is used as the spatial matching standard, with the time difference limited to ±0.5 seconds. After a successful match, the node is bound to the corresponding image frame and timestamp, generating a semantic image-bound node set. This set is structured as a multi-field table, containing node ID, two-dimensional coordinates, image frame index, and timestamp, facilitating subsequent spatiotemporal correlation analysis and querying.

[0151] Step S42: Sort all nodes in the semantic image binding node set by timestamp and aggregate them into frame segments by sliding time window to obtain a dynamic distribution map of road boundaries;

[0152] In this embodiment, all nodes in the semantic image binding node set are first sorted in ascending order according to their timestamps to ensure temporal continuity. Then, a sliding time window of 3 seconds with a step size of 1 second is set, and the nodes are divided into multiple frame segments according to the time window. Each frame segment contains all node data within the time window range. This segmentation process can effectively aggregate spatially similar node information to generate a dynamic distribution map of road boundaries. This map uses frame segments as the time unit, and the spatial distribution is based on node positions, supporting subsequent temporal analysis of dynamic targets. When constructing the dynamic distribution map of road boundaries, in addition to aggregating the semantic nodes of dynamic targets according to the time window, the spatial location points of static targets within the same time period are simultaneously incorporated into the map as independent nodes. These static target nodes are also bound to corresponding timestamps and two-dimensional coordinates, forming a node set parallel to the dynamic nodes. This ensures that the dynamic distribution map becomes a unified spatiotemporal representation structure containing both types of targets.

[0153] Step S43: Calculate the nearest neighbor spatial distance between dynamic and static targets in the dynamic distribution map of road boundaries, predict the nearest neighbor spatial distance trend of adjacent frames, construct the interaction map between dynamic and static targets, and obtain the target interaction map.

[0154] In this embodiment, the nearest neighbor spatial distance between dynamic and static target nodes in the dynamic distribution map of road boundaries is calculated. This distance calculation uses a two-dimensional Euclidean distance metric, with a threshold set at 2 meters for potential interaction objects. By analyzing the spatial distance trend across consecutive time frames, the relative motion trend between targets is determined using the rate of change of distance between consecutive frames. Based on this data, an interaction graph between dynamic and static targets is constructed. The graph structure is a weighted network of edges between nodes, with weights reflecting the distance and motion trend between targets, forming a comprehensive target interaction map.

[0155] Step S44: Select static target nodes affected by dynamic targets from the target interaction graph, perform change tendency analysis, and obtain the static target motion probability set;

[0156] In this embodiment, static target nodes affected by dynamic targets are selected from the interactive graph, and a change tendency analysis is performed on these nodes. The change tendency analysis first collects continuous spatial position change data of the target nodes within a sliding time window, calculates the rate of distance change and directional consistency between the target and surrounding dynamic targets, and combines this with the target's own velocity and acceleration characteristics to comprehensively assess whether the target has a trend of moving from a stationary state. This analysis uses a weighted fusion method, taking the distance shortening trend, velocity increasing trend, and motion direction stability as the main indicators. Time series smoothing is used to reduce noise interference, and finally, a probability value reflecting the possibility of target motion is output. The evaluation process uses a sliding time window of 4 seconds to calculate the frequency and trend of target state changes within the time window, ensuring the timeliness and accuracy of the probability value.

[0157] Step S45: For static targets whose motion probability concentration changes more than 0.7, perform continuous time window confidence verification. If the change probability is higher than 0.7 for two consecutive time windows, then mark the static target as a motion trend target, thereby obtaining the change of the static target's motion state.

[0158] In this embodiment, continuous time window confidence verification is further implemented for static targets with a motion probability higher than 0.7. Both continuous time windows must meet the probability threshold requirements to ensure the continuity and stability of change prediction. This dual-window verification mechanism prevents misjudgments caused by occasional noise. Static targets meeting the criteria are marked as moving trend targets, resulting in the final output of static target motion state changes. This state labeling result guides subsequent dynamic interactive early warning and proactive prompting functions, ensuring the security system's accurate perception of potential moving static targets.

[0159] Optionally, the predicted target motion state overlap time points in step S5 include:

[0160] Based on a spatial threshold of 1.5m and a temporal overlap window of 3s, interactive pairs of candidate dynamic targets and static targets are constructed from the changes in the motion state of static targets and the dynamic distribution map of road boundaries, resulting in a set of candidate target pairs.

[0161] In this embodiment, a spatial threshold of 1.5 meters and a temporal overlap window length of 3 seconds are set. By filtering the changes in the motion state of static targets and the spatiotemporal positions of dynamic targets in the dynamic distribution map of road boundaries, dynamic-static target pairs that may interact are matched. Specifically, the operation includes traversing the two-dimensional coordinates and motion state information of static targets within a specified time period, and combining this with the spatial positions of dynamic targets within the same time window to construct a candidate target pair set. This set contains all target pairs that meet the spatial distance and temporal overlap conditions, ensuring the spatiotemporal consistency and accuracy of the interaction analysis.

[0162] Perform constant-rate-mutation prediction on the static targets in the candidate target pair set to obtain the static target state prediction trajectory set;

[0163] In this embodiment, for static targets in a candidate target pair, motion state prediction is performed based on their most recent motion trajectory data, combining constant velocity with abrupt change trends. This prediction process includes extracting the mean and standard deviation of velocity within a continuous time window from the static target's center position sequence to determine motion stability; under the premise of stable low-speed motion, linear extension is performed along the velocity direction; simultaneously, based on the probability of motion abrupt changes, alternative trajectory branches with ±15-degree directional perturbations and ±25% velocity variations are generated to ensure that the prediction results cover possible motion changes. The trajectory data structure uses timestamps as indexes and position coordinates as the primary index.

[0164] Multi-time-step trajectory extrapolation is performed on the dynamic targets in the candidate target set to obtain the dynamic target state prediction trajectory set;

[0165] In this embodiment, for dynamic targets in a candidate target pair, their trajectory data sequence is used, combined with the velocity, acceleration, and direction change characteristics within a continuous time window, and the motion states of up to five spatially nearest targets in the neighborhood, to perform multi-step time-point trajectory extension prediction. This multi-time-step prediction output contains a temporal vector of the future position of the dynamic target. The data structure is presented in time series form, supporting hourly position queries and updates, ensuring a fine-grained reflection of the dynamic target's motion trend.

[0166] Based on the static target state prediction trajectory set and the dynamic target state prediction trajectory set of each group, calculate the Euclidean distance between the two prediction trajectories at each moment, mark the set of all time points where the Euclidean distance is less than 0.8m, and obtain the set of prediction conflict time points.

[0167] In this embodiment, within the predicted trajectory time window of each target pair, for each second of sampling time, the two-dimensional Euclidean distance between the predicted trajectory points of the static target and the predicted trajectory points of the dynamic target is calculated. Time points with a distance less than 0.8 meters are included as potential prediction conflict time points. This distance threshold is set based on the actual target size and safety distance to ensure effective identification of risky moments of spatial proximity. All time point sets that meet the conditions are centrally managed to form a prediction conflict time point set.

[0168] The motion overlap risk is determined for the set of predicted conflict time points. If the predicted conflict time point between two predicted trajectories in this set exceeds 0.5s, it is determined to be a motion overlap risk time point, and the potential motion conflict time segment set is obtained.

[0169] In this embodiment, continuous time periods within the predicted conflict time point set are evaluated. If a predicted conflict point lasts for more than 0.5 seconds in any continuous time period, that time period is identified as a motion overlap risk period, and a potential motion conflict time segment set is constructed. The determination of time duration helps filter occasional short-term proximity events, enhancing the stability and reliability of the judgment.

[0170] The time point with the smallest spatial distance in the set of potential motion conflict time segments is taken as the predicted overlapping time point to obtain the overlapping time point.

[0171] In this embodiment, the time point with the smallest spatial distance is selected from the set of potential motion conflict time segments as the predicted target motion state overlap time point. This time point is considered the most critical conflict moment and is used for subsequent security warnings and intervention operations. This overlap time point is sent in real time to the communication device of the static target through the IoT communication channel to ensure timely voice prompts, assist the target in safely avoiding the conflict, and improve the active protection capability of the overall security system.

[0172] Of particular importance is that the constant-rate mutation prediction specifically includes:

[0173] Extract the sequence of center point positions from static targets in a continuous time window from the candidate target pair set;

[0174] In this embodiment, for each static target in the candidate target pair set, based on the standard two-dimensional plane in the road coordinate system, and using a sequence of high-confidence frames that have been synchronously calibrated in a unified acquisition frame set, the center position point sequence of the static target within the past 5 seconds (indexed by timestamp) is extracted, with a sampling interval of 0.5 seconds, thus forming a continuous trajectory segment. This position point is converted into ground two-dimensional coordinates (in meters) through the center of the spatial detection box in the RGB image, combined with depth map or LiDAR mapping calibration, ensuring the geometric accuracy and comparability of the trajectory.

[0175] Calculate the mean velocity and standard deviation of the velocity vector between two consecutive time windows in the center point position sequence. If the mean velocity is >0.1 m / s and the standard deviation is <0.05, the static target is considered to have a stable low-speed trend between the two time windows. Extract the time window pairs with stable low-speed trends to obtain the target velocity vector characteristics.

[0176] In this embodiment, the center point position sequence is divided into multiple adjacent time windows (window length is 1 second), such as seconds 1-2, seconds 2-3, seconds 3-4, etc. For each pair of time windows, a velocity vector is calculated from the previous window to the next. The velocity vector is obtained by dividing the coordinate difference between the terminal point and the starting point within the window by the window interval time (i.e., 1 second), and each velocity vector segment is recorded sequentially. For all consecutive velocity vector sequences, the mean (representing trend velocity) and standard deviation (representing stability) of the velocity magnitude are calculated segment by segment. When the mean velocity of a certain trajectory segment is greater than 0.1 meters per second and the standard deviation is less than 0.05 meters per second, it indicates that the target continues to move at a low and uniform speed during that period, and is determined to be a stable low-speed trend segment.

[0177] A constant-velocity predicted trajectory is obtained by performing constant-velocity linear extrapolation between the mean velocity value of the target velocity vector features and the current position in the candidate target pair set;

[0178] In this embodiment, the current frame is used as the reference time point. The last position point and velocity vector at the end of the most recent stable trend segment are selected as initial conditions for constant-speed linear extrapolation. During constant-speed linear extrapolation, the end position of the static target in the stable low-speed motion trend segment is used as the starting point. Combined with the average velocity vector calculated within that time period, the future motion path of the target is extrapolated forward at fixed time intervals. The prediction duration is typically set to 3 seconds, with a time interval of 0.5 seconds, meaning that six future position points are calculated sequentially. These position points form a straight-line trajectory based on the assumption that the current speed and direction remain unchanged. While generating the trajectory, it is also checked whether each predicted point is within a passable area defined in the road semantic map, avoiding paths falling into non-passable areas such as medians or guardrail zones. If the trajectory fully meets the passability requirements, the constant-speed trajectory is considered a valid predicted path and serves as the basis for subsequent risk assessment and disturbance trajectory construction. The prediction period is fixed at 3 seconds, and the sampling interval remains at 0.5 seconds, generating a sequence of predicted positions at six time points, called the constant-speed predicted trajectory.

[0179] The probability of abrupt changes in the constant velocity predicted trajectory is evaluated, and based on the constant velocity predicted trajectory, a perturbation trajectory branch is generated by integrating the directional perturbation angle ±15° and the velocity perturbation amplitude ±25%.

[0180] In this embodiment, the behavioral characteristics of a static target, including the magnitude of acceleration changes, frequency of directional fluctuations, and number of trajectory curvature abrupt changes over the past 5 seconds, are used to assess the probability of abrupt changes through a weighted decision rule. This probability of abrupt changes is a continuous value (0–1), with a high value indicating that the static target is more likely to experience drastic changes in its direction or velocity in the future. In the perturbation construction, perturbation parameters are introduced in both velocity and direction dimensions: based on the current average velocity, a perturbation amplitude of ±25% is set, modifying the velocity to 0.75 times and 1.25 times the original velocity; in terms of direction, a deflection of ±15 degrees is added to the original velocity vector direction, forming multiple directional perturbation paths. This results in 9 perturbation branch trajectories (including 1 main path and 8 perturbation paths). Each trajectory is sampled every 0.5 seconds to form 6 nodes, ensuring temporal consistency. All perturbation trajectories have a path type label field, such as "main trajectory," "positive deviation + fast," or "negative deviation - slow," to facilitate path source identification in subsequent conflict analysis.

[0181] If the mutation probability is <0.7, then the linear trajectory in the constant velocity prediction trajectory is output as the static target state prediction trajectory set;

[0182] If the mutation probability is ≥0.7, the perturbation trajectory branch of the main branch is output as the static target state prediction trajectory set.

[0183] In this embodiment, if the mutation probability is less than 0.7, the target behavior is considered stable, and the constant velocity predicted trajectory can be directly used as the final predicted trajectory set. However, if the mutation probability is not less than 0.7, the target may exhibit uncertain behavior, and trajectory perturbation needs to be introduced. The output path set is selected based on the mutation probability: if stable, a linear trajectory from one constant velocity predicted trajectory is output; if unstable, a complete set of main branch paths is output. This predicted trajectory set will be used to perform time-point spatial proximity analysis with the trajectory set of dynamic targets, identify potential conflict intervals, and support accurate estimation of overlapping motion state time points. The entire process, through parameter-controllable perturbation modeling, clear velocity / direction rules, and thresholds for mutation conditions, constitutes a technically stable, structurally clear, and scene-adaptive static target trajectory extrapolation process.

[0184] Most importantly, the multi-timestep trajectory extrapolation is specifically as follows:

[0185] The spatial center point sequence is extracted from the dynamic target trajectory set, and the velocity vector, direction angle and acceleration information of the continuous time window are extracted to obtain the trajectory feature vector sequence.

[0186] In this embodiment, a continuous spatial center point sequence for each target within the current time window is extracted from the generated dynamic target trajectory set. The time interval corresponding to this sequence is typically set to 0.5 seconds, with a total duration of 4 seconds, thus forming a short trajectory segment of 8 frames. For each frame, its corresponding velocity vector, motion direction angle (based on true north, ranging from 0 to 360°), and linear acceleration values ​​calculated from the preceding and following frames are extracted synchronously, ultimately forming a temporal information sequence, namely, a trajectory feature vector sequence with dimensions of N×D, where N is the number of frames and D is the motion feature dimension of each frame.

[0187] Deep temporal encoding is performed on the trajectory feature vector sequence to output a dynamic target global temporal feature vector;

[0188] In this embodiment, after constructing the trajectory feature vector sequence, a set of temporal feature extraction units are used to encode time-related information. This extraction process is implemented through a specific structure, such as a gated structure or nested convolutional channels, to extract features that reflect the continuity, rate of change, and acceleration periodicity of the overall motion trend of the target. The final extracted temporal encoding result is output as a fixed-length vector, serving as the global temporal feature vector of the target at the current stage.

[0189] Extract the spatial neighbor set of the dynamic target in the global temporal feature vector of the dynamic target at the current moment in the dynamic target trajectory set, and extract the trajectory feature vector of the spatial neighbor set to obtain the neighbor trajectory feature vector;

[0190] In this embodiment, at the current moment, the spatial neighbor set of the target is retrieved from the dynamic target set. The selection principle of the neighbor set is to perform a spatial search in the road coordinate system with a radius of 2 meters, retaining a maximum of the 5 nearest valid neighbor targets. For these neighbor targets, their trajectory feature vector sequences within the current time window are also extracted and sorted from nearest to farthest in spatial distance. The sorted neighbor trajectory feature vectors are then concatenated and fused with the target's own global temporal feature vector to form a joint feature description, used to encode the target's behavioral context in the local environment.

[0191] By concatenating the global temporal feature vector of the dynamic target with the feature vectors of its neighboring trajectories, and progressively predicting the position of the target within future time windows, a set of predicted trajectories for the dynamic target's state is obtained.

[0192] In this embodiment, a step-by-step prediction operation is performed based on a future prediction time window set for the current scene (usually 2 seconds, with a total of 4 prediction time points, every 0.5 seconds). This process inputs joint features and historical motion states into a temporal position update module, calculating the target's future position point by point, and ensuring the consistency and physical plausibility of each prediction result. The final output is a set of continuous two-dimensional coordinate points constituting the dynamic target's trajectory within the prediction time window, which is the dynamic target state prediction trajectory set. This trajectory set not only reflects the target's potential movement trend but also includes its adaptive response in a local dynamic environment with a large population, providing accurate input for subsequent spatial overlap analysis.

[0193] Optionally, this application also provides a security visualization display system for performing the security visualization display method described above, the security visualization display system comprising:

[0194] The data stream compression module is used to acquire and compress the data stream collected by the security equipment to obtain a unified set of acquisition frames.

[0195] The target classification module is used to identify the road boundary point set in the unified acquisition frame set, and to detect dynamic and static targets at each moment in the unified acquisition frame set, using the road boundary point set as the main axis of the road coordinate system.

[0196] The dynamic target category differentiation module is used to differentiate the categories of dynamic targets based on the changes in the motion state of dynamic targets at various times in the unified collection of frames, and to perform semantic annotation on the categories of dynamic targets to obtain a semantic distribution map of road boundaries.

[0197] The static target motion prediction module is used to bind corresponding image slices and timestamps to each node in the road boundary semantic distribution map to obtain the road boundary dynamic distribution map; based on the road boundary dynamic distribution map, it analyzes the interaction between dynamic and static targets, and identifies and predicts the changes in the motion state of static targets.

[0198] The target collision warning module is used to predict the overlap time point of the target motion state based on the changes in the motion state of static targets and the changes in the motion state of dynamic targets, and transmits the overlap time point to the communication device of the static target through the Internet of Things to perform voice prompt tasks.

[0199] Optionally, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the security visualization display method described above.

[0200] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0201] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A method for visualizing security applications, characterized in that, Includes the following steps: Step S1: Acquire the data stream collected by the security device and compress the data stream to obtain a unified collection frame set; Step S2: Identify the road boundary point set in the unified acquisition frame set, and use the road boundary point set as the principal axis of the road coordinate system. Project all target detection bounding boxes in the unified acquisition frame set onto the centerline tangent plane according to the road coordinate system to obtain the spatial position of the target relative to the road space. The spatial position of the target relative to the road space at each time step is correlated across frames to determine the target's motion state, and the target is divided into dynamic targets and static targets based on the motion state. Step S3: Based on the changes in the motion state of dynamic targets at various times in the unified collection of frames, distinguish the categories of dynamic targets, and perform semantic annotation on the categories of dynamic targets to obtain a semantic distribution map of road boundaries; Step S3 includes: Step S31: Set a sliding time window, divide the target trajectory corresponding to the dynamic target in the target trajectory set, and obtain short-time trajectory segments; Calculate the motion vector sequence, velocity change rate, acceleration characteristics, and directional offset angle of the short-time trajectory segment within a continuous time window to obtain the motion trajectory feature set; Step S32: Embed each trajectory behavior in the motion trajectory feature set to obtain a trajectory behavior embedding vector set; Step S33: Remove outlier trajectory behaviors from the trajectory behavior embedding vector set, and use a predefined target category dictionary to label the remaining trajectory behaviors with category labels to obtain the category of the dynamic target; Step S34: Label each target category in the category of dynamic targets, and combine it with its timestamp and spatial coordinates projected in the road coordinate system to perform spatial node encoding, thereby obtaining a spatial node encoding set; Step S35: Perform inter-frame semantic continuity filling on the time axis of the spatial node encoding set, and at the same time construct the motion boundaries of various targets on the spatial axis, thereby constructing a road semantic partition map; Step S36: Project the road semantic zoning map onto a standard raster map established in the road coordinate system to construct a road boundary semantic distribution map; Step S4: Bind the corresponding image slices and timestamps to each node in the road boundary semantic distribution map to obtain the road boundary dynamic distribution map; analyze the interaction between dynamic and static targets based on the road boundary dynamic distribution map, and identify and predict the changes in the motion state of static targets; Step S5: Based on the changes in the motion state of the static target and the changes in the motion state of the dynamic target, predict the overlap time point of the target's motion state, and transmit the overlap time point to the communication device of the static target through the Internet of Things to perform the voice prompt task.

2. The security visualization display method according to claim 1, characterized in that, Step S1 includes: Step S11: Obtain the security equipment data stream and auxiliary status data through the security equipment deployed in the security area, wherein the security equipment data stream includes RGB visible light image sequence, depth map sequence and infrared thermal imaging data; Step S12: Perform device timestamp alignment processing on the data stream collected by the security equipment, and perform frame reconstruction interpolation on the timestamp alignment processing result based on the camera time series in the device auxiliary status data to obtain a synchronized image frame sequence; Step S13: Based on the pose parameters of each acquisition device and the lens intrinsic parameters in the auxiliary status data of the equipment, perform spatial viewpoint transformation processing on the synchronized image frame sequence to generate a viewpoint normalized image set. Step S14: Perform inter-frame difference feature extraction on the viewpoint normalized image set, remove redundant frames without target motion changes, and effectively sample and compress the remaining frames to generate a high-confidence frame sequence. Step S15: The RGB visible light image, depth map, infrared thermal image and timestamp in the high confidence frame sequence are stitched and encapsulated to obtain a unified acquisition frame set.

3. The security visualization display method according to claim 2, characterized in that, Step S14 includes: Step S141: Perform two-frame sliding pairing of the viewpoint normalized image set in chronological order, and perform pixel difference calculation on each frame pair of the sliding pairing result to obtain the inter-frame difference map. Step S142: Perform spatial connected component extraction on the inter-frame difference map, remove regions with a connected component area of ​​less than 20 pixels, and construct an inter-frame motion activation mask based on the maximum bounding box size and density distribution of the remaining regions. Step S143: Calculate the proportion of activated pixels in the inter-frame motion activation mask of each frame. If the proportion of activated pixels is less than 2% of the preset proportion threshold, the frame is marked as a redundant frame and discarded; the remaining frames are retained as a candidate frame set. Step S144: Set the time window to 2s, perform density distribution analysis on the frame time series in the candidate frame set, and select the frame with the largest average motion activation area from the density distribution analysis results in each time window as the representative frame to obtain the representative frame set. Step S145: Perform a confidence assessment on the representative frame set, and retain the frames with a confidence assessment result greater than 0.75 to obtain a high-confidence frame sequence.

4. The security visualization display method according to claim 1, characterized in that, Step S2 involves identifying the set of road boundary points within the unified acquisition frame set, including: Edge detection is performed on the RGB visible light image, depth map, and infrared thermal image corresponding to each frame in the unified acquisition frame set, and the edge detection results are weighted and fused to obtain a fused edge response map. Extract the set of main line segments from the fused edge response map, classify the set of main line segments according to their angular distribution, and select continuous line segments in the horizontal direction with a length exceeding 20% ​​of the image width as candidate road boundary line segments; Calculate the boundary symmetry of the left and right region line segment pairs in the candidate road boundary line segments, and retain the line segment pairs with a boundary symmetry greater than 0.85 to obtain the boundary line segment pairs; The boundary line segment pairs are reverse-mapped back to the frames of each view in the unified acquisition frame set. The boundary position reprojection error is evaluated on the mapping result. The boundary line segment pairs with projection errors of less than 4 pixels in each view frame are retained to obtain the verified boundary line segment pairs. The verification boundary line segments are fitted with curves to obtain the fitted curve; equidistant interpolation sampling is performed on both sides of the fitted curve to obtain the road boundary point set.

5. The security visualization display method according to claim 1, characterized in that, Step S2 involves detecting dynamic and static targets at various times within the unified acquisition frame set, including: The road boundary point set is divided into two boundary points, resulting in the left boundary point set and the right boundary point set; Spline curve fitting is performed on the left boundary point set and the right boundary point set respectively to obtain the left continuous boundary curve and the right continuous boundary curve; For each set of continuous boundary curves on the left and right sides, calculate the centerline points at the corresponding positions, and calculate the tangent vector and normal vector of the centerline points to construct the road coordinate system. The spatial positions of the bounding boxes at each time point in the unified collection of frames are projected onto the corresponding road coordinate system to obtain the spatial position data of the target to be analyzed. Cross-frame correlation is performed on the spatial location data of the target to be analyzed at each time point. Targets that appear in less than 3 frames are marked as misjudged targets and removed to obtain the target trajectory set. Calculate the center point motion vector of each target in the target trajectory set between consecutive frames, and determine the target motion state based on the center point motion vector to obtain the motion state labeled target set; The motion state marker target set is overlaid with the road coordinate system, and only targets within the boundary area are retained. The retained targets are divided into dynamic targets and static targets according to their motion state. The boundary area is the region enclosed by the continuous boundary curves on the left and right.

6. The security visualization display method according to claim 1, characterized in that, Step S4 includes: Step S41: Obtain the two-dimensional coordinates of each semantic node in the road coordinate system based on the semantic distribution map of the road boundary, and bind the image frame and timestamp in the high-confidence frame sequence that is closest to the two-dimensional coordinate point to obtain the semantic image binding node set; Step S42: Sort all nodes in the semantic image binding node set by timestamp and aggregate them into frame segments by sliding time window to obtain a dynamic distribution map of road boundaries; Step S43: Calculate the nearest neighbor spatial distance between dynamic and static targets in the dynamic distribution map of road boundaries, predict the nearest neighbor spatial distance trend of adjacent frames, construct the interaction map between dynamic and static targets, and obtain the target interaction map. Step S44: Select static target nodes affected by dynamic targets from the target interaction graph, perform change tendency analysis, and obtain the static target motion probability set; Step S45: For static targets whose motion probability concentration changes more than 0.7, perform continuous time window confidence verification. If the change probability is higher than 0.7 for two consecutive time windows, then mark the static target as a motion trend target, thereby obtaining the change of the static target's motion state.

7. The security visualization display method according to claim 1, characterized in that, Step S5 includes predicting the overlapping time points of the target motion state: Based on a spatial threshold of 1.5m and a temporal overlap window of 3s, interactive pairs of candidate dynamic targets and static targets are constructed from the changes in the motion state of static targets and the dynamic distribution map of road boundaries, resulting in a set of candidate target pairs. Perform constant-rate-mutation prediction on the static targets in the candidate target pair set to obtain the static target state prediction trajectory set; Multi-time-step trajectory extrapolation is performed on the dynamic targets in the candidate target set to obtain the dynamic target state prediction trajectory set; Based on the static target state prediction trajectory set and the dynamic target state prediction trajectory set of each group, calculate the Euclidean distance between the two prediction trajectories at each moment, mark the set of all time points where the Euclidean distance is less than 0.8m, and obtain the set of prediction conflict time points. The motion overlap risk is determined for the set of predicted conflict time points. If the predicted conflict time point between two predicted trajectories in this set exceeds 0.5s, it is determined to be a motion overlap risk time point, and the potential motion conflict time segment set is obtained. The time point with the smallest spatial distance in the set of potential motion conflict time segments is taken as the predicted overlapping time point to obtain the overlapping time point.

8. A security visualization display system, characterized in that, For performing the security visualization display method as described in claim 1, the security visualization display system includes: The data stream compression module is used to acquire and compress the data stream collected by the security equipment to obtain a unified set of acquisition frames. The target classification module is used to identify the road boundary point set in the unified acquisition frame set, and to detect dynamic and static targets at each moment in the unified acquisition frame set, using the road boundary point set as the main axis of the road coordinate system. The dynamic target category differentiation module is used to differentiate the categories of dynamic targets based on the changes in the motion state of dynamic targets at various times in the unified collection of frames, and to perform semantic annotation on the categories of dynamic targets to obtain a semantic distribution map of road boundaries. The static target motion prediction module is used to bind corresponding image slices and timestamps to each node in the road boundary semantic distribution map to obtain the road boundary dynamic distribution map; based on the road boundary dynamic distribution map, it analyzes the interaction between dynamic and static targets, and identifies and predicts the changes in the motion state of static targets. The target collision warning module is used to predict the overlap time point of the target motion state based on the changes in the motion state of static targets and the changes in the motion state of dynamic targets, and transmits the overlap time point to the communication device of the static target through the Internet of Things to perform voice prompt tasks.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the security visualization display method as described in any one of claims 1 to 7.

Images