Intelligent security video monitoring system based on internet of things

Abstract

The application relates to the technical field of intelligent security and protection video monitoring, and discloses an intelligent security and protection video monitoring system based on the Internet of Things, wherein a cross-camera pre-activation tracking system comprises an event collection module, a motion estimation module, a trajectory prediction module, a camera pre-activation module and a cross-camera association module. The event collection module obtains trigger event data from the Internet of Things sensor nodes and generates a space-time event point sequence; the motion estimation module performs weighted least square linear motion fitting on the sequence and generates an event propagation speed vector; the trajectory prediction module performs time extrapolation based on the vector and generates a camera predicted arrival list; the camera pre-activation module sends a pre-activation instruction to a target camera according to the list; and the cross-camera association module generates a cross-camera continuous tracking trajectory through visual appearance feature matching.

Description

Technical Field

[0001] This invention relates to the field of smart security video surveillance technology, and more specifically, to a smart security video surveillance system based on the Internet of Things. Background Technology

[0002] With the development of the Internet of Things (IoT), network cameras, and intelligent video analytics technologies, smart security video surveillance systems have been widely applied in industrial parks, warehousing and logistics areas, public buildings, campuses, communities, and key perimeter protection scenarios. Existing systems typically collect environmental information using devices such as fixed cameras, dome cameras, infrared detectors, access control sensors, and acoustic microphones, with the video feed, alarm events, and target trajectories centrally processed by a security management platform. For target detection and tracking within the field of view of a single camera, existing technologies can already achieve basic identification through target detection algorithms, motion detection algorithms, or manually configured alarm rules.

[0003] However, in real-world security scenarios, targets often do not remain within the field of view of a single camera, but rather move continuously along roads, corridors, fences, entrances / exits, or warehouse passageways, passing through the coverage areas of multiple cameras in sequence. To achieve continuous monitoring or post-event traceability, security systems need to promptly switch to the next camera after a target leaves the current camera's field of view, and associate targets appearing in the previous and subsequent cameras as the same security event. However, existing multi-camera video surveillance systems largely rely on pre-configured camera adjacency relationships, fixed patrol paths, or manual switching methods. The switching sequence between cameras is usually static, making it difficult to dynamically adjust based on the target's actual movement direction, speed, and the order in which sensor nodes are triggered.

[0004] In park, warehouse, or perimeter security systems, before a target enters the camera's field of view, it often triggers nearby passive infrared sensors, infrared beam sensors, acoustic sensors, or other IoT sensing nodes. These triggering events actually contain spatiotemporal information such as the target's location, trigger time, and signal strength, reflecting the target's movement trend. However, existing systems typically use these sensing nodes only as alarm trigger sources, that is, only notifying security personnel to view the footage after the sensor is triggered, or simply linking nearby cameras to record. They fail to perform spatiotemporal sequence analysis on continuous triggering events, nor do they use triggering events to predict the camera coverage area that the target will enter. Therefore, when the target moves along a non-preset path, or moves in the opposite direction, turns back, or crosses diagonally, the fixed adjacency switching rules are prone to selecting the wrong camera, causing tracking blind spots during the target's cross-camera switching process.

[0005] Furthermore, network dome cameras typically need to perform actions such as pan-tilt rotation, focus adjustment, exposure parameter configuration, frame rate boosting, and resolution switching when performing cross-region tracking. If the system issues the linkage command only after the target has already entered the next camera's field of view, the camera will experience a certain response delay when switching from low-power or normal acquisition mode to high frame rate and high-resolution mode. Within this response delay, the target may have already crossed the edge of the field of view or entered an obstructed area, causing the crucial footage of the target just entering the new field of view to be missed, affecting the real-time tracking effect and reducing the completeness of trajectory reconstruction during subsequent evidence collection.

[0006] Therefore, it is necessary to propose a smart security video surveillance system that performs motion estimation and trajectory prediction based on IoT-triggered events, and completes camera pre-activation and cross-camera association before the target arrives, in order to improve the integrity and real-time performance of cross-camera continuous tracking. Summary of the Invention

[0007] This invention provides an IoT-based smart security video surveillance system that solves the technical problems in related technologies, such as blind spots in tracking when the target switches between cameras due to passive response of security cameras, and the inability to associate the detection results of multiple cameras with a unified event identifier.

[0008] This invention discloses an intelligent security video surveillance system based on the Internet of Things, comprising:

[0009] The event acquisition module is configured to acquire trigger event data from IoT sensor nodes in real time and generate a spatiotemporal event point sequence by arranging the trigger timestamps in ascending order.

[0010] The motion estimation module is configured to perform weighted least squares linear motion fitting on the spatiotemporal event point sequence, using a function that decreases inversely proportionally to the vertical distance from each event point to the fitted trajectory line as weights, and solve to generate an event propagation velocity vector.

[0011] The trajectory prediction module is configured to perform linear time extrapolation of the motion trajectory based on the event propagation speed vector, perform geometric intersection calculation between the predicted location trajectory line segment and the spatial topology map of the security camera coverage area, and generate a list of predicted camera arrivals.

[0012] The camera pre-activation module is configured to send a pre-activation command to the target camera before the expected arrival time window to complete the pan-tilt angle adjustment and acquisition parameter configuration.

[0013] The cross-camera association module is configured to associate multiple detection results with a unified security event identifier through cosine similarity matching of visual appearance feature descriptors, thereby generating a continuous tracking trajectory across cameras.

[0014] Furthermore, each event record in the trigger event data includes the spatial coordinates of the sensor node, the trigger timestamp, and the signal strength value; after generating the spatiotemporal event point sequence, the event acquisition module performs Z-score standardization on the spatial coordinate components and timestamp of each event point to eliminate the dimensional differences between the spatial coordinates and the timestamp.

[0015] Furthermore, in the weighted least squares linear motion fitting, the first... Weight of each event point Defined as: ,in For the first The vertical distance from each event point to the currently fitted trajectory line. To prevent small positive constants with a denominator of zero, the fitted trajectory in the initial iteration is taken as a straight line segment connecting the first and last event points of the spatiotemporal event point sequence. In each subsequent iteration, the weight of each event point is recalculated using the fitting result of the previous iteration as a reference trajectory.

[0016] Furthermore, the event propagation velocity vector includes a propagation velocity scalar and a heading angle parameter. The propagation velocity scalar is the magnitude of the displacement rate components of the security event target along the horizontal and vertical directions, and the heading angle parameter is the arctangent value of the direction of the displacement rate components. Before trajectory extrapolation, the propagation velocity scalar is restored to its original physical units by inverse Z-score transformation.

[0017] Furthermore, in the spatial topology diagram of the security camera coverage area, the effective monitoring field of view of each camera is represented by a coverage polygon, and the vertex coordinates of the coverage polygon are calculated based on the camera installation position, focal length, and field of view angle; the geometric intersection calculation is to determine the intersection point coordinates by sequentially intersecting the predicted position trajectory line segment with the boundary of each coverage polygon.

[0018] Furthermore, the camera predicted arrival list is generated as follows: the predicted arrival time of the target at the boundary of each camera's coverage area is calculated by dividing the distance between the coordinates of each trajectory intersection point and the starting extrapolation point by the propagation speed scalar, and then arranged in ascending order of the predicted arrival time; each item in the camera predicted arrival list includes the camera identifier, the corresponding trajectory intersection point coordinates, and the expected arrival time window.

[0019] Furthermore, the camera pre-activation module calculates the azimuth and pitch parameters of the pan-tilt unit based on the spatial geometric relationship between the coordinates of the trajectory intersection point and the installation position of the target camera; the pre-activation command carries the azimuth, pitch, maximum acquisition frame rate and maximum resolution parameters of the pan-tilt unit, and after receiving the pre-activation command, the target camera completes the pan-tilt rotation and acquisition parameter switching to enter the pre-configuration ready state.

[0020] Furthermore, the visual appearance feature descriptor is a fixed-dimensional vector representation, obtained by performing convolutional neural network feature encoding on the target region image; the training phase of the convolutional neural network uses a triplet loss function as the objective and employs the Adam optimization algorithm to optimize the network parameters, so as to minimize the distance between the visual appearance feature descriptors of the same target under different camera views and maximize the distance between the visual appearance feature descriptors of different targets.

[0021] Furthermore, when the cosine similarity value exceeds a preset matching threshold, the cross-camera association module determines that the two detection results correspond to the same security event target, associates the detection result in the current camera with the unified security event identifier to which the source camera detection result belongs, and concatenates the target position sequence under the unified security event identifier in the source camera with the target position sequence in the current camera according to the timestamp order to generate the cross-camera continuous tracking trajectory.

[0022] Compared with the prior art, the present invention has the following beneficial effects.

[0023] The motion estimation module uses weighted least squares fitting with inverse distance weights to suppress the influence of abnormal trigger points deviating from the main trajectory on the fitting results. The resulting event propagation velocity vector can reflect the actual motion direction and speed of the target, thus avoiding camera selection errors caused by the static switching rule's inability to perceive the motion direction.

[0024] The trajectory prediction module performs time extrapolation along the heading direction based on the event propagation velocity vector and performs geometric intersection calculation with the spatial topology map of the security camera coverage area. This enables the security platform to determine the next camera the target will enter and the expected arrival time window before the target arrives, providing a time basis for subsequent pre-activation operations.

[0025] The camera pre-activation module completes the pan-tilt-zoom (PTZ) orientation adjustment and high frame rate acquisition parameter configuration before the expected arrival time window, moving the camera's response action forward to before the target arrives. This ensures that the target is in a high frame rate and high resolution acquisition state from the initial moment it enters the new field of view, eliminating the omission of key images caused by response delay in passive response mode.

[0026] The cross-camera association module uses cosine similarity matching of visual appearance feature descriptors to associate the detection results of front and rear cameras with a unified security event identifier, and splices the target location sequence according to the timestamp order to form a spatiotemporally continuous cross-camera continuous tracking trajectory on the security platform side, which meets the requirement of trajectory integrity. Attached Figure Description

[0027] Figure 1 This is a flowchart of an IoT-based smart security video surveillance system provided in an embodiment of the present invention. Detailed Implementation

[0028] Example 1

[0029] In intelligent security video surveillance systems, multiple network cameras and IoT sensor nodes collaborate to cover various monitoring zones within a park or public area. When a security target moves along a path in physical space and sequentially triggers sensor nodes along the way, the security management platform needs to switch the monitoring view from the current camera to the next camera the target is about to enter in order to achieve video relay tracking. Existing systems, with their static switching rules based on fixed spatial adjacency relationships, cannot perceive the target's actual direction and speed of movement, resulting in a mismatch between the camera response sequence and the target's actual movement path. Furthermore, adjacent cameras only initiate high-frame-rate acquisition and PTZ parameter adjustments after receiving the switching command. This response delay causes crucial footage of the target entering a new field of view to be missed, ultimately preventing the security platform from forming a continuous, spatiotemporally consistent cross-camera tracking trajectory.

[0030] See Figure 1 As shown, this embodiment provides an IoT-based smart security video surveillance system. The cross-camera pre-activation tracking system includes an event acquisition module, a motion estimation module, a trajectory prediction module, a camera pre-activation module, and a cross-camera association module.

[0031] The event acquisition module is configured to acquire trigger event data from IoT sensor nodes in real time and generate a spatiotemporal event point sequence.

[0032] The event acquisition module obtains trigger event data in real time from spatially distributed IoT sensor nodes within the coverage area of ​​the smart security system. These sensor nodes include passive infrared motion detectors, infrared beam sensors, and acoustic pickups. Each event record contains three fields: the spatial coordinates of the sensor node (a two-dimensional position represented in a Cartesian coordinate system), the trigger timestamp, and the signal strength value. The event acquisition module sorts all event records in ascending order of trigger timestamp, generating a spatiotemporal event point sequence. Each event point in this sequence represents the trigger response of a security event target to a sensor node at a specific time and spatial location.

[0033] Before generating the spatiotemporal event point sequence, the trigger event data is first used to determine the attribution to the same security event.

[0034] For any two trigger event records Ea and Eb, if the time difference between their triggers satisfies |ta−tb|≤Tgate, the spatial distance satisfies ||Pa−Pb||≤Dgate, and the angle between their trigger directions and the formed event trajectory does not exceed [the specified value], then [the following conditions apply]. If both are true, they will be grouped into the same candidate security event set. The Tgate can be set to 3-10 seconds, and the Dgate can be set to 1.5 times the average distance between adjacent sensor nodes. The angle can be 45° to 60°. When the number of event points in the candidate security event set is not less than 3, and the signal strength of at least two event points is greater than the signal strength threshold, a valid spatiotemporal event point sequence is generated; otherwise, the candidate set is marked as an event to be confirmed or a noise-triggered set.

[0035] Before performing motion fitting, the spatial coordinate components and timestamps of each event point in the spatiotemporal event point sequence are Z-score standardized to eliminate the influence of the dimensional difference between spatial coordinates (in meters) and timestamps (in seconds) on the weighted least squares fitting operation. The standardized spatial coordinates and timestamps are dimensionless values.

[0036] The motion estimation module is configured to perform weighted least-squares linear motion fitting on a sequence of spatiotemporal event points and generate an event propagation velocity vector.

[0037] The motion estimation module performs weighted least-squares linear motion fitting on the spatiotemporal event point sequence. The specific processing procedure is as follows:

[0038] Iterative weight calculation: For each event point in the spatiotemporal event point sequence, calculate the perpendicular distance from its spatial coordinates to the currently fitted trajectory line, and set the weight as an inversely decreasing function of this perpendicular distance. Let the th... The perpendicular distance from each event point to the currently fitted trajectory line is ,in For event point index, , Given the total number of event points in the spatiotemporal event point sequence, the weight of that event point is... Defined as:

[0039] ;

[0040] in For the first The weight of each event point For the first The vertical distance from each event point to the currently fitted trajectory line. To prevent small positive constants with a denominator of zero, the fitted trajectory is initially taken as a straight line segment connecting the first and last event points in the spatiotemporal event point sequence. In each subsequent iteration, the weights of each event point are recalculated using the previous iteration's fitting result as the reference trajectory. This weight assignment suppresses the contribution of abnormal trigger points that deviate from the main trajectory in space to the fitting result, while event points adjacent to the main trajectory gain a higher fitting influence.

[0041] Velocity vector solution: The linear relationship between the standardized spatial coordinates and standardized timestamps of each event point is estimated using the weighted least squares criterion. This yields the displacement velocity components of the security event target along the horizontal and vertical directions. The propagation velocity scalar (i.e., the magnitude of the velocity component) and the heading angle parameter (i.e., the arctangent value of the velocity component direction) are then calculated. Combining the propagation velocity scalar and the heading angle parameter generates the event propagation velocity vector.

[0042] It should be noted that the weighted least squares fitting solves for the linear parameters in the standardized coordinate space, and the resulting velocity components are dimensionless ratios in the standardized space. Before being used in the subsequent trajectory extrapolation step, they need to be restored to the original physical unit (m / s) propagation velocity scalar by the inverse Z-score transformation to ensure the dimensionality consistency of the extrapolation calculation in the physical coordinate system.

[0043] The calculation process of the weighted least squares objective function:

[0044] The motion estimation module uses standardized time. As the independent variable, using standardized spatial coordinates , For the dependent variable, establish the following linear relationship:

[0045] ;

[0046] ;

[0047] The weighted least squares objective function is:

[0048] arrive ;

[0049] Where ax and ay are the standardized velocity slope parameters, and bx and by are the fitting intercepts. Let be the weight of the j-th event point. After each iteration, recalculate the vertical distance dj from the event point to the fitted trajectory and update . , This is a preset minimum constant. When the change in v obtained from two consecutive iterations is less than... The iteration stops when the number of iterations reaches Nmax. A speed of 0.05 m / s can be used, and Nmax can be taken as 5-10 times.

[0050] The specific process of restoring the original physical unit by the inverse Z-score transform is as follows:

[0051] Let the original coordinates and time of the j-th event point be... The corresponding standardized value is:

[0052] ;

[0053] ;

[0054] ;

[0055] Weighted least squares fitting yields the slope parameters ax and ay in the standardized coordinate space, where ax represents Compared to The rate of change, ay represents Compared to The rate of change. Since the velocity component is the derivative of the spatial coordinates with respect to time, the velocity component in the original physical units is reconstructed by the following formula:

[0056] ;

[0057] ;

[0058] ;

[0059] ;

[0060] in, , , These represent the mean of the spatial x-coordinate, spatial y-coordinate, and timestamp, respectively. , , These are the corresponding standard deviations. and The unit is meters per second. For the propagation speed scalar, This is the heading angle.

[0061] The trajectory prediction module is configured to extrapolate the motion trajectory over time based on the event propagation velocity vector and generate a list of predicted camera arrivals.

[0062] The trajectory prediction module uses the event propagation velocity vector and the spatial coordinates of the latest event point in the spatiotemporal event point sequence as the starting extrapolation point. It performs linear time extrapolation of the motion trajectory along the heading angle, using the propagation velocity scalar as the step rate, to generate the predicted position trajectory segment for the future time period. The time extrapolation is performed in the original physical spatial coordinate system (units are meters and seconds). The spatial coordinates of the starting extrapolation point are restored to the original physical coordinates from standardized coordinates through an inverse transformation before use. Simultaneously, the propagation velocity scalar is restored to its original physical units through an inverse transformation before participating in the extrapolation calculation.

[0063] Intersection Calculation of Coverage Areas: The predicted location trajectory segments are geometrically intersected with a pre-stored spatial topology map of the security camera coverage areas. In this map, the effective monitoring field of view for each camera is represented by a coverage polygon. The vertex coordinates of these polygons are calculated during system initialization based on the camera's installation location, focal length, and field of view. The predicted location trajectory segments are sequentially intersected with the boundaries of each coverage polygon to determine the coordinates of the intersection point.

[0064] The camera coverage polygon is generated during system initialization or camera calibration. Let the ground projection coordinates of the k-th camera be... The horizontal orientation angle of the optical axis is The horizontal field of view is If the maximum effective recognition distance is Rk, then the effective coverage boundary of the camera on the ground plane can be composed of three or more points, and its left boundary endpoint is:

[0065] ;

[0066] The right boundary endpoint is:

[0067] ;

[0068] When an approximate fan-shaped field of view is required, to Multiple boundary points are sampled at fixed angular intervals, and Ck and each boundary point are connected in angular order to form a coverage polygon. If the camera is a PTZ camera, the coverage polygon is bound to the current preset position or pre-activated PTZ angle and is updated after the PTZ angle changes.

[0069] Arrival Time Sorting: Based on the distance between the coordinates of each trajectory intersection point and the starting extrapolation point, divided by the propagation speed scalar, the predicted arrival time of the target at the boundary of each camera's coverage area is calculated. These predicted arrival times are then sorted in ascending order to generate a list of predicted camera arrival times. Each item in this list includes the camera identifier, the corresponding trajectory intersection point coordinates, and the expected arrival time window.

[0070] The camera pre-activation module is configured to send a pre-activation command to the target camera based on the predicted arrival list and generate a pre-configured, ready-to-use set of camera parameters.

[0071] The camera pre-activation module extracts the top-ranked camera identifiers, their corresponding trajectory intersection point coordinates, and estimated arrival time windows from the predicted camera arrival list in sorted order, and performs the following parameter pre-configuration operations:

[0072] Gimbal angle calculation: Based on the spatial geometric relationship between the coordinates of the trajectory intersection point and the installation position of the target camera, calculate the azimuth and pitch angle parameters that the camera gimbal needs to be adjusted to, so that the optical axis of the camera points to the predicted target entry position.

[0073] Let the target camera installation point be... The intersection point of the trajectories is ,in The height of the plane containing the target can be taken as 0-1.7 meters. The horizontal distance is:

[0074] ;

[0075] The azimuth angle of the gimbal is: ;

[0076] The gimbal's pitch angle is: ; where atan2 is used to determine the angular quadrant based on the signs of the horizontal and vertical coordinates. A negative value indicates that the camera is looking downwards. After receiving the pre-activation command, the target camera adjusts the pan-tilt azimuth angle to... Adjust the pitch angle to And switch the capture frame rate and resolution to the corresponding levels;

[0077] Acquisition Parameter Configuration: Before the expected arrival time window, a pre-activation command is sent to the target camera. This command carries parameters such as the pan / tilt azimuth angle, tilt angle, maximum acquisition frame rate, and maximum resolution. Upon receiving the pre-activation command, the target camera completes pan / tilt rotation and acquisition parameter switching, entering a pre-configuration ready state. The camera pre-activation module summarizes the configured parameters of each target camera, generating a set of camera parameters in the pre-configuration ready state.

[0078] The cross-camera association module is configured to associate multiple detection results with a unified security event identifier and generate a cross-camera continuous tracking trajectory by matching visual appearance features.

[0079] When a security event target enters the field of view of a camera that is in a pre-configured ready state, the cross-camera association module performs the following processing:

[0080] Feature Extraction: Target regions are detected from the high frame rate video stream captured by the currently pre-activated camera, and visual appearance feature descriptors of the target are extracted. These visual appearance feature descriptors are fixed-dimensional vector representations, obtained by performing convolutional neural network feature encoding on the target region image. The input layer of the convolutional neural network receives cropped image patches of the target region, and after multiple convolution and pooling operations, the fully connected output layer outputs a fixed-dimensional visual appearance feature descriptor vector. During the training phase, the Adam optimization algorithm is used to optimize the network parameters using a triplet loss function, minimizing the distance between visual appearance feature descriptors of the same target from different camera perspectives and maximizing the distance between visual appearance feature descriptors of different targets.

[0081] Feature matching: Cosine similarity is calculated between the visual appearance feature descriptors extracted from the current camera and the visual appearance feature descriptors extracted and cached from the source camera (i.e., the camera before the target left). Both vectors involved in the calculation are visual appearance feature descriptor vectors of the same fixed dimension output from the fully connected output layer of a convolutional neural network. When the cosine similarity value exceeds a preset matching threshold, it is determined that the detection results from both ends correspond to the same security event target, and the detection result from the current camera is associated with the unified security event identifier belonging to the detection result from the source camera.

[0082] The input to the visual appearance feature encoding network is an image patch cropped and scaled from the target detection box. The image patch size can be set to 128×256 pixels or 224×224 pixels.

[0083] The network consists of several convolutional layers, normalization layers, activation layers, and global pooling layers, finally outputting a 512-dimensional visual appearance feature descriptor f through a fully connected layer. During training, images of the same target from different cameras are used as positive sample pairs, and images of different targets are used as negative sample pairs, trained according to a triplet loss function.

[0084] ;

[0085] Where fa represents the anchor sample feature. Positive sample features for the same target The negative sample features for different targets are represented by m, which is the interval parameter and can be between 0.2 and 0.5. The matching threshold Sth is determined by annotating the historical cross-camera sample set, with a preferred range of 0.75 to 0.90; in this example, Sth is set to 0.80.

[0086] It is also recommended to add spacetime gating:

[0087] The cross-camera association module does not only determine target consistency based on visual similarity, but also uses a predicted arrival time window for gating. Cosine similarity is calculated and association is performed only when the timestamp of the current camera detection result falls into Wk, and the distance between the current target's location and the trajectory intersection point Qk is less than Dmatch. Dmatch can be taken as 10% of the maximum side length of the camera's coverage area or 3-8 meters.

[0088] Trajectory stitching: The target location sequence under the unified security event identifier in the source camera is stitched together with the target location sequence in the current camera according to the timestamp order to generate a cross-camera continuous tracking trajectory with a unified security event identifier. This cross-camera continuous tracking trajectory contains a complete spatiotemporal position record of the target from the source camera's field of view to the current camera's field of view.

[0089] Anomaly Handling: When the number of valid event points is less than 3, the motion estimation module does not perform weighted least squares fitting; instead, the system generates a temporary pre-activation list using the current camera's adjacency relationship and the location of the most recently triggered sensor node. When the propagation velocity scalar v is less than the velocity threshold vmin, the system determines that the target is in a stationary or low-speed state, extends the current camera's tracking time, and reduces the pre-activation priority of the next camera. When the predicted trajectory segment does not intersect with any camera coverage polygon, the system selects the camera with the smallest angle to the heading direction and the closest distance as a candidate pre-activation camera. When the target camera is offline, busy, or the pan / tilt unit cannot rotate, the system selects the next sorted camera from the predicted arrival list and records the abnormal state in the security event log.

[0090] Technical effects of this embodiment

[0091] In this embodiment, the motion estimation module performs weighted least-squares fitting on the spatiotemporal sequence of events triggered by IoT sensor nodes, using inverse distance weights to suppress the influence of abnormal trigger points deviating from the main trajectory, thereby obtaining an event propagation velocity vector reflecting the actual direction and speed of the target's movement. The trajectory prediction module performs time extrapolation along the heading direction based on the event propagation velocity vector and performs geometric intersection calculations with the spatial topology map of the security camera's coverage area. This allows the security platform to determine the next camera the target will enter and its arrival time window before the target arrives, thus overcoming camera selection errors caused by the static switching rule's inability to perceive the direction of movement. The camera pre-activation module completes pan-tilt-zoom (PTZ) orientation adjustment and high-frame-rate acquisition configuration before the expected arrival time window, shifting the camera response delay to the period before the target's arrival. This ensures that the target is in a high-frame-rate, high-resolution acquisition state from the initial moment of entering the new field of view, eliminating image omissions in passive response mode. The cross-camera association module uses visual appearance feature descriptor cosine similarity matching to associate the detection results of front and rear cameras with a unified security event identifier and splice the target location sequence, forming a spatiotemporally continuous cross-camera continuous tracking trajectory on the security platform side, which meets the requirements of real-time deployment and post-event evidence collection for trajectory integrity.

[0092] On a certain night in 20XX, the security management platform of an industrial park detected an unauthorized person entering from near the eastern fence and moving westward along the main road towards the warehouse area. The park is equipped with six network dome cameras (CAM-01 to CAM-06) and nine IoT sensor nodes (including passive infrared motion detectors PIR-01 to PIR-05, infrared beam sensors BM-01 to BM-02, and acoustic microphones AC-01 to AC-02). Each sensor node is positioned using a Cartesian coordinate system, with the origin at the southwest corner of the park, in meters. The target's movement from east to west sequentially triggered four sensor nodes: PIR-03, BM-01, PIR-01, and PIR-02. The security management platform needed to track the target in real time and implement cross-camera video relay.

[0093] Event collection module

[0094] The event acquisition module obtains trigger event records from the four sensor nodes mentioned above, arranges them in ascending order of trigger timestamp, and generates a spatiotemporal event point sequence. The original spatial coordinates and trigger timestamps of each event point are shown in the table below.

[0095] Table 1. Raw data of spatiotemporal event point sequences

[0096]

[0097] Z-score normalization was performed on the spatial coordinate components and timestamps of each event point to eliminate the dimensional difference between meters and seconds. Taking coordinates as an example, mean meters, standard deviation Meters; average timestamp seconds, standard deviation Seconds. The standardized dimensionless coordinates and timestamps of each event point will be used as input to the motion estimation module.

[0098] Motion estimation module

[0099] The motion estimation module performs iterative weighted least squares linear fitting on the standardized spatiotemporal event point sequence. The initial iteration uses the straight line segment connecting event point 1 and event point 4 as the reference trajectory, calculating the perpendicular distance from each event point to this line. , and according to ( Calculate the weights.

[0100] Table 2 Iterative weight calculation results

[0101]

[0102] Event point 2 (BM-01) The coordinates are slightly off-center from the main trajectory (vertical distance 0.04), and their weights are suppressed to 20.00. Event points 1 and 4, which are adjacent to the main trajectory, have the highest weights and the greatest influence on the fitting results. After one iteration, the weighted least squares fitting obtains the horizontal velocity component in the standardized coordinate space. (Standardized dimensionless), vertical velocity component (Standardized dimensionless).

[0103] After being restored to the original physical units by the inverse Z-score transform, the horizontal velocity component m / s, vertical velocity component meters per second, propagation speed scalar meters per second, heading angle (With due east as 0°, measured clockwise), the target is moving primarily westward with a slight deviation to the south. This generates an event propagation velocity vector: velocity 3.73 m / s, heading angle 182.3°.

[0104] Trajectory prediction module

[0105] With the latest event point (PIR-02, coordinates) Starting from the extrapolation point, linear time extrapolation is performed along the heading angle of 182.3° at a step rate of 3.73 m / s to generate the predicted position trajectory line segment within the next 30 seconds.

[0106] The coordinates of the intersection point are determined by geometrically intersecting the predicted trajectory segment with the coverage polygons of each camera in the spatial topology map of the security camera coverage area. The predicted arrival time is calculated by dividing the distance between the intersection point and the initial extrapolation point by the propagation speed scalar. ,in The Euclidean distance between the intersection point and the initial extrapolation point is given. meters per second.

[0107] Table 3. Predicted camera arrival list

[0108]

[0109] Taking CAM-04 as an example, its intersection point and the starting extrapolation point The distance is 18.02 meters, and the predicted arrival time is... Seconds. The list of predicted camera arrival times is sorted in ascending order, with CAM-04 ranked first and thus prioritized for the pre-activation process.

[0110] Camera pre-activation module

[0111] The camera pre-activation module performs parameter pre-configuration on CAM-04, which is ranked high on the camera arrival prediction list. The installation location of CAM-04 is... Installation height: 6.5 meters; coordinates of the trajectory intersection point: The ground height is 0 meters. Calculate the azimuth angle based on the spatial geometric relationship between the installation location and the intersection point of the trajectory. Pitch angle (down).

[0112] Table 4 Camera Pre-Activation Parameter Set

[0113]

[0114] The pre-activation command is issued 2.5 seconds before the target is expected to arrive. Before the target actually enters its field of view, the CAM-04 has completed the pan-tilt rotation and switched to 30fps, 1080P acquisition mode, and entered the pre-configuration ready state.

[0115] Cross-camera association module

[0116] After the target enters the CAM-04's field of view, the cross-camera association module detects the target region from the CAM-04's high frame rate video stream. A 512-dimensional visual appearance feature descriptor vector is then extracted via convolutional neural network feature encoding, denoted as... Simultaneously, retrieve the cached 512-dimensional visual appearance feature descriptor vector from the source camera CAM-02 (the camera before the target left). Calculate the cosine similarity between the two:

[0117] ;

[0118] Table 5 Cross-camera feature matching results

[0119]

[0120] The cosine similarity of 0.94 exceeds the preset matching threshold of 0.80, indicating that the target detected in CAM-04 and the target in CAM-02 are the same security event target, and are associated with the unified security event identifier EVT-20XX-0031. Subsequently, the target location sequence under this event identifier in CAM-02 and the target location sequence in CAM-04 are concatenated in timestamp order to generate a continuous tracking trajectory across cameras.

[0121] Table 6. Continuous Tracking Trajectories Across Cameras (Partial)

[0122]

[0123] Throughout the implementation process, data starts from event recording triggered by IoT sensor nodes, is sorted by the event acquisition module to generate a spatiotemporal event point sequence, and then the motion estimation module extracts the event propagation velocity vector through iterative weighted least squares fitting. This vector is then passed to the trajectory prediction module for time extrapolation and geometric intersection calculation with the camera coverage polygon, outputting a list of predicted camera arrivals. The camera pre-activation module configures the pan-tilt unit and acquisition parameters before the target arrives based on this list, ensuring that the target is in a high frame rate acquisition state from the initial moment it enters the new field of view. Finally, the cross-camera association module uses cosine similarity matching to associate the detection results of the preceding and following cameras with the unified security event identifier EVT-20XX-0031, and splices together the position sequences to form a spatiotemporally continuous cross-camera continuous tracking trajectory on the security platform side, completely recording the entire path of the target moving from the east side of the park to the west side of the warehouse area.

[0124] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A smart security video surveillance system based on the Internet of Things, characterized in that, include: The event acquisition module is configured to acquire trigger event data from IoT sensor nodes in real time and generate a spatiotemporal event point sequence by arranging the trigger timestamps in ascending order. The motion estimation module is configured to perform weighted least squares linear motion fitting on the spatiotemporal event point sequence, using a function that decreases inversely proportionally to the vertical distance from each event point to the fitted trajectory line as weights, and solve to generate an event propagation velocity vector. The trajectory prediction module is configured to perform linear time extrapolation of the motion trajectory based on the event propagation speed vector, perform geometric intersection calculation between the predicted location trajectory line segment and the spatial topology map of the security camera coverage area, and generate a list of predicted camera arrivals. The camera pre-activation module is configured to send a pre-activation command to the target camera before the expected arrival time window to complete the pan-tilt angle adjustment and acquisition parameter configuration. The cross-camera association module is configured to associate multiple detection results with a unified security event identifier through cosine similarity matching of visual appearance feature descriptors, thereby generating a continuous tracking trajectory across cameras.

2. The IoT-based smart security video surveillance system according to claim 1, characterized in that, Each event record in the trigger event data includes the spatial coordinates of the sensor node, the trigger timestamp, and the signal strength value. After generating the spatiotemporal event point sequence, the event acquisition module performs Z-score standardization on the spatial coordinate components and timestamp of each event point to eliminate the dimensional differences between the spatial coordinates and the timestamp.

3. The IoT-based smart security video surveillance system according to claim 1, characterized in that, In the weighted least squares linear motion fitting, the first... Weight of each event point Defined as: ,in For the first The vertical distance from each event point to the currently fitted trajectory line. To prevent small positive constants with a denominator of zero, the fitted trajectory in the initial iteration is taken as a straight line segment connecting the first and last event points of the spatiotemporal event point sequence. In each subsequent iteration, the weight of each event point is recalculated using the fitting result of the previous iteration as a reference trajectory.

4. The IoT-based smart security video surveillance system according to claim 1, characterized in that, The event propagation velocity vector includes a propagation velocity scalar and a heading angle parameter. The propagation velocity scalar is the magnitude of the displacement rate components of the security event target along the horizontal and vertical directions, and the heading angle parameter is the arctangent value of the direction of the displacement rate components. Before trajectory extrapolation, the propagation velocity scalar is restored to its original physical units by inverse Z-score transformation.

5. The IoT-based smart security video surveillance system according to claim 1, characterized in that, In the spatial topology diagram of the security camera coverage area, the effective monitoring field of view of each camera is represented by a coverage polygon. The vertex coordinates of the coverage polygon are calculated based on the camera installation position, focal length, and field of view angle. The geometric intersection calculation is to determine the intersection point coordinates by sequentially intersecting the predicted position trajectory line segment with the boundary of each coverage polygon.

6. The IoT-based smart security video surveillance system according to claim 5, characterized in that, The camera predicted arrival list is generated as follows: the predicted arrival time of the target at the boundary of each camera's coverage area is calculated by dividing the distance between the coordinates of each trajectory intersection point and the starting extrapolation point by the propagation speed scalar, and then sorted in ascending order of predicted arrival time. Each item in the camera predicted arrival list includes the camera identifier, the corresponding trajectory intersection point coordinates, and the expected arrival time window.

7. The IoT-based smart security video surveillance system according to claim 1, characterized in that, The camera pre-activation module calculates the azimuth and pitch parameters of the pan-tilt unit based on the spatial geometric relationship between the coordinates of the trajectory intersection point and the installation position of the target camera. The pre-activation command carries the azimuth, pitch, maximum acquisition frame rate, and maximum resolution parameters of the pan-tilt unit. After receiving the pre-activation command, the target camera completes the pan-tilt rotation and acquisition parameter switching to enter the pre-configuration ready state.

8. The IoT-based smart security video surveillance system according to claim 1, characterized in that, The visual appearance feature descriptor is a fixed-dimensional vector representation, obtained by performing convolutional neural network feature encoding on the target region image; the training phase of the convolutional neural network uses a triplet loss function as the objective and employs the Adam optimization algorithm to optimize the network parameters, minimizing the distance between the visual appearance feature descriptors of the same target under different camera views and maximizing the distance between the visual appearance feature descriptors of different targets.

9. The IoT-based smart security video surveillance system according to claim 1, characterized in that, When the cosine similarity value exceeds the preset matching threshold, the cross-camera association module determines that the two detection results correspond to the same security event target and associates the detection result in the current camera with the unified security event identifier to which the detection result of the source camera belongs. The target location sequence under the unified security event identifier in the source camera is concatenated with the target location sequence in the current camera according to the timestamp order to generate the cross-camera continuous tracking trajectory.

Images