High-low point linkage AR real scene VGIS three-dimensional cloud alarm linkage method

Abstract

The application relates to the technical field of video monitoring and intelligent security and protection, and discloses a high-low point linkage AR real scene VGIS three-dimensional cloud alarm linkage method. The method comprises the following steps: a target is recognized by a high point camera and a two-dimensional trajectory is generated, the two-dimensional trajectory is mapped to a VGIS three-dimensional coordinate system to obtain a space coordinate; trajectory prediction is carried out in combination with a space topological structure; the containing degree of a predicted position is judged and occlusion detection is carried out based on a low point camera field of view range; if the predicted position does not fall into the field of view, the time is iteratively updated and re-predicted until the predicted position falls into the field of view and a target low point camera is determined; the device coordinates of the target low point camera are extracted, a relative space vector and a straight line distance are calculated, and update compensation is carried out according to a mechanical response delay time; finally, control instructions containing angle adjustment and focal length adjustment are generated, and the application improves the response speed and positioning accuracy of cross-view angle target tracking.

Description

Technical Field

[0001] This application relates to the fields of video surveillance and intelligent security technology, and in particular to a high-low point linkage AR real-scene VGIS three-dimensional cloud alarm linkage method. Background Technology

[0002] In a community intelligent security monitoring system, high-point cameras, with their wide field of view, can conduct preliminary monitoring of human activities within a large area, while low-point cameras, relying on their close-up shooting advantage, can obtain key details such as facial features of targets. The coordinated operation of these two types of equipment forms the basic architecture of a three-dimensional monitoring network, which is of great significance for the continuous tracking of suspected targets.

[0003] However, current surveillance systems generally suffer from the following shortcomings: First, most systems employ a passive response mechanism, meaning that the recognition and recording process is only triggered after the target enters the camera's field of view. This lag leads to frequent issues such as image shift and focus misalignment when low-point cameras capture moving targets. During the transition phase after the target disappears from the high-point field of view and enters the low-point coverage area, the low-point device can only wait at a default angle for the target to naturally re-enter the frame, easily resulting in missed capture opportunities. Second, there is a lack of effective spatial mapping between the target movement trajectory information captured by the high-point device and the pre-adjusted control of the low-point device. The trajectory data tracked by the high-point camera remains only in a two-dimensional image coordinate system and cannot be directly converted into specific control commands such as the pan-tilt rotation angle and focus configuration required by the low-point camera. Third, existing high-low point linkage solutions mostly rely on manual retrieval or simple picture-in-picture display, lacking the ability to proactively predict and automatically link control based on target movement trends.

[0004] Therefore, how to integrate the target movement trajectory data captured from a high-point perspective with the spatial topology in a geographic information system, predict the low-point monitoring area that the target is about to enter, and convert the prediction results into precise pre-adjustment parameters for the low-point camera pan-tilt unit, so as to achieve fully automated connection from trajectory prediction to equipment control, has become a key issue that urgently needs to be addressed in collaborative monitoring across high and low point perspectives. Summary of the Invention

[0005] To address the aforementioned technical issues, this application provides a high-low point linkage AR real-scene VGIS stereoscopic cloud alarm linkage method, which is used to achieve automated linkage control for cross-view target tracking, thereby improving tracking response speed and positioning accuracy.

[0006] Firstly, this application provides a high-low point linkage AR real-scene VGIS three-dimensional cloud alarm linkage method, the method comprising: Step S1: Acquire continuous video frames captured by the high-point camera, extract the pixel coordinate sequence of the target object in the continuous video frames through the target recognition model, generate a two-dimensional movement trajectory and map it to the preset VGIS three-dimensional spatial coordinate system to obtain the first spatial coordinates corresponding to the two-dimensional movement trajectory; Step S2: Based on the first spatial coordinates and the spatial topology contained in the VGIS three-dimensional spatial coordinate system, a trajectory prediction algorithm is used to predict the trajectory of the target object's movement state, thereby obtaining the predicted spatial position of the target object at the target time. Step S3: Based on the VGIS three-dimensional spatial coordinate system, obtain the field of view of at least one low-point camera in the monitoring area, perform inclusion degree determination and occlusion detection on the predicted spatial position, and determine whether the predicted spatial position is within the field of view of any low-point camera. Step S4: If the target is not within the field of view of any low-point camera, update the target time and reacquire the latest spatial coordinates of the target object for trajectory prediction until the predicted spatial position within the field of view of any low-point camera is obtained, and the target low-point camera is determined. Step S5: Extract the device coordinates of the target low-point camera, calculate the relative spatial vector and straight-line distance between the predicted spatial position and the device coordinates, and update the relative spatial vector and the straight-line distance according to the mechanical response delay time of the target low-point camera; Step S6: Determine the angle adjustment parameters and focal length parameters of the target low-point camera based on the updated relative space vector and straight-line distance, and generate control commands for the target low-point camera.

[0007] Compared with the prior art, the beneficial effects of the present invention are at least as follows: This application maps the target trajectory captured by the high-point camera to the VGIS three-dimensional spatial coordinate system, performs trajectory prediction in combination with the spatial topology, and uses the field of view of the low-point camera to determine the inclusion degree and detect occlusion of the predicted position, thereby realizing the automated closed-loop control of high-point target trajectory prediction and low-point camera pre-linkage. This application iteratively updates the target time and re-predicts until the target object enters the field of view of the low-point camera to determine the target low-point camera. Combined with compensation processing for mechanical response delay time, it effectively eliminates the tracking deviation caused by the device execution delay, so that the low-point camera can complete accurate alignment before the target arrives, improving the tracking continuity and capture success rate when the target crosses between different viewpoints. This application accurately converts the predicted spatial location into angle adjustment parameters and focal length parameters of the low-point camera and automatically generates control commands, realizing a complete automated process from target detection to device control, and improving the linkage response speed and positioning accuracy of the three-dimensional monitoring network. Attached Figure Description

[0008] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a flowchart of the high and low point linkage AR real-scene VGIS stereoscopic cloud alarm linkage method in the embodiments of this application; Figure 2 This is a side view of the occlusion detection principle in an embodiment of this application. Detailed Implementation

[0010] This application provides a high-low point linkage AR real-scene VGIS three-dimensional cloud alarm linkage method. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0011] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 The high-low point linkage AR real-scene VGIS stereoscopic cloud alarm linkage method in this application embodiment includes: Step S1: Acquire continuous video frames captured by the high-point camera, extract the pixel coordinate sequence of the target object in the continuous video frames through the target recognition model, generate a two-dimensional movement trajectory and map it to the preset VGIS three-dimensional spatial coordinate system to obtain the first spatial coordinates corresponding to the two-dimensional movement trajectory.

[0012] Step S1, generating the two-dimensional movement trajectory of the target object, includes: acquiring continuous video frames captured by a high-point camera; parsing the continuous video frames using a video decoder to obtain a first video frame sequence; extracting the pixel points of the target object in the first video frame sequence using a target recognition model to obtain a pixel coordinate sequence of the target object; calculating the coordinate value difference between adjacent pixels in the pixel coordinate sequence; if the coordinate value difference is greater than a preset difference threshold, removing the corresponding pixel point to obtain a cleaned target coordinate sequence; connecting the coordinate values ​​in the cleaned target coordinate sequence in chronological order to generate the initial two-dimensional movement trajectory of the target object; and calculating the average coordinate value of each moving point in the initial two-dimensional trajectory line using a sliding window method, and generating a smoothed two-dimensional movement trajectory based on the average coordinate value.

[0013] Specifically, high-position cameras refer to front-end video acquisition devices deployed at high locations within a monitored area, possessing a top-down field of view and used to acquire overall or large-scale scene images of the monitored area. They are typically mounted on building rooftops, towers, lampposts, or high on mountainsides, with an installation height significantly higher than surrounding ground features to obtain a wide-angle, top-down view. The field of view of high-position cameras covers major roads, entrances and exits, squares, and the locations of multiple low-position cameras within the monitored area, used for macroscopic monitoring and initial detection of target objects within the monitored area. High-position cameras have pan-tilt-zoom (PTZ) control functions, capable of adjusting the horizontal rotation angle, vertical tilt angle, and lens magnification to perform cruise scanning and fixed-point tracking over a large area.

[0014] First, continuous video frames are acquired from a high-position camera. These continuous video frames are a sequence of images continuously captured and output by the high-position camera at a predetermined frame rate. Each frame in this sequence carries a timestamp to identify the capture time, thus providing a time reference for subsequently constructing the target object's movement trajectory over time. Since the video stream output from high-position cameras is typically compressed using encoding standards such as H.264 or H.265 to reduce data transmission bandwidth, it cannot be directly used for image analysis. Therefore, the continuous video frames must first be decoded using a video decoder to restore the compressed video stream to original image frames arranged in the capture sequence. These original image frames together constitute the first video frame sequence. Each frame in the first video frame sequence is an uncompressed bitmap image, which can be directly input into the subsequent target recognition model for processing.

[0015] After obtaining the first video frame sequence, this sequence is input into the target recognition model. The target recognition model performs target detection on each frame, extracting the pixel positions of the target objects. The target recognition model adopts the YOLO series network architecture, which internally extracts image features by sliding multiple convolutional kernels across the image, abstracting edges, corners, textures, and overall semantic features of the target from the bottom up. Finally, it outputs the position parameters of the target bounding box through a regression subnetwork. In actual operation, each frame of the first video frame sequence is sequentially input into the target recognition model trained above. For a given input image frame, the model first downsamples and extracts features from the image using a backbone network, generating multi-scale feature maps. Then, it densely plots anchor boxes of a preset size on the feature maps. Each anchor box is predicted by a regression branch, along with its offset relative to the target's true bounding box and the confidence level of the target's presence within that anchor box. Finally, a non-maximum suppression operation filters out redundant predicted boxes with an overlap rate exceeding a preset threshold, retaining the predicted box with the highest confidence as the target object's bounding box output. This bounding box is represented in pixels in the image coordinate system, containing the row and column coordinates of the target object's top-left vertex, as well as the bounding box's pixel width and pixel height. The row center coordinates are obtained by adding half the height to the row coordinates of the top-left vertex, and the column center coordinates are obtained by adding half the width to the column coordinates of the top-left vertex.

[0016] Each frame of the first video frame sequence is input into the target recognition model in sequence. The coordinates of the center point of the bounding box of the target object in each frame are recorded and arranged from front to back according to the frame order. This yields a sequence of pixel coordinates of the target object that changes over time. Each pixel coordinate value in this sequence corresponds one-to-one with a frame image, reflecting the position of the target object in the image plane at each sampling time.

[0017] Because high-position cameras operate in outdoor environments, the monitoring footage may be affected by factors such as changes in light, rain and fog, obstruction by birds or leaves, and motion blur caused by the target's own movement. This can lead to abnormal detection results in individual video frames, where the target recognition model outputs abnormal detection results. These abnormal detection results manifest as significant jumps in the center point coordinates of the target object between adjacent frames relative to the target's actual movement. If such abnormal values ​​are retained in the sequence, they will cause serious distortion in the subsequently generated trajectory, making it impossible to accurately reflect the target's actual movement path.

[0018] To eliminate these outliers, for each pair of adjacent pixels in the pixel coordinate sequence, the difference between them in the row direction and the difference in the column direction are calculated. Specifically, for the pixel coordinates of the i-th frame and the (i+1)-th frame arranged in chronological order, the row direction difference is obtained by subtracting the row coordinate of the i-th frame from the row coordinate of the (i+1)-th frame, and the column direction difference is obtained by subtracting the column coordinate of the i-th frame from the column coordinate of the (i+1)-th frame. The row direction difference and column direction difference are compared with preset row direction difference thresholds and column direction difference thresholds, respectively. The row direction difference threshold and column direction difference threshold represent the maximum number of pixels that the target object is allowed to move in the vertical and horizontal directions of the image between two adjacent frames, respectively. The unit is pixels, and its essence is the upper limit of vertical and horizontal displacement in the image space. The thresholds are determined as follows: the preset maximum movement speed of the target object in the monitoring scene is multiplied by the time interval between adjacent frames to obtain the maximum physical distance that the target object may move between frames. Then, according to the installation height of the high-point camera, the geometric projection relationship between the field of view and the image resolution, the maximum physical distance is converted into pixel distance on the image plane. The vertical component of this pixel distance is used as the row direction difference threshold, and the horizontal component is used as the column direction difference threshold. The maximum movement speed needs to be manually calibrated or categorized based on the actual security control targets in the monitored area. For example, if the controlled targets are pedestrians, the speed value can be an empirical constant between 1.5 m / s and 3.5 m / s; if the controlled targets include non-motorized vehicles, the speed value can be an empirical constant between 4 m / s and 6 m / s. In actual deployment, this value can be flexibly set according to the scenario, but once set, it will serve as a fixed calculation benchmark for generating the difference threshold during system operation.

[0019] If the absolute value of the row direction difference is greater than the row direction difference threshold, or the absolute value of the column direction difference is greater than the column direction difference threshold, the pixel corresponding to the (i+1)th frame is determined to be an outlier deviating from the actual motion trajectory of the target, and this outlier is removed from the pixel coordinate sequence. If both the row direction difference and the column direction difference are less than or equal to the corresponding difference threshold, the pixel corresponding to the (i+1)th frame is determined to be a normal pixel and is retained. All adjacent pixel combinations in the pixel coordinate sequence are traversed sequentially, and the retained pixels constitute the cleaned target coordinate sequence. Through the above difference comparison and removal operations, outliers introduced by identification errors are effectively filtered out, making the trajectory data more accurate and reliable.

[0020] After obtaining the cleaned target coordinate sequence, each coordinate value in the sequence still carries timestamp information, reflecting the spatial position of the target object at the time the corresponding frame image was captured. The coordinate values ​​in the cleaned target coordinate sequence are connected sequentially according to time, i.e., two coordinate points at adjacent times are connected by straight line segments in the image coordinate system, forming the initial two-dimensional movement trajectory of the target object in the image plane as it changes over time. This trajectory is a polyline formed by connecting a set of discrete coordinate points arranged in chronological order with line segments. Each vertex on the polyline corresponds to the actual detection position of the target object in a frame image. The direction of the polyline reflects the movement direction of the target object in the monitored scene, and the length of the line segment reflects the pixel distance the target object moves in the image plane between adjacent frames.

[0021] Because the target recognition model detects pixel-level positional fluctuations in the center points of bounding boxes between adjacent frames, the initial 2D motion trajectory, despite outlier removal, may still exhibit subtle jagged jitter in local areas. This jitter originates from image noise, pixel-level blurring of target edges, and minor random deviations in the model output, and does not reflect the true movement trend of the target. Directly using this jagged trajectory for subsequent spatial coordinate mapping and alarm localization will result in unnecessary minor offset errors in the mapped spatial coordinates, reducing the accuracy of the alarm.

[0022] To eliminate this jitter and improve the smoothness of the trajectory, a sliding window method is used to smooth the initial two-dimensional movement trajectory. The specific implementation of the sliding window method is as follows: A window containing an odd number of trajectory points is set, such as five or seven. The geometric center point of the window is aligned with the trajectory point to be smoothed. Using this trajectory point as the center, the arithmetic mean of the coordinates of all trajectory points within the window in the row direction is taken as the smoothed row coordinates of the center point. Similarly, the arithmetic mean of the coordinates of all trajectory points within the window in the column direction is taken as the smoothed column coordinates of the center point. Then, the window is moved one trajectory point backward along the trajectory direction. At this point, the window contains a new set of trajectory points centered on the second trajectory point in the original window. The process of taking the average value is repeated to generate smoothed coordinate values ​​for the next trajectory point.

[0023] As the window slides from the starting point to the end point along the entire trajectory, a corresponding smoothed coordinate value is generated for each trajectory point in the initial two-dimensional movement trajectory. These smoothed coordinate values ​​are then reconnected in chronological order to form the smoothed two-dimensional movement trajectory. The smoothed two-dimensional movement trajectory retains the overall direction and approximate speed of the target while effectively removing local pixel-level random jitter, making the trajectory line smoother and more continuous. This more realistically reflects the actual movement path of the target object within the monitoring area, providing a high-quality trajectory data input foundation for mapping the two-dimensional movement trajectory to the VGIS three-dimensional spatial coordinate system in subsequent steps.

[0024] In step S1, obtaining the first spatial coordinates corresponding to the two-dimensional movement trajectory includes: obtaining multiple trajectory points contained in the two-dimensional movement trajectory; performing transformation processing on the multiple trajectory points using a coordinate mapping algorithm; mapping the two-dimensional movement trajectory to a preset VGIS three-dimensional spatial coordinate system to obtain the first spatial coordinates corresponding to the two-dimensional movement trajectory; calculating the offset between the first spatial coordinates and the reference point; if the offset is greater than a preset judgment value, adding a correction term to the first spatial coordinates to obtain the corrected first spatial coordinates.

[0025] Specifically, based on the smoothed two-dimensional movement trajectory, in order to further apply the trajectory information to the high-low point linkage alarm, it is necessary to convert the two-dimensional trajectory data in the image plane into three-dimensional spatial coordinates in the real world, so as to establish the spatial correlation between the detection results of the high point camera and the control of the low point camera.

[0026] Specifically, multiple trajectory points are obtained from the smoothed two-dimensional motion trajectory. Each trajectory point exists in the form of pixel coordinates in the image coordinate system, including row and column coordinates. A coordinate mapping algorithm is used to perform transformation processing on the above multiple trajectory points. The coordinate mapping algorithm is established based on the imaging geometry model of the high-point camera. This model describes the correspondence between image pixels and spatial points in the VGIS three-dimensional spatial coordinate system through pre-calibrated intrinsic and extrinsic parameter matrices of the high-point camera. The intrinsic parameter matrix contains the focal length, principal point coordinates, and distortion coefficients of the high-point camera, which are used to describe the imaging characteristics of the camera itself; the extrinsic parameter matrix contains the rotation matrix and translation vector of the high-point camera in the world coordinate system, which are used to describe the installation position and orientation of the camera in the VGIS three-dimensional spatial coordinate system.

[0027] The core principle of the coordinate mapping algorithm is to construct an inverse perspective projection transformation from two-dimensional image coordinates to three-dimensional spatial coordinates using the aforementioned intrinsic and extrinsic matrices. In specific implementation, for each trajectory point in the two-dimensional movement trajectory, the intrinsic matrix is ​​first used to correct the distortion of the pixel coordinates to eliminate the positional deviation caused by lens distortion, resulting in normalized image coordinates. Subsequently, the normalized image coordinates are combined with the rotation matrix and translation vector in the extrinsic matrix to establish the equation of a spatial ray originating from the optical center of the camera and passing through the pixel point. Finally, the spatial ray is intersected with the pre-defined ground plane (i.e., the horizontal plane of the ground height of the monitored area) in the VGIS three-dimensional spatial coordinate system, and the three-dimensional coordinates of the intersection point are the actual spatial position of the trajectory point on the ground.

[0028] The above transformation process is performed sequentially on each trajectory point in the two-dimensional movement trajectory to obtain the corresponding three-dimensional spatial coordinates of each trajectory point. These three-dimensional spatial coordinates together constitute the first spatial coordinates of the two-dimensional movement trajectory in the VGIS three-dimensional spatial coordinate system. Through the above coordinate mapping, the pixel movement trajectory of the target in the high-point camera video image is converted into a sequence of three-dimensional spatial coordinates with real geospatial location information, thereby establishing a mapping relationship between the two-dimensional movement trajectory and the first spatial coordinates.

[0029] However, in actual engineering deployments, physical quantities such as the installation position, pitch angle, and lens distortion parameters of high-point cameras may experience slight drifts due to wind load, temperature changes, or mechanical loosening during long-term operation. In addition, measurement errors introduced during calibration can cause deviations between the intrinsic and extrinsic parameter matrices obtained from the initial calibration and the actual physical parameters. This results in a systematic offset in absolute position of the first spatial coordinates calculated based on the aforementioned calibration parameters.

[0030] To eliminate this offset, after obtaining the first spatial coordinates, at least one pre-recorded ground reference point in the VGIS three-dimensional spatial coordinate system is used as a benchmark. This ground reference point is a ground landmark with obvious geographical features and known coordinates selected within the monitoring area, and its coordinates are obtained through actual measurement using high-precision measuring equipment. For each ground reference point, the spatial offset between the mapping point corresponding to that reference point in the first spatial coordinates and the reference point itself is calculated.

[0031] If the offset is greater than the preset judgment value, it indicates that there is a non-negligible systematic error in the current coordinate mapping result. In this case, a correction term is added to each coordinate value in the first spatial coordinate system based on the offset. This correction term is obtained by interpolating and fitting the offsets measured at multiple ground reference points, and the corrected coordinate values ​​are used as the corrected first spatial coordinates. If the offset is less than or equal to the preset judgment value, it indicates that the current mapping accuracy meets the requirements, and the first spatial coordinates are directly used as the corrected first spatial coordinates. The preset judgment value is determined based on the coordinate mapping accuracy obtained during the system calibration phase. Specifically, after the high-point camera is installed, several ground reference points with known geographical locations are selected within the monitoring area. The mapped coordinates of each reference point are calculated using a coordinate mapping algorithm. The difference between the mapped coordinates and the measured coordinates is used as the calibration error. The absolute mean or root mean square error of the calibration errors of all reference points is statistically analyzed, and the preset judgment value is twice this statistical error value.

[0032] Through the above offset calculation and correction compensation process, the systematic mapping error introduced by camera calibration parameter drift and environmental factors is effectively suppressed, and the spatial transformation accuracy of high point video target positioning to VGIS three-dimensional spatial coordinate system is further improved, providing a highly reliable spatial coordinate reference for the accurate execution of subsequent trajectory prediction and low point camera linkage control.

[0033] Step S2: Based on the first spatial coordinates and the spatial topology contained in the VGIS three-dimensional spatial coordinate system, a trajectory prediction algorithm is used to predict the trajectory of the target object's movement state, thereby obtaining the predicted spatial position of the target object at the target time.

[0034] The step S2, obtaining the predicted spatial position of the target object at the target time, includes: acquiring the position of the first spatial coordinates within the VGIS three-dimensional spatial coordinate system; extracting the spatial topology contained in the VGIS three-dimensional spatial coordinate system to obtain a set of topological nodes; selecting the initial positioning node of the target object based on the first spatial coordinates according to the set of topological nodes to obtain the initial movement state of the target object; if the initial movement state meets the preset active conditions, then using the Kalman filter algorithm to process the initial movement state to obtain a smooth movement state; matching the smooth movement state with the spatial topology to determine the continuous movement path of the target object within the VGIS three-dimensional spatial coordinate system; using a trajectory prediction algorithm to process the continuous movement path to obtain the initial predicted spatial position of the target object at the target time; if the initial predicted spatial position exceeds the coverage area of ​​the spatial topology, then correcting the initial predicted spatial position according to the edge nodes of the coverage area to obtain the final predicted spatial position of the target object at the target time.

[0035] Specifically, based on the corrected first spatial coordinates, in order to predict the spatial position of the target object at future moments and thus provide a time lead for the pre-linkage control of the low-point camera, it is necessary to further predict the movement trend of the target object by combining the geographic spatial constraints of the monitoring area.

[0036] Specifically, after obtaining the specific location of the corrected first spatial coordinates in the VGIS three-dimensional spatial coordinate system, the spatial topology structure stored in the VGIS three-dimensional spatial coordinate system is extracted. This spatial topology structure is a topology map formed by abstractly modeling spatial elements such as roads, intersections, passage intersections, and building distribution within the monitoring area. It contains multiple topology nodes and connecting edges. The topology nodes correspond to key location points within the monitoring area, such as the center of an intersection, the corner of a passage, and the location of entrances and exits. The connecting edges represent the path relationship between adjacent topology nodes that can be passed by the target object. After obtaining the spatial topology structure, all topology nodes are extracted from it to form a topology node set.

[0037] Based on the set of topological nodes, the initial positioning node of the target object is selected based on the first spatial coordinates. Since the first spatial coordinates contain the three-dimensional spatial coordinates of the target object at multiple historical moments, forming a continuous spatial coordinate sequence, the spatial coordinates corresponding to the latest moment are selected as the current position of the target object. The spatial distance between the current position and each topological node in the set of topological nodes is calculated. This spatial distance is obtained by calculating the Euclidean distance between the three-dimensional coordinates of the current position and the three-dimensional coordinates of each topological node. The topological node with the smallest spatial distance is selected as the initial positioning node of the target object. This initial positioning node is the current position node of the target object in the spatial topology.

[0038] After obtaining the initial positioning node, the target object's moving speed and direction are calculated by combining the spatial coordinate differences between adjacent moments in the first spatial coordinate sequence, thus obtaining the target object's initial moving state. Specifically, the spatial coordinates corresponding to the two most recent adjacent moments in the first spatial coordinate sequence are selected, and the coordinate difference vector between them is calculated. This difference vector is divided by the time interval between the two adjacent moments to obtain the target object's moving speed vector. The magnitude of this moving speed vector is the magnitude of the moving speed, and its direction is the moving direction vector. The target object's current position, moving speed magnitude, and moving direction vector together constitute the initial moving state.

[0039] After obtaining the initial movement state, it is determined whether the initial movement state meets the preset active condition, that is, whether the magnitude of the movement speed in the initial movement state is greater than the preset speed threshold. This speed threshold is an empirical critical value for distinguishing between a moving and stationary target, and can be set according to the lower limit of the normal walking speed of pedestrians in the monitored scene. If the magnitude of the movement speed is not greater than the speed threshold, it indicates that the target object is currently stationary or in a state of slight movement in place, and does not have the tendency to move across the field of view. There is no need to trigger subsequent trajectory prediction and low-point camera linkage. At this time, the prediction process ends and waits for the next detection cycle. If the magnitude of the movement speed is greater than the speed threshold, it indicates that the target object is in a state of continuous movement, and trajectory prediction is necessary. At this time, the Kalman filter algorithm is used to process the initial movement state, filtering out observation noise and motion noise introduced by factors such as detection errors of the target recognition model, spatial transformation errors in the coordinate mapping process, and random fluctuations generated during the target's own movement, thereby obtaining the optimal estimate of the target object's true movement state.

[0040] The Kalman filter algorithm is an optimal state estimation algorithm based on a linear Gaussian state-space model. It estimates the true state of the system through two iterative stages: prediction and update. In this application, the input data of the algorithm are the position coordinates, velocity magnitude, and direction vector of the initial movement state. The algorithm first sets the state vector of the target object, which includes the position and velocity components of the target in the three coordinate axes of the VGIS three-dimensional spatial coordinate system, for a total of six dimensions. In the prediction stage, the optimal state estimate of the previous time step is extrapolated to the current time step using the state transition matrix to obtain the prior state estimate of the current time step. At the same time, the error covariance matrix is ​​updated to characterize the uncertainty of the prediction. In the update stage, the actual position observation value obtained from the first spatial coordinate sequence at the current time step is used as the correction basis. The Kalman gain matrix is ​​calculated, and the prior state estimate is weighted and corrected using this gain matrix. The prediction result and the observation result are fused, and the smoothed movement state after noise filtering and smoothing is output.

[0041] After obtaining the smooth movement state, the smooth movement state is matched with the spatial topology to determine the continuous movement path of the target object in the VGIS 3D spatial coordinate system. The matching process is as follows: based on the position coordinates in the smooth movement state, the topological node where the target object is currently located is repositioned in the set of topological nodes. Combined with the movement direction vector in the smooth movement state, the subsequent topological nodes in the spatial topology are searched along the connecting edge in that direction. The topological node where the target object is currently located and each subsequent topological node along the movement direction are connected in sequence to form a series of nodes arranged in spatial order. There are connecting edges between adjacent nodes in this node sequence. The polyline path formed by the nodes is the continuous movement path of the target object in the VGIS 3D spatial coordinate system.

[0042] After determining the continuous movement path, a trajectory prediction algorithm is used to process the path to obtain the initial predicted spatial position of the target object at the target time. This trajectory prediction algorithm adopts a uniform motion model, assuming that the target object maintains its current speed for a short period of time and moves at a constant speed along the continuous movement path. In specific implementation, the target object's current position is taken as the starting point, and the movement speed in the smooth movement state is extrapolated forward along the direction of the continuous movement path. The arc length of the path traversed by the target object during the time interval from the current time to the target time is calculated. Then, the arc length of the path is accumulated from the starting point on the continuous movement path, and the position of the path point corresponding to the accumulated arc length reaching the calculated value is taken as the initial predicted spatial position of the target object at the target time.

[0043] If the initial predicted spatial location falls within the coverage area of ​​the spatial topology, it is directly used as the final predicted spatial location of the target object at the target time. If the initial predicted spatial location exceeds the coverage area of ​​the spatial topology, i.e., when extrapolating along the continuous movement path, the calculated cumulative path arc length exceeds the coverage of the farthest topological node on that path, the initial predicted spatial location is corrected based on the edge nodes of the spatial topology's coverage area. Specifically, the edge topological node closest to the initial predicted spatial location is selected on the continuous movement path. An edge topological node is either the last node in the spatial topology located in the direction of the end of the continuous movement path or the path's end node with the closest straight-line distance to the initial predicted spatial location. The spatial coordinates of this edge topological node are used as the final predicted spatial location of the target object at the target time. This method ensures that the trajectory prediction results both follow the target's own motion inertia and are effectively constrained by the actual geographic spatial path, avoiding the problem of predicted locations deviating from the actual passable area. This provides an accurate and geographically constrained predicted location basis for subsequent pre-linkage control of low-point cameras.

[0044] Step S3: Based on the VGIS three-dimensional spatial coordinate system, obtain the field of view of at least one low-point camera in the monitoring area, perform inclusion degree determination and occlusion detection on the predicted spatial location, and determine whether the predicted spatial location is within the field of view of any low-point camera.

[0045] Step S3, determining whether the predicted spatial location is within the field of view of any low-point camera, includes: obtaining the intrinsic and extrinsic parameter matrices of the low-point camera, determining the coverage area of ​​the low-point camera in the VGIS 3D spatial coordinate system, and extracting multiple edge vertex coordinates from the coverage area to construct a 3D projection surface; using a ray crossing algorithm to perform an intersection test between the predicted spatial location and the 3D projection surface; if the number of intersections is odd, it is determined that the predicted spatial location is within the coverage area of ​​the field of view of the low-point camera; extracting the depth map corresponding to the 3D projection surface, and obtaining the actual depth value corresponding to the predicted spatial location from the depth map; if the actual depth value is less than the distance from the predicted spatial location to the optical center of the low-point camera, it is determined that the predicted spatial location is occluded and removed from the field of view.

[0046] Specifically, after obtaining the predicted spatial location of the target object at the target time, in order to determine whether the predicted location can be actually captured by the low-point camera, and thus decide whether to trigger the pre-linkage control of the low-point camera, it is necessary to determine the spatial inclusion relationship between the predicted spatial location and the field of view of each low-point camera based on the VGIS three-dimensional spatial coordinate system.

[0047] Low-point cameras are front-end video acquisition devices deployed at low altitudes within a monitored area. They have a level or upward-looking field of view and are used to acquire detailed image information such as facial features, clothing details, and behavioral movements of the target object. They are typically mounted near entrance / exit gates, on poles along roadsides, at the base of building exteriors, or at corners of passageways. Their installation height is roughly the same as or slightly higher than the target object's height to obtain a close-range, level or slightly downward-looking shooting perspective. Low-point cameras generally have a narrow field of view, but their imaging resolution is high, clearly capturing local details of the target object, making them an important supplement to the wide-angle macroscopic field of view of high-point cameras. In actual monitoring scenarios, multiple low-point cameras are usually deployed within the monitored area. The fields of view of each low-point camera are spatially connected or partially overlapped, forming a fine-grained ground coverage network for the monitored area. Each low-point camera has pan-tilt-zoom (PTZ) control functionality, allowing it to rotate horizontally and vertically to adjust the shooting angle and supports lens zoom to adjust the focal length. This allows it to change the alignment direction and image clarity of its field of view upon receiving control commands.

[0048] To determine whether a predicted spatial location is within the field of view of any low-point camera, the intrinsic and extrinsic parameter matrices of the low-point cameras within the monitoring area are first obtained. The intrinsic parameter matrix contains imaging parameters such as the focal length, principal point coordinates, and distortion coefficient of the low-point cameras, used to describe the imaging characteristics of the cameras themselves. The extrinsic parameter matrix contains the installation position and orientation parameters of the low-point cameras in the VGIS 3D spatial coordinate system. Based on the intrinsic and extrinsic parameter matrices, combined with the lens field of view parameters of the low-point cameras, the coverage area of ​​the low-point cameras in the VGIS 3D spatial coordinate system is determined. This coverage area is a spatial quadrangular pyramid formed by extending in the shooting direction with the optical center of the low-point camera as the vertex, the field of view angle as the subtended angle, and the maximum depth of field. The maximum depth of field is the farthest object distance that the low-point camera can clearly image, and its value is determined based on the lens focal length and image sensor parameters of the low-point cameras, or preset according to the actual coverage requirements of the monitoring scene, used to limit the longitudinal extension range of the spatial quadrangular pyramid when determining the field of view. Multiple edge vertex coordinates are extracted from the coverage area, including the four vertices of the base of the pyramid and the optical center position. These edge vertex coordinates collectively define the theoretical coverage boundary of the low-point camera in three-dimensional space. A three-dimensional projection surface is constructed using these multiple edge vertex coordinates. This three-dimensional projection surface is a spatial polygonal plane established in the VGIS three-dimensional spatial coordinate system based on the imaging plane of the low-point camera. Its shape is enclosed by the lines connecting the edge vertex coordinates of the coverage area, representing the maximum imaging range of the low-point camera in a certain direction.

[0049] After obtaining the 3D spatial value of the predicted spatial location, a ray intersection algorithm is used to test the intersection of this 3D spatial value with the 3D projection surface. The basic principle of the ray intersection algorithm is as follows: a ray is emitted from the point to be tested along any direction, and the number of intersection points between the ray and the boundary polygon of the 3D projection surface is counted. If the number of intersection points is odd, the point to be tested is determined to be inside the polygon; if the number of intersection points is even, the point to be tested is determined to be outside the polygon. Specifically, in this application, a ray is emitted from the predicted spatial location along any coordinate axis direction, and the intersection points between the ray and each edge of the 3D projection surface are calculated and counted. If the number of intersections is odd, the predicted spatial location is determined to be within the field of view coverage of the low-point camera; if the number of intersections is even, the predicted spatial location is determined to be outside the field of view of the low-point camera, and no further processing is required.

[0050] Since the above inclusion determination only confirms that the predicted point is within the field of view of the low-point camera in the angular direction, in the actual monitoring scenario, there may be obstacles such as building walls, trees, and pillars between the predicted spatial location and the low-point camera, which may cause the actual light path to be blocked even though the predicted point is within the field of view, and the low-point camera cannot capture the target object at that location.

[0051] To solve this problem, after determining that the predicted spatial location is within the field of view's angular coverage area, the three-dimensional coordinates of the predicted spatial location are projected onto the image pixel coordinate system of the low-point camera using the intrinsic parameter matrix and coordinate system transformation, thus obtaining the corresponding image pixel coordinates.

[0052] Subsequently, the depth map corresponding to the 3D projection surface is extracted. The depth map is pre-acquired as follows: After deployment in the monitoring area, a 3D laser scanning device or binocular stereo vision system is used to reconstruct the monitoring scene in 3D, generating depth distance information for each spatial point in the scene relative to the optical center of each low-point camera. This information is recorded in the form of a 2D image matrix, recording the distance values ​​of each spatial point within the monitoring area relative to the optical center of the low-point camera. The pixel coordinates of each pixel in the matrix correspond one-to-one with its spatial position on the 3D projection surface, and the pixel value of each pixel represents the depth distance from the object surface at that spatial position to the optical center of the low-point camera. The depth value corresponding to the predicted spatial position is obtained from this depth map. That is, based on the mapping relationship of the predicted spatial position on the 3D projection surface, the pixel point corresponding to the mapped position is found in the depth map. The pixel value of this pixel is the actual depth distance from the object surface at that position to the optical center of the low-point camera. The pixel value at the corresponding coordinate position in the depth map is indexed based on the pixel coordinates obtained from the projection image, serving as the actual depth value corresponding to the predicted spatial position. Figure 2 The diagram shows a side view of the occlusion detection principle. An obstacle blocks the line of sight from the optical center of the low-point camera towards the prediction point. The dashed line in the diagram indicates the line of sight emitted from the optical center towards the prediction point; this line of sight is interrupted by the obstacle before reaching the prediction point.

[0053] The actual depth value obtained from the depth map is compared with the straight-line distance from the predicted spatial location to the optical center of the low-point camera. If the actual depth value is less than the distance from the predicted spatial location to the optical center of the low-point camera, it indicates that there is a closer spatial object between the predicted spatial location and the low-point camera that is blocking the predicted point, causing the low-point camera to be unable to capture the target object at the predicted location. In this case, the predicted spatial location is determined to be blocked and removed from the true field of view. If the depth value is not less than the distance from the predicted spatial location to the optical center of the low-point camera, it indicates that there is no closer obstruction between the low-point camera and the predicted spatial location. The predicted spatial location is determined to be unobstructed and within the true field of view of the low-point camera.

[0054] By combining inclusion determination and occlusion detection into a two-level field of view determination mechanism, the ray cross algorithm is first used for preliminary screening within the angular range, and then the depth map is used for accurate verification of actual visibility. This effectively avoids the defect of misjudging targets in the occluded area as cameraable targets by relying solely on the angular range determination. It ensures that the field of view determination result can truly reflect whether the low-point camera can actually capture the target object, providing a reliable decision basis for subsequent accurate selection of the target low-point camera for pre-linkage control.

[0055] Step S4: If the target is not within the field of view of any low-point camera, update the target time and reacquire the latest spatial coordinates of the target object to predict the trajectory until the predicted spatial position is obtained within the field of view of any low-point camera, and determine the target low-point camera.

[0056] The process of updating the target time and re-acquiring the latest spatial coordinates of the target object for trajectory prediction includes: obtaining the initial coordinate value of the target object at the current time; using a Kalman filter algorithm to perform state extrapolation on the initial coordinate value to obtain the predicted point of the target object at the target time; determining whether the predicted point is within the field of view range based on the predicted point and the pre-established field of view range value; if it is not within the field of view range value, performing time accumulation on the target time according to the preset sampling rate to determine the updated second target time; and using the optimal posterior state estimate as a benchmark, performing state extrapolation on the Kalman filter algorithm to obtain the predicted spatial position of the target object at the second target time.

[0057] Specifically, after obtaining the predicted spatial position of the target object at the target time, if it is determined that the predicted spatial position is not within the actual field of view of any low-point camera, it indicates that the target object has not yet entered the effective area that can be captured by the low-point camera at the current predicted target time. At this time, it is necessary to push the predicted time backward and re-predict the trajectory based on the new time to continuously track the movement trend of the target object until it enters the field of view of a certain low-point camera.

[0058] Specifically, when the iterative prediction process is triggered, the initial coordinates of the target object at the current moment are first obtained. The current moment is the actual moment of the high-point video frame observation, and the target moment is the future moment to be predicted after the current moment. There is a prediction time advance between the current moment and the target moment, which is preset by the system based on the response time required for the low-point camera linkage. When iterative prediction begins, the current moment corresponds to the latest observed moment in the first spatial coordinate sequence, and the target moment is the current moment plus the prediction time advance. The initial coordinates of the current moment are the spatial coordinates corresponding to the latest moment in the first spatial coordinate sequence. The Kalman filter algorithm is used to extrapolate the state of this initial coordinate value. The Kalman filter algorithm is an optimal state estimation algorithm based on a linear Gaussian state-space model. It estimates the true state of the system through two iterative stages: prediction and update. In this step, only the prediction stage is executed, not the update stage. That is, the state estimate at the current moment is extrapolated to the target moment using the state transition matrix to obtain the prior state estimate of the target object at the target moment. The position component in this prior state estimate is the predicted point of the target object at the target moment.

[0059] The state transition matrix is ​​a mathematical tool used in the Kalman filter algorithm to describe the evolution of the system state over time. Essentially, it is a square matrix that records the linear transformation relationships between variables in the state vector at adjacent time points. The state vector is a mathematical expression used in the Kalman filter algorithm to describe the current motion state of the target object. In this application, a six-dimensional state vector is used, containing the position and velocity components of the target object in the X, Y, and Z axes of the VGIS three-dimensional spatial coordinate system. That is, the state vector can be represented as... Where x, y, and z represent the spatial positions of the target object along the three coordinate axes, respectively. , , These represent the moving velocities of the target object along the three coordinate axes. The number of rows and columns of the state transition matrix is ​​equal to the dimension of the state vector, which is a 6×6 matrix in this application. Its rows correspond to the variables in the predicted state vector, and its columns correspond to the variables in the unpredicted state vector. Each element in the matrix represents the weight coefficient of the corresponding row variable affected by the corresponding column variable.

[0060] Taking a uniform motion model as an example, the diagonal elements corresponding to position in the state transition matrix are set to 1, indicating that the position at the new moment inherits the position information from the previous moment; the elements corresponding to velocity are also set to 1, indicating that the velocity remains constant under the uniform motion assumption; the off-diagonal elements located at the intersection of the position row and the velocity column are set to the time step Δt, indicating that the change in position is equal to the velocity multiplied by the time interval; the remaining elements are set to 0. Multiplying the state transition matrix by the current state vector yields the prior state estimate of the target object at the target moment. The basic principle of state extrapolation is: based on the target object's current position and velocity, the position and velocity of the target after a specified time interval are calculated through matrix multiplication of the state transition matrix and the state vector, thereby obtaining the predicted state for the future moment.

[0061] After obtaining the predicted point, it is determined whether the predicted point is within the pre-established field of view range value based on the predicted point and the pre-established field of view range value. The field of view range value is the boundary parameter in space of the true field of view range determined by the intrinsic and extrinsic parameter matrices of the low-point camera in the previous steps. If the determination result is that the predicted point is not within the field of view range value of any low-point camera, it indicates that the target object has not yet entered the effective area that can be captured by the low-point camera at the current prediction time. At this time, the target time needs to be shifted backward, specifically by performing time accumulation processing on the target time according to the preset sampling rate. The preset sampling rate is the time step preset by the system. Its value is adapted to the video frame rate of the high-point camera and the moving speed of the target object. It can be set to an integer multiple of the time interval between adjacent video frames to ensure that each time accumulation corresponds to the actual video frame acquisition time. By accumulating a sampling rate time step on the target time, an updated second target time is determined. This second target time is shifted backward on the time axis by the preset time interval compared to the original target time.

[0062] During the iteration process, since the second target time is a future time that has not yet occurred, the system cannot obtain the corresponding video frame as an observation. Therefore, the optimal posterior state estimate output by the Kalman filter is used as the benchmark. The optimal posterior state estimate is the optimal state estimate obtained after the Kalman filter update stage based on the latest observation (i.e., the spatial coordinates corresponding to the latest time in the first spatial coordinate sequence). It includes the optimal position and velocity estimates of the target object at the current time. The Kalman filter algorithm only performs the state prediction stage. The optimal posterior state estimate is extrapolated from the current time to the second target time using the state transition matrix. That is, the time step of the prediction stage is equal to the time difference between the second target time and the current time. The state transition matrix is ​​multiplied by the optimal posterior state estimate at the current time to calculate the prior state estimate of the target object at the second target time. The position component in this prior state estimate is the predicted spatial position of the target object at the second target time.

[0063] Through the iterative loop of the aforementioned time accumulation and Kalman filter state prediction phase, the prediction time is shifted forward by one sampling rate step each time, and the same optimal posterior state estimate is used as the starting benchmark to continuously extrapolate forward, obtaining the predicted spatial position at each updated target time. After obtaining a new predicted spatial position in each iteration, the aforementioned field-of-view determination method is used, i.e., inclusion determination is performed using the ray crossing algorithm and occlusion detection is performed using the depth map, to determine whether the predicted spatial position is within the true field of view of any low-point camera. If the predicted spatial position obtained in the current iteration is still not within the true field of view of any low-point camera, the current prediction time is accumulated again at the preset sampling rate, and the pure extrapolation operation of the Kalman filter prediction phase is executed again to enter the next iteration. During the entire iteration process, no observations of future times are acquired, nor is the Kalman filter update phase executed. When the predicted spatial position obtained after a certain iteration is confirmed by the field-of-view determination to be within the true field of view of a certain low-point camera, the iteration loop is terminated, and that low-point camera is identified as the target low-point camera. If the iteration continues beyond the system's preset maximum prediction time boundary and still does not fall within any field of view, the current prediction process will be terminated and the system will wait for the next detection cycle.

[0064] When the predicted spatial position obtained after a certain iteration is confirmed by the field of view judgment to be within the actual field of view of a certain low point camera, it indicates that the target object will enter the spatial area that the low point camera can actually capture at the corrected target time. At this time, the iteration loop is terminated and the low point camera is identified as the target low point camera.

[0065] If the predicted spatial location is within the field of view of multiple low-point cameras, the low-point camera with the smallest straight-line distance from the predicted spatial location to the optical center of each low-point camera is selected as the target low-point camera.

[0066] Specifically, in actual monitoring scenarios, multiple low-point cameras are typically deployed within the monitored area. The fields of view of each low-point camera are spatially connected or partially overlap, collectively forming a fine-grained ground coverage network for the monitored area. Therefore, after obtaining the updated predicted spatial location and confirming through field-of-view judgment that it is within the actual field of view of multiple low-point cameras, it is necessary to select the optimal low-point camera as the target low-point camera to ensure the optimal viewing angle and best imaging clarity for coordinated tracking.

[0067] For each low-point camera identified as containing the predicted spatial location, the optical center coordinates of each low-point camera in the VGIS 3D spatial coordinate system are extracted. The spatial straight-line distance between the 3D coordinates of the predicted spatial location and each optical center coordinate is calculated. This spatial straight-line distance is obtained by summing the squares of the differences between the predicted spatial location and the optical center coordinates along the X, Y, and Z axes, and then taking the square root. This reflects the distance from the predicted spatial location to each low-point camera. After calculating the spatial straight-line distances between the predicted spatial location and each low-point camera, all calculated spatial straight-line distances are compared, and the low-point camera with the smallest distance value is selected as the target low-point camera. The closest camera has the shortest object distance, which allows for higher imaging resolution and clearer detail information under the same focal length conditions. Simultaneously, a smaller rotation angle can shorten the gimbal response time, improving the speed and accuracy of the linkage response.

[0068] Through the aforementioned iterative prediction mechanism, the system can continuously track the movement trend of the target object before it enters the field of view of the low-point camera, and continuously update the prediction results until the linkage triggering conditions are met, thus ensuring the timeliness and continuity of high-low point linkage control.

[0069] Step S5: Extract the device coordinates of the target low-point camera, calculate the relative spatial vector and straight-line distance between the predicted spatial position and the device coordinates, and update the relative spatial vector and straight-line distance according to the mechanical response delay time of the target low-point camera.

[0070] The step S5, which calculates the relative spatial vector and straight-line distance between the predicted spatial location and the device coordinates, includes: obtaining the device coordinates of the target low-point camera in the VGIS three-dimensional spatial coordinate system; subtracting the three-dimensional coordinates of the predicted spatial location from the three-dimensional spatial coordinates of the device coordinates to obtain the relative spatial vector; and calculating the straight-line distance between the predicted spatial location and the device coordinates based on the three coordinate axis directional components of the relative spatial vector.

[0071] Specifically, after identifying the target low-point camera, the device coordinates of the target low-point camera in the VGIS 3D spatial coordinate system are extracted. These device coordinates include the 3D spatial position coordinates of the target low-point camera, i.e., its installation position values ​​along the X, Y, and Z axes in the VGIS 3D spatial coordinate system, as well as its initial horizontal orientation angle and initial vertical pitch angle. This device coordinate information has been measured, calibrated, and stored in the device parameter library of the VGIS 3D spatial coordinate system during system deployment. The updated predicted spatial position in the VGIS 3D spatial coordinate system is then obtained, i.e., the X, Y, and Z axis coordinate values ​​corresponding to the predicted position of the target object at the target time.

[0072] The predicted spatial position's 3D coordinates are subtracted from the device's 3D spatial position coordinates. Specifically, the X-coordinate of the predicted spatial position is subtracted from the X-coordinate of the device coordinates along the X-axis, the Y-coordinate of the predicted spatial position is subtracted from the Y-coordinate of the device coordinates along the Y-axis, and the Z-coordinate of the predicted spatial position is subtracted from the Z-coordinate of the device coordinates along the Z-axis. These three coordinate differences form a coordinate difference vector, which serves as the relative spatial vector between the predicted spatial position and the device coordinates. This relative spatial vector reflects the orientational offset of the predicted spatial position relative to the target low-point camera in 3D space, with its direction pointing from the target low-point camera to the predicted spatial position. This vector is used to subsequently determine the pan-tilt angle and vertical pitch angle of the target low-point camera.

[0073] After obtaining the relative spatial vector, the straight-line distance between the predicted spatial position and the device coordinates is calculated based on the three coordinate axis directional components of the relative spatial vector. Specifically, the component values ​​of the relative spatial vector in the X-axis, Y-axis and Z-axis directions are taken respectively, the squares of the three components are summed, and the square root of the summation result is taken to obtain the straight-line distance from the predicted spatial position to the optical center of the target low-point camera. This straight-line distance is used as the basis for subsequent calculation of the focal length parameters of the target low-point camera.

[0074] Using the VGIS 3D spatial coordinate system as a unified spatial reference, the spatial relationship between the predicted spatial location and the target low-point camera is quantified into a relative spatial vector and a straight-line distance. The relative spatial vector describes the spatial orientation of the predicted location relative to the camera, directly supporting the calculation of subsequent gimbal angle adjustment parameters. The straight-line distance describes the spatial distance from the predicted location to the camera, directly supporting the calculation of subsequent focal length parameters. This provides complete and quantified data input for generating precise control commands for the target low-point camera.

[0075] In step S5, updating the relative spatial vector and straight-line distance based on the mechanical response delay time of the target low-point camera includes: obtaining the mechanical response delay time of the target low-point camera, which includes the gimbal horizontal rotation delay time, the gimbal vertical pitch delay time, and the lens zoom delay time; correcting the target time based on the mechanical response delay time to obtain the corrected linkage time; re-obtaining the predicted spatial position of the target object based on the corrected linkage time, and recalculating the relative spatial vector and straight-line distance based on the re-obtained predicted spatial position.

[0076] Specifically, after obtaining the relative spatial vector and straight-line distance, due to the mechanical response delay between the target low-point camera receiving the control command and actually completing the gimbal rotation and lens zoom, and the target object still being in continuous movement during this delay, if the control command is directly generated according to the current predicted spatial position, the target object will have moved to a new position by the time the low-point camera actually executes the command, causing the field of view center of the low-point camera to deviate from the actual position of the target object. To solve this problem, the mechanical response delay time of the target low-point camera is obtained.

[0077] Mechanical response delay time refers to the physical time required from when the target low-point camera receives a control command to when the gimbal actually rotates to the target angle and the lens actually zooms to the target focal length. This includes the gimbal's horizontal rotation delay time, vertical tilt delay time, and lens zoom delay time. The horizontal rotation delay time and vertical tilt delay time depend on the rated speed of the gimbal motor and the angle difference between the current angle and the target angle; the larger the angle difference, the longer the required rotation time. The lens zoom delay time depends on the response speed of the zoom motor and the focal length difference between the current focal length and the target focal length; the larger the focal length difference, the longer the required zoom time. These delay times can be obtained in two ways: firstly, as fixed values ​​pre-stored in the system parameter configuration module, which are measured during the system deployment and calibration phase based on the rated speed of the gimbal motor and the rated response speed of the lens zoom motor of the target low-point camera; secondly, as dynamic values, calculated in real time based on the angle difference between the current gimbal angle and the target angle, and the focal length difference between the current focal length and the target focal length. The larger the angle difference or the focal length difference, the longer the corresponding delay time. After obtaining the mechanical response delay time, the target time is corrected based on this mechanical response delay time, that is, the mechanical response delay time is added as a compensation to the original target time to obtain the corrected linkage time. This corrected linkage time represents the moment when the target low-point camera actually completes the mechanical action and can achieve stable imaging.

[0078] The trajectory prediction is re-executed based on the corrected linkage time. Specifically, in the VGIS 3D spatial coordinate system, starting with the current motion state of the target object, the spatial position of the target object at that moment is recalculated according to the corrected linkage time, resulting in an updated predicted spatial position. Based on the updated predicted spatial position, the device coordinates of the target's low-point camera are re-extracted, and the coordinate difference and straight-line distance between the predicted spatial position and the device coordinates are calculated again. The updated relative spatial vector and straight-line distance replace the previous relative spatial vector and straight-line distance. Through this delay compensation process, the angle adjustment parameters and focal length parameters used to generate control commands strictly correspond to the spatial position of the target object at the actual arrival time of the low-point camera, rather than the predicted position of the target object at the moment the command is issued. This effectively eliminates the impact of mechanical response delay on linkage accuracy, ensuring that the low-point camera is precisely aligned with the actual arrival position of the target object at the moment the mechanical action is completed after the control command is issued, achieving precise matching of high and low point linkage in the time dimension.

[0079] Step S6: Determine the angle adjustment parameters and focal length parameters of the target low-point camera based on the updated relative space vector and straight-line distance, and generate control commands for the target low-point camera.

[0080] Step S6 includes: calculating the yaw and pitch angle adjustments of the target low-point camera using inverse trigonometric functions based on the components of the relative spatial vector along the three coordinate axes of the VGIS three-dimensional spatial coordinate system, and using the yaw and pitch angle adjustments as angle adjustment parameters of the target low-point camera; determining the focal length adjustment value of the target low-point camera based on the straight-line distance and in conjunction with the image sensor size parameters of the target low-point camera, and using the focal length adjustment value as the focal length parameter of the target low-point camera; and encapsulating the angle adjustment parameters and focal length parameters according to a preset communication protocol format to generate control commands for the target low-point camera.

[0081] After obtaining the relative spatial vector and straight-line distance of the target low-point camera, these spatial geometric quantities need to be converted into mechanical control parameters that the target low-point camera can execute, so as to drive the gimbal rotation and lens zoom of the target low-point camera, so that the target low-point camera can complete alignment and focus before the target object reaches the predicted spatial position, thereby achieving seamless relay capture of the target object.

[0082] Specifically, based on the components of the relative spatial vector along the three coordinate axes of the VGIS 3D spatial coordinate system, the yaw and pitch angle adjustments of the target low-point camera are calculated using a four-quadrant arctangent function. The yaw angle adjustment refers to the angle the target low-point camera needs to rotate horizontally. It is calculated using the two components of the relative spatial vector in the horizontal plane, specifically the components along the X and Y axes. Using the X-axis component as the independent variable and the Y-axis component as the dependent variable, the yaw angle adjustment is calculated using a four-quadrant arctangent function. The pitch angle adjustment refers to the angle the target low-point camera needs to rotate vertically. It is calculated using the composite component of the relative spatial vector in the horizontal plane and its component along the Z-axis. Specifically, the sum of the squares of the X and Y components of the relative spatial vector in the X and Y axes is first taken, and the square root is taken to obtain the composite distance in the horizontal plane. Then, the ratio of the Z-axis component to this composite distance is calculated, and the arctangent function is applied to this ratio to obtain the pitch angle adjustment of the target low-point camera. A positive value indicates upward rotation, and a negative value indicates downward rotation.

[0083] After completing the angle calculation, the yaw and pitch angle adjustments are limited. The purpose of this limitation is to ensure that the calculated results do not exceed the maximum rotation angle range allowed by the gimbal's mechanical structure at the target low-point camera. The gimbal has physical limits in both the horizontal and vertical directions. If the calculated adjustment exceeds these limits, directly driving the gimbal to rotate beyond the limit boundaries will cause mechanical damage or prevent control commands from being executed. The specific limitation method is as follows: the yaw angle adjustment is compared with the preset maximum permissible angle in the horizontal direction. If the yaw angle adjustment is greater than the maximum permissible angle, it is set to the maximum permissible angle; if the yaw angle adjustment is less than the minimum permissible angle, it is set to the minimum permissible angle. Similarly, the pitch angle adjustment is compared with the preset maximum and minimum permissible angles in the vertical direction, and any pitch angle adjustment exceeding the limits is truncated to the corresponding limit boundary value. The yaw angle adjustment and pitch angle adjustment after limiting are used together as the angle adjustment parameters for the target low-point camera. These angle adjustment parameters drive the gimbal of the target low-point camera to rotate in both the horizontal and vertical directions, aligning the central optical axis of the target low-point camera with the predicted spatial position. The dimensions of the maximum and minimum permissible angles in the horizontal direction, the maximum and minimum permissible angles in the vertical direction are all in degrees. Their specific values ​​are determined based on the physical limits of the gimbal's mechanical structure. They are obtained during system deployment by reading the gimbal's parameters or by actual rotation measurements and are pre-stored in the system parameter configuration module.

[0084] Simultaneously, the focal length parameter of the target low-point camera is determined based on the straight-line distance. The focal length parameter refers to the focal length value that the target low-point camera lens needs to be adjusted to, which determines the clarity and size ratio of the target object in the image. The focal length value is calculated based on the similar triangle projection principle: the actual physical height of the target object, the straight-line distance from the target object to the optical center of the camera, and the preset expected height ratio of the target object in the image are known. The actual physical height of the target object is determined by the target category output by the target recognition model. The corresponding height value preset by the system is called according to the target category, or the default average adult height is used. The straight-line distance is the spatial straight-line distance from the predicted spatial position to the optical center of the target low-point camera calculated in step S5. The expected height ratio is the ratio of the pixel height of the target object in the vertical direction of the image to the total number of pixels in the vertical direction of the image sensor. It is set according to the detail recognition requirements of the monitoring scene, and the value range is 10% to 30%. Substituting the above three known quantities into the similar triangle ratio relationship, the required focal length value can be deduced so that the image size of the target object in the image meets the preset expected height ratio when it reaches the predicted position. The farther the target is, the longer the focal length is required to compensate for the reduction in target size when imaging at a distance.

[0085] The calculated focal length adjustment value is used as the focal length parameter of the target low-point camera. This focal length parameter drives the zoom motor of the target low-point camera to adjust the lens focal length, ensuring that the target object appears at an appropriate size and sharpness in the image when it reaches the predicted spatial position. Optionally, before determining the focal length parameter, the current focal length value of the target low-point camera is also obtained. The calculated focal length adjustment value is compared with the current focal length value. If the absolute value of the difference between the two is less than a preset zoom threshold, the current focal length is kept unchanged to avoid image jitter and motor wear caused by frequent zooming. The zoom threshold is measured in millimeters, and its specific value is preset based on the response characteristics of the target low-point camera's zoom motor and the requirements for image stability in the monitoring scene. A typical value is a preset constant between one percent and five percent of the current focal length value.

[0086] After obtaining the angle adjustment parameters and focal length parameters, these parameters are encapsulated according to a preset communication protocol format to generate control commands for the target low-point camera. The preset communication protocol format refers to the pre-agreed data exchange format between the target low-point camera and the control system. This protocol specifies the data frame structure, field meanings, byte order, and verification methods for the control commands. Specifically, according to this protocol format, the device address of the target low-point camera, the yaw angle adjustment value, the pitch angle adjustment value, and the focal length parameter value are filled into the specified fields of the command frame. Verification fields are calculated according to the protocol requirements, and the completed data frame is encapsulated into a control command conforming to the protocol format. After generating the control command, it is sent to the target low-point camera through the communication interface, driving the target low-point camera to perform horizontal rotation and vertical pitch movements of the gimbal according to the angle adjustment parameters, and lens zoom movements according to the focal length parameters. This allows the target low-point camera to complete the pre-adjustment of its shooting posture and focal length before the target object reaches the predicted spatial position.

[0087] By using the above method, the spatial geometric relationship between the predicted spatial location and the target low-point camera is accurately quantified into pan-tilt angle control and focal length control quantities, and issued in the form of standardized control commands. This realizes the automatic conversion from spatial prediction results to equipment control commands, so that the high-low point linkage process does not require manual intervention, significantly improving the response speed and automation level of the monitoring system.

[0088] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0089] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0090] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for linking high and low point AR real-scene VGIS three-dimensional cloud-based alarm systems, characterized in that, The method includes: Step S1: Acquire continuous video frames captured by the high-point camera, extract the pixel coordinate sequence of the target object in the continuous video frames through the target recognition model, generate a two-dimensional movement trajectory and map it to the preset VGIS three-dimensional spatial coordinate system to obtain the first spatial coordinates corresponding to the two-dimensional movement trajectory; Step S2: Based on the first spatial coordinates and the spatial topology contained in the VGIS three-dimensional spatial coordinate system, a trajectory prediction algorithm is used to predict the trajectory of the target object's movement state, thereby obtaining the predicted spatial position of the target object at the target time. Step S3: Based on the VGIS three-dimensional spatial coordinate system, obtain the field of view of at least one low-point camera in the monitoring area, perform inclusion degree determination and occlusion detection on the predicted spatial location, and determine whether the predicted spatial location is within the field of view of any low-point camera. Step S4: If the target is not within the field of view of any low-point camera, update the target time and reacquire the latest spatial coordinates of the target object to predict the trajectory until the predicted spatial position is obtained within the field of view of any low-point camera, and determine the target low-point camera. Step S5: Extract the device coordinates of the target low-point camera, calculate the relative spatial vector and straight-line distance between the predicted spatial position and the device coordinates, and update the relative spatial vector and straight-line distance according to the mechanical response delay time of the target low-point camera; Step S6: Determine the angle adjustment parameters and focal length parameters of the target low-point camera based on the updated relative space vector and straight-line distance, and generate control commands for the target low-point camera.

2. The high-low point linkage AR real-scene VGIS three-dimensional cloud alarm linkage method according to claim 1, characterized in that, Step S1 generates the two-dimensional movement trajectory of the target object, including: Acquire continuous video frames captured by a high-position camera, and parse the continuous video frames using a video decoder to obtain a first video frame sequence; Based on the first video frame sequence, the pixel points of the target object in the first video frame sequence are extracted by the target recognition model to obtain the pixel coordinate sequence of the target object; For the pixel coordinate sequence, the difference in coordinate values ​​between adjacent pixels is calculated. If the difference in coordinate values ​​is greater than a preset difference threshold, the corresponding pixel is removed to obtain the cleaned target coordinate sequence. By connecting the coordinate values ​​in the cleaned target coordinate sequence in chronological order, the initial two-dimensional movement trajectory of the target object is generated. The average coordinates of each moving point in the initial two-dimensional trajectory line are calculated using the sliding window method, and a smoothed two-dimensional moving trajectory is generated based on the average coordinates.

3. The high-low point linkage AR real-scene VGIS three-dimensional cloud alarm linkage method according to claim 1, characterized in that, Step S1 obtains the first spatial coordinates corresponding to the two-dimensional movement trajectory, including: The two-dimensional movement trajectory contains multiple trajectory points. A coordinate mapping algorithm is used to perform transformation processing on the multiple trajectory points, and the two-dimensional movement trajectory is mapped to a preset VGIS three-dimensional spatial coordinate system to obtain the first spatial coordinates corresponding to the two-dimensional movement trajectory. Calculate the offset between the first spatial coordinates and the reference point. If the offset is greater than a preset judgment value, add a correction term to the first spatial coordinates to obtain the corrected first spatial coordinates.

4. The high-low point linkage AR real-scene VGIS three-dimensional cloud alarm linkage method according to claim 1, characterized in that, The predicted spatial location of the target object at the target time obtained in step S2 includes: Obtain the position of the first spatial coordinates in the VGIS three-dimensional spatial coordinate system, extract the spatial topology structure contained in the VGIS three-dimensional spatial coordinate system, and obtain the set of topology nodes; Based on the set of topological nodes, the initial positioning node of the target object is selected based on the first spatial coordinates to obtain the initial movement state of the target object; If the initial movement state meets the preset active condition, the Kalman filter algorithm is used to process the initial movement state to obtain a smooth movement state. The smooth movement state is matched with the spatial topology to determine the continuous movement path of the target object in the VGIS three-dimensional spatial coordinate system; The continuous movement path is processed using a trajectory prediction algorithm to obtain the initial predicted spatial position of the target object at the target time. If the initial predicted spatial position exceeds the coverage area of ​​the spatial topology, the initial predicted spatial position is corrected based on the edge nodes of the coverage area to obtain the final predicted spatial position of the target object at the target time.

5. The high-low point linkage AR real-scene VGIS three-dimensional cloud alarm linkage method according to claim 1, characterized in that, Step S3, determining whether the predicted spatial location is within the field of view of any low-point camera, includes: Obtain the intrinsic and extrinsic parameter matrices of the low-point camera, determine the coverage area of ​​the low-point camera in the VGIS three-dimensional spatial coordinate system, and extract the coordinates of multiple edge vertices from the coverage area to construct a three-dimensional projection surface; The ray crossing algorithm is used to test the intersection between the predicted spatial location and the three-dimensional projection surface. If the number of intersections is odd, it is determined that the predicted spatial location is within the field of view coverage of the low-point camera. Extract the depth map corresponding to the three-dimensional projection surface, and obtain the actual depth value corresponding to the predicted spatial position from the depth map; If the actual depth value is less than the distance from the predicted spatial position to the optical center of the low-point camera, then the predicted spatial position is determined to be occluded and removed from the field of view.

6. The high-low point linkage AR real-scene VGIS three-dimensional cloud alarm linkage method according to claim 1, characterized in that, Step S4 involves updating the target time and re-acquiring the latest spatial coordinates of the target object for trajectory prediction, including: The initial coordinates of the target object at the current time are obtained, and the Kalman filter algorithm is used to perform state extrapolation on the initial coordinates to obtain the predicted point of the target object at the target time. Based on the predicted point and the pre-established field of view range value, it is determined whether the predicted point is within the field of view range value. If it is not within the field of view range value, the target time is accumulated according to the preset sampling rate to determine the updated second target time. Based on the optimal posterior state estimate, the Kalman filter algorithm is used for state extrapolation to obtain the predicted spatial position of the target object at the second target time.

7. The high-low point linkage AR real-scene VGIS stereoscopic cloud alarm linkage method according to claim 1, characterized in that, Step S5, which calculates the relative spatial vector and straight-line distance between the predicted spatial location and the device coordinates, includes: Obtain the device coordinates of the target low-point camera in the VGIS three-dimensional spatial coordinate system; The relative spatial vector is obtained by subtracting the three-dimensional coordinates of the predicted spatial location from the three-dimensional spatial coordinates of the device coordinates. The straight-line distance between the predicted spatial location and the device coordinates is calculated based on the three coordinate axis directional components of the relative spatial vector.

8. The high-low point linkage AR real-scene VGIS stereoscopic cloud alarm linkage method according to claim 1, characterized in that, Step S5, which updates the relative spatial vector and the straight-line distance based on the mechanical response delay time of the target low-point camera, includes: The mechanical response delay time of the target low-point camera is obtained, and the mechanical response delay time includes the gimbal horizontal rotation delay time, the gimbal vertical pitch delay time, and the lens zoom delay time. The target time is corrected based on the mechanical response delay time to obtain the corrected linkage time; The predicted spatial position of the target object is reacquired based on the corrected linkage time, and the relative spatial vector and the straight-line distance are recalculated based on the reacquired predicted spatial position.

9. The high-low point linkage AR real-scene VGIS three-dimensional cloud alarm linkage method according to claim 1, characterized in that, Step S6 includes: Based on the components of the relative spatial vector in the three coordinate axes of the VGIS three-dimensional spatial coordinate system, the yaw angle adjustment and pitch angle adjustment of the target low point camera are calculated using inverse trigonometric functions, and the yaw angle adjustment and pitch angle adjustment are used as the angle adjustment parameters of the target low point camera. Based on the straight-line distance and in conjunction with the image sensor size parameters of the target low-point camera, the focal length adjustment value of the target low-point camera is determined, and the focal length adjustment value is used as the focal length parameter of the target low-point camera. The angle adjustment parameters and the focal length parameters are encapsulated according to a preset communication protocol format to generate control commands for the target low-point camera.

10. The high-low point linkage AR real-scene VGIS three-dimensional cloud alarm linkage method according to claim 1, characterized in that, If the predicted spatial location is within the field of view of multiple low-point cameras, then the low-point camera with the smallest straight-line distance from the predicted spatial location to the optical center of each low-point camera is selected as the target low-point camera.

Images