Intelligent park visual management method and medium

Abstract

This application discloses a smart park visualization management method: constructing a park BIM model; obtaining the installation parameters of each camera in the park; based on the installation parameters, constructing a corresponding view frustum model for each camera in the coordinate system of the BIM model; acquiring security alarm events; determining the spatial inclusion relationship between the spatial coordinates of the alarm point and each view frustum in the view frustum dataset, and filtering out cameras whose alarm point coordinates fall within the effective range of the view frustum; sorting the effective coverage camera set in ascending order according to the spatial distance between each camera and the alarm point, and retrieving the real-time video stream of the best-ranked camera; if the effective coverage set is empty, retrieving the real-time video stream of the camera with the closest spatial distance to the alarm point and generating a field-of-view limitation indicator; this application automatically filters and retrieves the real-time video stream of the camera with effective field-of-view coverage of the alarm point and the optimal distance and pushes it to the visualization dashboard interface.

Description

Technical Field

[0001] This application relates to the field of smart parks, and in particular to a method and medium for visual management of smart parks. Background Technology

[0002] With the advancement of urbanization, the construction of smart parks has become an important direction for improving the overall management level of parks. Park management involves the unified collection and real-time monitoring of multi-dimensional data, including security monitoring, energy consumption management, personnel and vehicle access, asset status, and network security posture. Existing technologies typically use a visual dashboard to present the above data in the form of charts such as line graphs, bar charts, and digital dashboards to show the overall situation. The system also marks the locations of devices such as cameras and access control on the 3D model of the park, allowing users to click to view real-time detailed information and data statistics. Some systems also combine voice control and AI semantic analysis to achieve automatic retrieval of equipment resources and automatic navigation of analysis pages.

[0003] In security monitoring scenarios, numerous cameras are deployed within a park to cover various areas. When a security alarm is triggered, the system needs to automatically locate the alarm point and push the corresponding real-time video feed to the visual dashboard interface. However, existing technologies typically retrieve video streams based solely on the spatial distance between the camera and the alarm point or a fixed device binding relationship during alarm response. This fails to comprehensively consider the constraints imposed by installation parameters such as the camera's orientation angle, tilt angle, field of view, and effective sensing distance on the actual field of view coverage. Consequently, the camera corresponding to the retrieved video stream may be unable to effectively identify the alarm target because its field of view does not cover the alarm point or the alarm point is beyond the camera's effective imaging distance. This hinders security personnel's timely assessment and handling of alarm events. Summary of the Invention

[0004] In response to the problem that existing technologies cannot automatically filter out cameras with effective field of view coverage of the alarm point after a security alarm is triggered, resulting in the inability of the retrieved video stream to effectively identify the alarm target, this application provides a smart park visualization management method and medium. By determining the spatial inclusion relationship between the spatial coordinates of the alarm point and the view frustum model constructed based on the camera installation parameters, the method automatically filters and retrieves the real-time video stream of the camera with effective field of view coverage of the alarm point and the optimal distance, and pushes it to the visualization cockpit interface.

[0005] One aspect of this application provides a smart park visualization management method, comprising: constructing a park BIM model; obtaining the installation parameters of each camera in the park, including the camera's spatial coordinates, orientation angle, pitch angle, field of view, and maximum effective sensing distance; the maximum effective sensing distance is the farthest spatial distance at which the camera can identify a target under the condition of meeting a preset imaging resolution; based on the installation parameters, constructing a corresponding frustum model for each camera in the coordinate system of the BIM model to obtain a frustum dataset; obtaining security alarm events, each security alarm event carrying the device ID that triggered the alarm and the corresponding spatial coordinates of the alarm point; determining the spatial inclusion relationship between the spatial coordinates of the alarm point and each frustum in the frustum dataset, filtering out cameras whose alarm point coordinates fall within the effective range of the frustum, and obtaining a set of cameras with effective coverage; sorting the cameras in the set of cameras with effective coverage in ascending order of spatial distance from the alarm point, and retrieving the real-time video stream of the camera with the best ranking; if the set of cameras with effective coverage is empty, retrieving the real-time video stream of the camera with the closest spatial distance to the alarm point and generating a field-of-view limitation marker; and jointly pushing the real-time video stream and the location marker of the alarm point in the BIM model to the visualization dashboard interface. Furthermore, a corresponding frustum model is constructed for each camera in the coordinate system of the BIM model, including: using the spatial coordinates of the camera as the vertices of the frustum; constructing an initial geometric model in the coordinate system of the BIM model based on the orientation angle, pitch angle, field of view angle, and maximum effective sensing distance; projecting and rendering the BIM model using the corresponding camera as the projection viewpoint based on the spatial coordinates, orientation angle, and pitch angle of the camera to obtain a BIM rendering outline; determining the static occlusion pixel area corresponding to each camera based on the BIM rendering outline and the images captured by the camera; and correcting the initial geometric model based on the static occlusion pixel area to obtain the frustum model of the corresponding camera. Furthermore, based on the BIM rendered contour map and the images captured by the cameras, the static occlusion pixel regions corresponding to each camera are determined, including: performing edge detection on multiple frames of images captured by the cameras within a preset time window to obtain the measured edge contour map corresponding to each frame; performing contour matching between the measured edge contour map of each frame and the BIM rendered contour map frame by frame to extract the pixel regions in the measured edge contour map of each frame that are inconsistent with the BIM rendered contour map; performing pixel-by-pixel frequency statistics on the inconsistent pixel regions in each frame within the preset time window, and determining the pixel regions whose frequency exceeds a preset frequency threshold as static occlusion pixel regions; Furthermore, within a preset time window, edge detection is performed on multiple frames of images captured by the camera to obtain the measured edge contour map corresponding to each frame. This includes: within the preset time window, calculating the motion saliency score of each frame based on the inter-frame pixel difference for multiple frames of images captured by the camera, and marking frames with motion saliency scores exceeding a preset motion threshold as dynamic interference frames; before performing edge detection on each frame, dynamic interference frames are removed, and edge detection is performed only on the remaining frames to obtain the measured edge contour map corresponding to each frame after eliminating dynamic interference. Furthermore, the initial geometric model is corrected based on the static occlusion pixel region to obtain the view frustum model of the corresponding camera. This includes: identifying adjacent cameras from the view frustum dataset that have spatial overlap with the initial geometric model of the current camera; selecting the camera with the shortest baseline distance from the current camera as the reference camera from the adjacent cameras; obtaining the extrinsic parameter matrices of the current camera and the reference camera; calculating the corresponding pixel coordinates of each pixel coordinate in the static occlusion pixel region in the reference camera image based on the extrinsic parameter matrices of the two cameras; calculating the disparity between each pixel coordinate in the static occlusion pixel region and its corresponding pixel coordinate based on the intrinsic parameter matrices of the current camera and the baseline distance between them, to obtain the depth value corresponding to each pixel coordinate; back-projecting the pixel coordinates of the static occlusion pixel region onto the coordinate system of the BIM model based on the intrinsic parameter matrix of the current camera and the depth value corresponding to each pixel coordinate, to obtain the three-dimensional point set of the actual occlusion region; removing the spatial range enclosed by the three-dimensional point set of the actual occlusion region from the initial geometric model to obtain the corrected geometric model, which serves as the view frustum model of the current camera. Furthermore, the spatial inclusion relationship between the alarm point spatial coordinates and each view frustum in the view frustum dataset is determined, and cameras whose alarm point coordinates fall within the effective range of the view frustum are selected to obtain a set of cameras with effective coverage. This includes: constructing a corresponding axis-aligned bounding box for each view frustum model based on the spatial range of each view frustum model in the view frustum dataset; using the axis-aligned bounding boxes of each view frustum model as leaf nodes, merging them from bottom to top according to spatial position relationships to construct a hierarchical bounding volume tree; and performing layer-by-layer spatial inclusion determination on the alarm point spatial coordinates based on the hierarchical bounding volume tree to obtain a set of cameras with effective coverage. Furthermore, based on the hierarchical bounding volume tree, a layer-by-layer spatial inclusion judgment is performed on the spatial coordinates of the alarm points to obtain the set of cameras with effective coverage. This includes: obtaining the trigger timestamp of the security alarm event; comparing the trigger timestamp with the most recent update timestamp of the view frustum model corresponding to each leaf node in the hierarchical bounding volume tree; selecting leaf nodes whose most recent update timestamp is later than the most recent construction timestamp of the hierarchical bounding volume tree as leaf nodes to be verified; performing consistency verification on the hierarchical bounding volume tree based on the leaf nodes to be verified to obtain the updated hierarchical bounding volume tree; and comparing the spatial coordinates of the alarm points with the updated hierarchical bounding volume tree. The root node's bounding box is used to determine point inclusion. If the spatial coordinates of the alarm point fall within the current node's bounding box, the child nodes are recursively traversed downwards, and the point inclusion determination is repeated for each child node's bounding box until a leaf node is reached. If the spatial coordinates of the alarm point do not fall within the current node's bounding box, the corresponding node and all its corresponding child nodes are pruned and skipped. For the view frustum model that passes the point inclusion determination of the leaf node's bounding box, the corresponding camera is added to the set of valid covered cameras using the camera device ID corresponding to the view frustum model as an index. The updated hierarchical bounding volume tree is traversed, and the set of valid covered cameras is output. Furthermore, a consistency check is performed on the hierarchical bounding volume tree based on the leaf nodes to be checked, resulting in an updated hierarchical bounding volume tree. This includes: recalculating the axis-aligned bounding box of the current spatial extent of the corresponding frustum model for each leaf node to be checked; comparing the boundary coordinates of the recalculated axis-aligned bounding box with the existing bounding box of the node to be checked; if the boundary coordinate deviation exceeds a preset deviation threshold, the existing bounding box of the node to be checked is replaced with the recalculated axis-aligned bounding box, and the bounding box of each node is gradually recalculated upwards along the node to be checked until the root node; otherwise, the existing bounding box is retained unchanged; after checking all leaf nodes to be checked, the updated hierarchical bounding volume tree is obtained. Furthermore, the real-time video stream and the location markings of the alarm point in the BIM model are pushed together to the visualization cockpit interface, including: using the spatial coordinates of the alarm point as the center, rendering the park scene within the preset radius of the alarm point according to the three-dimensional spatial structure of the BIM model to obtain a three-dimensional scene view, and pushing the real-time video stream in a picture-in-picture format onto the three-dimensional scene view to the visualization cockpit interface. Another aspect of this application provides a computer-readable storage medium that stores computer instructions that, when executed by a processor, implement the method of this application.

[0006] Compared to existing technologies, the advantages of this application are: This application maps the spatial coordinates, orientation angle, pitch angle, field of view, and maximum effective sensing distance of the camera to the BIM model coordinate system, constructing a view frustum model that corresponds one-to-one with the actual effective sensing space of the camera. This allows the directional, angular, and distance constraints of the camera to be explicitly quantified in the form of geometric boundaries. Based on this, the determination of the effective coverage of the alarm point by the camera is transformed from a qualitative, empirical judgment to a calculation of the spatial inclusion relationship between the spatial coordinates of the alarm point and the geometric boundaries of the view frustum. Simultaneously, the maximum effective sensing distance is introduced as a constraint on the far-end section of the view frustum, excluding spatial ranges exceeding the effective imaging distance from the effective range of the view frustum. This ensures that the cameras selected through spatial inclusion judgment not only cover the alarm point in their field of view but also that the alarm point is within the effective recognition distance of the camera that satisfies the preset imaging resolution. Furthermore, the set of cameras with effective coverage is arranged in ascending order of spatial distance, prioritizing the retrieval of the video stream from the camera closest to the alarm point. This ensures that the real-time video stream finally pushed to the visual cockpit interface guarantees both the effectiveness of the field of view coverage and the imaging resolution. Attached Figure Description

[0007] This application will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein: Figure 1 This is an exemplary flowchart illustrating a smart park visualization management method according to some embodiments of this application; Figure 2 This is an exemplary flowchart of a modified view frustum dataset according to some embodiments of this application; Figure 3 This is an exemplary flowchart illustrating the construction of a current camera view frustum model according to some embodiments of this application; Figure 4 This is an exemplary flowchart illustrating the construction of an effective set of cameras according to some embodiments of this application; Figure 5 This is an exemplary flowchart illustrating the acquisition of an effective set of covered cameras according to some embodiments of this application. Detailed Implementation

[0008] The methods and systems provided in the embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0009] like Figure 1As shown, the smart park visualization management method of this application is aimed at the comprehensive management scenario of parks, and aims to solve the problem that after a security alarm is triggered, it is impossible to automatically filter out cameras with effective field of view coverage of the alarm point, resulting in the inability of the retrieved video stream to effectively identify the alarm target. In practical applications, the park management platform needs to uniformly collect multi-dimensional data such as security monitoring alarm data, energy consumption overview data, personnel and vehicle access statistics, asset status overview, and network security situation, and present the overall situation in the visualization dashboard interface through charts such as line charts, bar charts, and digital dashboards; the overall BIM model of the park (including the park's facade) is embedded in the middle of the dashboard, and the locations of equipment such as cameras and access control are marked on the model. Managers can click on the locations to view real-time detailed information and data statistics; at the same time, combined with voice control and AI semantic analysis, it supports the automatic retrieval of surrounding equipment resource data or corresponding analysis pages through natural language commands, such as turning on the camera 50m away from the east gate of the park or turning on the energy consumption dashboard. The core method of this application relies on the aforementioned park management platform. By constructing a park BIM model, building a view cone model for each camera, and filtering effectively covered cameras based on the spatial inclusion relationship between the view cone and the alarm point, the optimal video stream and alarm location markers are automatically pushed to the visual cockpit interface for joint display after the security alarm is triggered.

[0010] Step S100: Construct the park's BIM model: In this embodiment, a BIM model of the park is constructed using professional modeling software (such as Revit, ArchiCAD, etc.) based on the park's as-built drawings and measured data. The model's accuracy level is no less than LOD300 to ensure the spatial positioning accuracy of the subsequent view frustum models of each camera. After the BIM model is constructed, it is imported into the park management platform as a 3D base map for the visual dashboard interface. This base map is used to carry the spatial annotations of equipment locations such as cameras and access control systems, as well as the coordinate system reference benchmark for subsequent view frustum models.

[0011] Step S200: Obtain the installation parameters of each camera in the park: Obtain the installation parameters of each camera in the park. The installation parameters include the camera's spatial coordinates, orientation angle, pitch angle, field of view, and maximum effective sensing distance. The maximum effective sensing distance is the farthest spatial distance at which the camera can identify a target under the condition of meeting the preset imaging resolution.

[0012] Spatial coordinates: The three-dimensional coordinates (x, y, z) of the camera lens optical center in the BIM model coordinate system, obtained through on-site measurement or by reading the equipment installation position from the BIM model, are used to determine the vertex position of the view frustum model.

[0013] Orientation angle: The angle between the projection direction of the camera's optical axis in the horizontal plane and the north direction (or specified reference direction) of the BIM model coordinate system. The value ranges from 0° to 360° and is obtained by measuring with a compass on site or by marking on the installation drawings.

[0014] Pitch angle: The angle between the camera's optical axis and the horizontal plane. Downward is a negative value, and upward is a positive value. It is measured on-site with a protractor or obtained by marking on the installation drawings. It is used to determine the tilt angle of the viewing cone in the vertical direction.

[0015] Field of view: The horizontal field of view of the camera lens (in some embodiments, it also includes the vertical field of view), which is obtained from the camera hardware specifications and is used to determine the angular range of the view frustum model.

[0016] Maximum effective sensing distance: The farthest spatial distance at which the camera can identify a target while meeting a preset imaging resolution. Within this distance, the pixel percentage of the target in the image captured by the camera must not be less than a preset minimum pixel density threshold (e.g., the pixel height of the target human body in the image must not be less than 50 pixels). Beyond this distance, the recognition requirement cannot be met. The maximum effective sensing distance is obtained by jointly calibrating the camera hardware parameters (focal length, sensor resolution) and the preset imaging resolution requirement. It is used to determine the position of the far-end section of the frustum model, excluding the spatial range beyond the effective imaging distance from the effective range of the frustum.

[0017] In this embodiment, several cameras are deployed within the park. The installation parameters of each camera are calibrated and entered by engineers after installation and stored in the equipment information database of the park management platform, managed using the camera device ID as an index. Before constructing the frustum model, the installation parameters of each camera are uniformly read from the equipment information database as input for subsequent frustum model construction. For example, taking the camera CAM-001 at the east entrance of the park as an example, its installation parameters are as follows: spatial coordinates are (12.5, 8.0, 4.2) meters, indicating that the optical center of the camera lens is located at 12.5 meters on the X-axis, 8.0 meters on the Y-axis, and 4.2 meters on the elevation in the BIM model coordinate system; the orientation angle is 185°, indicating that the projection direction of the camera's optical axis in the horizontal plane is deviated by approximately 5° towards due south; the pitch angle is -15°, indicating that the camera's optical axis is tilted downwards by 15°; the horizontal field of view is 92°, and the vertical field of view is 54°; the maximum effective sensing distance is calibrated to be 30 meters, meaning that within this distance, the pixel height of the target human body in the image captured by the camera is not less than 50 pixels, meeting the recognition requirements. The typical range of the above parameters is: orientation angle 0° to 360°, pitch angle -45° to 0°, horizontal field of view 60° to 120°, and maximum effective sensing distance 15 meters to 50 meters, with specific values ​​determined according to the camera model and installation location.

[0018] Step S300: Construct the view frustum dataset based on the installation parameters: like Figure 2 and Figure 3 As shown, the view frustum model is a three-dimensional geometric representation of the actual effective sensing space of the camera. It is a frustum-shaped geometric body constructed in the BIM model coordinate system, with the camera's spatial coordinates as vertices, the field of view angle as the subtended angle, and the maximum effective sensing distance as the distance to the far end section. The view frustum model explicitly quantifies the camera's directional constraints (facing angle, pitch angle), angular constraints (field of view angle), and distance constraints (maximum effective sensing distance) in the form of geometric boundaries. This allows the determination of the camera's field of view coverage for any spatial point to be transformed into the calculation of the spatial inclusion relationship between that spatial point and the view frustum's geometric boundary. Specifically, in this application, the depth information comes from measured images of adjacent cameras, fully utilizing the scene characteristics of multiple cameras overlapping in a park.

[0019] The view frustum dataset is a collection of view frustum models corresponding to all cameras in the park. It is organized and managed using camera device IDs as indexes and is used for batch judgment of the relationship between alarm points and the effective field of view coverage of each camera.

[0020] Step S310: Construct the view frustum model for each camera: In the coordinate system of the BIM model, a corresponding frustum model is constructed for each camera, including: using the spatial coordinates of the camera as the vertices of the frustum; constructing an initial geometric model in the coordinate system of the BIM model based on the orientation angle, pitch angle, field of view, and maximum effective sensing distance; projecting and rendering the BIM model with the corresponding camera as the projection viewpoint based on the spatial coordinates, orientation angle, and pitch angle of the camera to obtain a BIM rendering outline; determining the static occlusion pixel area corresponding to each camera based on the BIM rendering outline and the images captured by the camera; and correcting the initial geometric model based on the static occlusion pixel area to obtain the frustum model of the corresponding camera.

[0021] Specifically, the spatial coordinates (x, y, z) of the camera are used as the vertex of the viewing cone, i.e., the geometric starting point of the viewing cone. The unit direction vector of the camera's optical axis in the BIM model coordinate system is determined based on the orientation and pitch angles. Then, using the field of view angle as the subtended angle and the maximum effective sensing distance as the axial length, an initial frustum-shaped geometric model with the optical axis as the axis of symmetry is constructed as the initial geometric model. This initial geometric model represents the maximum effective sensing space range of the camera under ideal, unobstructed conditions.

[0022] In actual park environments, fixed components such as building walls, columns, and green vegetation can obstruct the camera's field of view, preventing some spatial areas in the initial geometric model from being effectively perceived by the camera. Therefore, using the corresponding camera as the projection viewpoint, the BIM model is projected and rendered based on its spatial coordinates, orientation angle, and pitch angle to obtain a BIM rendering outline. The BIM rendering outline is a projection of the park's buildings and components from the camera's perspective under ideal conditions (without considering actual dynamic objects). It reflects the theoretical edge distribution of each component in the BIM model on the camera's image plane and is used for subsequent comparison with actual camera images to identify static obstructions in the real environment that exceed the BIM model's expectations.

[0023] Step S320: Determine the static occlusion pixel area: Based on the BIM rendered outline map and the images captured by the cameras, the static occlusion pixel regions corresponding to each camera are determined, including: performing edge detection on multiple frames of images captured by the cameras within a preset time window to obtain the measured edge outline map corresponding to each frame; performing outline matching between the measured edge outline map of each frame and the BIM rendered outline map frame by frame to extract the pixel regions in the measured edge outline map of each frame that are inconsistent with the BIM rendered outline map; performing pixel-by-pixel frequency statistics on the inconsistent pixel regions in each frame within the preset time window, and determining the pixel regions whose frequency exceeds a preset frequency threshold as static occlusion pixel regions.

[0024] Static occlusion pixel regions refer to the set of pixels in the camera image coordinate system that are fixed occlusion objects (such as billboards, temporary buildings, equipment boxes, etc.) that are outside the modeling range of the BIM model in the actual environment. The spatial range corresponding to these pixel regions is actually imperceptible in the view frustum model and needs to be removed from the initial geometric model.

[0025] Specifically, the preset time window is the time interval used to collect camera image samples. In this embodiment, it is set to 24 hours to cover image samples under different lighting conditions and reduce the impact of lighting changes on edge detection results. Within the preset time window, camera images are collected at a fixed sampling interval (e.g., one frame every 5 minutes). The Canny algorithm is used to perform edge detection on each frame to obtain the measured edge contour map corresponding to each frame. The measured edge contour map reflects the actual edge distribution of all objects in the scene at the camera's viewpoint at the corresponding time, including the edges of modeled components in the BIM model and the edges of unmodeled occluders in the actual environment.

[0026] The measured edge contour maps of each frame are matched pixel-by-pixel with the BIM rendered contour maps. Pixel regions in each frame where the edge pixel positions in the measured edge contour maps are inconsistent with those in the BIM rendered contour maps are extracted. These are extra edge pixel regions that appear only in the measured edge contour maps but not in the BIM rendered contour maps, and are designated as inconsistent pixel regions for each frame. Inconsistent pixel regions may be caused by dynamic objects (pedestrians, vehicles) or static occlusions.

[0027] To distinguish between dynamic objects and static occlusions, pixel-by-pixel frequency statistics are performed on inconsistent pixel regions across all frames within a preset time window, counting the number of frames in which each pixel coordinate appears in the inconsistent pixel region. Dynamic objects have randomly changing positions over time, resulting in a lower frequency of occurrence for their corresponding pixel coordinates; static occlusions have fixed positions across frames, leading to a consistently high frequency of occurrence for their corresponding pixel coordinates. Therefore, pixel regions whose frequency exceeds a preset frequency threshold (set to 80% of the total number of sampled frames in this embodiment) are identified as static occlusion pixel regions for subsequent occlusion correction of the initial geometric model.

[0028] Step S321: Remove dynamic interference frames: Before performing edge detection on multiple frames of images captured by the camera within a preset time window, the motion saliency score of each frame is calculated based on the inter-frame pixel difference. Frames with motion saliency scores exceeding a preset motion threshold are marked as dynamic interference frames and removed. Edge detection is performed only on the remaining frames to obtain the measured edge contour map corresponding to each frame after eliminating dynamic interference.

[0029] Specifically, the sum of the absolute values ​​of pixel grayscale differences between each frame and its adjacent reference frames is calculated as the motion saliency score, reflecting the degree of change in image content between adjacent frames: the score is higher when there is a clear moving target in the image, and close to zero when the image is static. In this embodiment, the preset motion threshold is set to the mean of the sum of the absolute values ​​of the pixel grayscale differences of all images plus twice the standard deviation; taking a 1920×1080 resolution camera as an example, the inter-frame pixel difference result exceeds 5×10 6 Time markers are identified as dynamic interference frames and removed. This threshold can be set to 3×10 based on the actual background noise level. 6 Up to 8×10 6 Adjustments are made within the specified range. By removing dynamic interference frames, the interference of dynamic targets such as pedestrians and vehicles on edge detection results is effectively eliminated, improving the accuracy of static occluded pixel region recognition.

[0030] Step S330: Correct the initial geometric model based on binocular parallax: The initial geometric model is corrected based on the static occlusion pixel region to obtain the corresponding camera's frustum model. This includes: identifying adjacent cameras from the frustum dataset that have spatial overlap with the initial geometric model of the current camera, and selecting the camera with the shortest baseline distance as the reference camera; obtaining the extrinsic parameter matrices of the current camera and the reference camera, and calculating the corresponding pixel coordinates of each pixel coordinate in the static occlusion pixel region in the reference camera image based on the extrinsic parameter matrices; calculating the disparity of each pixel coordinate based on the intrinsic parameter matrix and the baseline distance, and obtaining the depth value corresponding to each pixel using the disparity-depth relationship formula (depth value = focal length × baseline distance / disparity value); back-projecting the pixel coordinates of the static occlusion pixel region to the BIM model coordinate system based on the intrinsic parameter matrix and the depth value to obtain the three-dimensional point set of the actual occlusion region; and removing the spatial range enclosed by the three-dimensional point set of the actual occlusion region from the initial geometric model to obtain the corrected frustum model.

[0031] Specifically, selecting the adjacent camera with the shortest baseline distance as the reference camera ensures stable geometric configuration for binocular parallax calculation and high depth estimation accuracy. The extrinsic parameter matrix describes the rotation and translation relationship between the camera coordinate system and the BIM model coordinate system, calculated from the camera installation parameters; the intrinsic parameter matrix describes internal optical parameters such as focal length, pixel size, and principal point offset, obtained from factory calibration data. For example, taking a camera pair with a focal length of 800 pixels and a baseline distance of 5 meters, if the parallax value of a certain occluded pixel is 40 pixels, the corresponding depth value = 800 × 5 / 40 = 100 meters; when the distance of the occluded object is about 10 meters, the parallax value can reach 400 pixels, and the depth estimation error is usually no more than ±0.5 meters. Based on the depth values ​​of each pixel, the three-dimensional point set in the BIM coordinate system is restored through back projection transformation. The spatial range enclosed by this set is the three-dimensional spatial area that is actually occluded by the current camera. After removing this area from the initial geometric model, the corrected frustum model simultaneously satisfies the constraints of directionality, angle, distance, and occlusion, and can accurately reflect the effective perception spatial range of the camera in the actual park environment.

[0032] Step S400: Obtain security alarm events: The system acquires security alarm events, each carrying the device ID that triggered the alarm and the corresponding spatial coordinates of the alarm point. Security alarm events are alarm information reported to the park management platform by security sensing devices within the park (including but not limited to door magnetic sensors, infrared detectors, smoke alarms, access control card readers, etc.) when they detect an abnormal state. These events carry the device ID that triggered the alarm and the spatial coordinates of the alarm point (i.e., the three-dimensional coordinates of the triggering device in the BIM model coordinate system, which are entered into the system and bound to the device ID during device installation, and automatically attached by the system when the alarm event is triggered). For example, when the door magnetic sensor (device ID: SENSOR-BW-03) on the west side of Building B in the park detects that the door has been abnormally opened, triggering an alarm, the alarm event includes the device ID "SENSOR-BW-03" and the spatial coordinates of the alarm point (35.2, 18.6, 1.0) meters. The end-to-end response latency from receiving the alarm event to pushing the video stream to the visual dashboard interface does not exceed 3 seconds. The spatial coordinates of the alarm point also serve as the basis for generating highlighted positioning marks in the BIM model, enabling managers to intuitively locate the spatial position where the alarm occurred.

[0033] Step S500: Filter the set of cameras that can effectively cover the area. like Figure 4 and Figure 5 As shown, the spatial coordinates of the alarm point are compared with the spatial inclusion relationships of each frustum in the frustum dataset to filter out cameras whose alarm point coordinates fall within the effective range of the frustum, thus obtaining a set of cameras with effective coverage. Cameras in the set of cameras with effective coverage have their field of view covering the alarm point, and the alarm point is within an effective recognition distance that meets the preset imaging resolution requirements, enabling effective imaging and recognition of the alarm target. Specifically, in filtering the set of cameras with effective coverage, this application uses a first-level hierarchical bounding volume tree (BVH) to spatially index the frustum dataset, quickly eliminating frustums that cannot possibly contain the alarm point through recursive traversal and pruning, narrowing the precise judgment range to at least a few candidate frustums. The second level performs surface normal vector half-space judgment on the candidate frustums filtered by BVH to complete the precise point inclusion determination. This two-level structure solves both the problem of point inclusion judgment for irregular frustums and the problem of low efficiency in sequential traversal in multi-camera scenarios within a park.

[0034] Step S510: Spatial inclusion determination based on hierarchical bounding volume tree: The spatial inclusion relationship between the spatial coordinates of the alarm point and each view frustum in the view frustum dataset is determined, including: constructing a corresponding axis-aligned bounding box (AABB) for each view frustum based on the spatial range of each view frustum model; using each axis-aligned bounding box as a leaf node, constructing a hierarchical bounding volume tree (BVH) from bottom to top according to the spatial position relationship; and performing layer-by-layer spatial inclusion determination on the spatial coordinates of the alarm point based on the hierarchical bounding volume tree to obtain the set of cameras with effective coverage.

[0035] An axis-aligned bounding box is the smallest bounding cuboid parallel to the coordinate axes of the BIM model coordinate system, represented by six boundary coordinate values ​​(xmin, xmax, ymin, ymax, zmin, zmax). It is used to quickly exclude view frustums that cannot contain alarm points, reducing the computational load for accurate spatial inclusion determination. The hierarchical bounding box tree uses the axis-aligned bounding boxes of each view frustum as leaf nodes. View frustums of adjacent cameras are preferentially merged into the same parent node, and the parent node's bounding box is the smallest bounding box of all child node bounding boxes, and so on until the root node, organized as a hierarchical spatial index. Subtrees that do not contain alarm points can be pruned and skipped entirely, avoiding traversing all view frustums one by one. For example, with 100 cameras deployed in a park, the hierarchical bounding box tree depth is approximately 7 levels (log2100≈7). Each alarm response requires an average traversal of only about 20 to 30 nodes, reducing the time required for a single spatial inclusion determination by 60% to 80%, meeting the time requirements for real-time security alarm response.

[0036] Step S520: Layer-by-layer spatial inclusion determination with timestamp consistency check: Specifically, the topology of the BVH tree is generated all at once after the view frustum dataset is constructed, and its node grouping reflects the initial spatial distribution of the view frustum. If some view frustum models are deformed due to occlusion correction (weight 4), a deviation occurs between the bounding boxes of the original BVH tree nodes and the actual spatial extent of the view frustum, leading to missed or incorrect point inclusion judgments, but the system is unaware of this deviation. Therefore, before performing point inclusion judgment, this application performs a consistency check between the axis-aligned bounding boxes of the BVH tree leaf nodes and the current spatial extent of the corresponding view frustum model. If a deviation exceeding a preset threshold is detected, a local update is performed on the bounding boxes of the leaf node and its ancestor nodes, instead of a full reconstruction of the BVH tree. Details are as follows: Based on the hierarchical bounding volume tree, the spatial coordinates of alarm points are spatially contained layer by layer. This includes: obtaining the trigger timestamp of the security alarm event and comparing it with the most recent update timestamp of the view frustum model corresponding to each leaf node in the hierarchical bounding volume tree; selecting leaf nodes whose most recent update timestamp is later than the most recent construction timestamp of the hierarchical bounding volume tree as leaf nodes to be verified; performing consistency verification on the hierarchical bounding volume tree based on the leaf nodes to be verified to obtain the updated hierarchical bounding volume tree; performing point inclusion verification on the spatial coordinates of alarm points from the root node down layer by layer from top to bottom—if it falls within the current node's bounding box, recursively traversing the child nodes downwards; if it does not fall within, pruning and skipping the corresponding node and all its child nodes; further performing precise view frustum geometric boundary point inclusion verification on the view frustum model that passes the leaf node bounding box point inclusion verification; adding the cameras that pass the verification to the set of valid covered cameras using the device ID as the index; and outputting the set of valid covered cameras after traversal.

[0037] By comparing the trigger timestamp with the most recent update timestamp of each leaf node, leaf nodes that have undergone view frustum model updates after the hierarchical bounding volume tree is constructed can be identified. These leaf nodes are marked as leaf nodes to be verified for consistency checks, ensuring that the hierarchical bounding volume tree is in the latest state before spatial inclusion judgment, and avoiding misjudgment or omission due to view frustum model updates not being synchronized to the tree structure.

[0038] Step S521: Consistency check of the leaf nodes to be checked: The consistency of the hierarchical bounding volume tree is checked based on the leaf nodes to be checked, including: recalculating the axis-aligned bounding box of the current spatial extent of the corresponding frustum model for each leaf node to be checked; comparing the boundary coordinates of the recalculated axis-aligned bounding box with the existing bounding box of the node to be checked; if the boundary coordinate deviation between the two exceeds a preset deviation threshold, the existing bounding box is replaced with the recalculated axis-aligned bounding box, and the bounding boxes of each ancestor node are recalculated layer by layer upwards along the node to be checked until the root node; otherwise, the existing bounding box is retained unchanged; after checking all leaf nodes to be checked, the updated hierarchical bounding volume tree is obtained.

[0039] In this embodiment, the preset deviation threshold is set to 0.1 meters. That is, if the deviation between the recalculated axis-aligned bounding box and the existing bounding box in any boundary coordinate dimension exceeds 0.1 meters, the bounding box of the leaf node is considered to have undergone substantial changes and needs to be updated. After replacing the bounding box, since the parent node bounding box is the smallest bounding box of all its child node bounding boxes, the bounding boxes of each ancestor node need to be recalculated layer by layer upwards along the node to be verified until the root node, in order to ensure the spatial consistency of the entire hierarchical bounding body tree.

[0040] Step S600: Retrieve the optimal camera's real-time video stream: From the set of cameras with effective coverage, each camera is sorted in ascending order of its spatial distance (Euclidean distance) from the alarm point, and the real-time video stream of the camera with the best ranking is retrieved. If the set of cameras with effective coverage is empty, the real-time video stream of the camera with the closest spatial distance to the alarm point is retrieved, and a limited field of view indicator is generated. Provided that both effective field of view coverage and imaging resolution meet the requirements, closer distances mean higher pixel density and higher recognition accuracy for the camera's image of the alarm target. For example, for the door magnetic sensor alarm point (35.2, 18.6, 1.0), the set of cameras with effective coverage includes three cameras: CAM-001 (8.3 meters), CAM-005 (12.1 meters), and CAM-012 (19.7 meters). After being sorted in ascending order, the real-time video stream of CAM-001 is retrieved first and pushed to the visual cockpit interface.

[0041] When the effective coverage set of cameras is empty, the system resorts to retrieving the real-time video stream from the camera spatially closest to the alarm point as an auxiliary reference. Simultaneously, it generates a field-of-view limitation indicator (an orange warning icon superimposed on the upper right corner of the video stream image along with the text "Field of view limited, for reference only"), indicating to management personnel that the current video stream's coverage of the alarm point is limited and the image may not clearly identify the alarm target, requiring comprehensive analysis in conjunction with other information. For example, when the alarm point is located on level B2 of the underground parking lot (z-coordinate approximately -6.0 meters), and all cameras on this level have no corresponding valid records in the frustum data due to long-term offline status, the system will retrieve the video stream from the nearest camera at the entrance of level B1 (approximately 12 meters away) and simultaneously generate a field-of-view limitation indicator.

[0042] Step S700: Jointly push to the visual cockpit interface: The real-time video stream and the location markings of the alarm point in the BIM model are pushed together to the visualization cockpit interface. This includes: using the spatial coordinates of the alarm point as the center, rendering the park scene within a preset radius of the alarm point according to the three-dimensional spatial structure of the BIM model to obtain a three-dimensional scene view, and pushing the real-time video stream in a picture-in-picture format onto the three-dimensional scene view to the visualization cockpit interface.

[0043] Location markers are visual marker elements generated using the spatial coordinates of the alarm point as anchor points. They contain attribute information such as the 3D coordinates of the alarm point, the device ID that triggered the alarm, and the alarm time. Managers can click on the location markers to view alarm details. Centered on the spatial coordinates of the alarm point, a 3D scene within a preset radius (set to 50 meters in this embodiment) is extracted from the BIM model and rendered in real time to obtain a 3D scene view with the alarm point as the visual focus, which is then displayed as the main view. The real-time video stream is embedded in the lower right corner of the 3D scene view in a picture-in-picture format, allowing managers to simultaneously obtain the 3D spatial location information of the alarm point and the real-time image information from the camera on the same interface. This achieves a joint display of spatial positioning and image monitoring, completing a visual response loop for security alarm events.

[0044] In the visual cockpit interface, combined with voice control and AI semantic analysis, managers can also trigger the retrieval of video streams from cameras within a specified spatial range by using natural language commands (such as: turn on the camera 50m away from the east gate of the park). The system uses AI semantic analysis to parse the spatial range description in the command, convert it into a query range in the BIM model coordinate system, and retrieve the real-time video streams of each camera within the query range and push them to the cockpit interface; or automatically jump to the corresponding analysis page by command (such as: turn on the energy consumption cockpit).

[0045] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform an image processing method.

[0046] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct a computing device to perform the method of this application.

[0047] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A smart park visual management method, characterized in that, include: Build a BIM model of the park; Obtain the installation parameters of each camera in the park. The installation parameters include the camera's spatial coordinates, orientation angle, pitch angle, field of view, and maximum effective sensing distance. The maximum effective sensing distance is the farthest spatial distance at which the camera can identify a target under the condition of meeting the preset imaging resolution. Based on the installation parameters, a corresponding frustum model is constructed for each camera in the coordinate system of the BIM model to obtain the frustum dataset; Acquire security alarm events, which carry the device ID that triggered the alarm and the corresponding spatial coordinates of the alarm point; The spatial inclusion relationship between the spatial coordinates of the alarm point and each view frustum in the view frustum dataset is determined, and cameras whose alarm point coordinates fall within the effective range of the view frustum are selected to obtain the set of cameras with effective coverage. From the set of cameras with effective coverage, sort them in ascending order according to the spatial distance between each camera and the alarm point, and retrieve the real-time video stream of the camera with the best ranking. If the effective coverage set is empty, retrieve the real-time video stream from the camera that is spatially closest to the alarm point and generate a field-of-view limitation indicator; The real-time video stream and the location markings of alarm points in the BIM model are jointly pushed to the visual cockpit interface.

2. The smart park visualization management method according to claim 1, characterized in that: In the BIM model's coordinate system, construct a corresponding view frustum model for each camera, including: The spatial coordinates of the camera are used as the vertices of the view frustum; Based on the orientation angle, pitch angle, field of view angle, and maximum effective sensing distance, an initial geometric model is constructed in the coordinate system of the BIM model; Based on the spatial coordinates, orientation angle and pitch angle of the camera, the BIM model is projected and rendered using the corresponding camera as the projection viewpoint to obtain the BIM rendering outline. Based on the BIM rendered outline and the images captured by the cameras, determine the static occlusion pixel area corresponding to each camera; The initial geometric model is corrected based on the static occlusion pixel area to obtain the corresponding camera's view frustum model.

3. The smart park visualization management method according to claim 2, characterized in that: Based on the BIM rendered outline and images captured by cameras, determine the static occlusion pixel areas corresponding to each camera, including: Within a preset time window, edge detection is performed on multiple frames of images captured by the camera to obtain the measured edge contour map corresponding to each frame; The measured edge contour map of each frame is matched with the BIM rendered contour map frame by frame to extract the pixel areas in the measured edge contour map that are inconsistent with the BIM rendered contour map. Pixel-by-pixel frequency statistics are performed on inconsistent pixel regions in each frame within a preset time window, and pixel regions with frequencies exceeding a preset frequency threshold are identified as static occlusion pixel regions.

4. The smart park visualization management method according to claim 3, characterized in that: Within a preset time window, edge detection is performed on multiple frames of images captured by the camera to obtain the measured edge contour map corresponding to each frame, including: Within a preset time window, the motion saliency score of each frame is calculated based on the inter-frame pixel difference for multiple frames of images captured by the camera. Frames with motion saliency scores exceeding a preset motion threshold are marked as dynamic interference frames. Before performing edge detection on each frame, frames with dynamic interference are removed, and edge detection is performed only on the remaining frames to obtain the measured edge contour maps corresponding to each frame after eliminating dynamic interference.

5. The smart park visualization management method according to claim 3, characterized in that: The initial geometric model is corrected based on the static occlusion pixel region to obtain the corresponding camera's view frustum model, including: Identify adjacent cameras from the frustum dataset that have spatial overlap with the initial geometric model of the current camera, and select the camera with the shortest baseline distance to the current camera from the adjacent cameras as the reference camera; Obtain the extrinsic matrix of the current camera and the reference camera. Based on the extrinsic matrix, calculate the corresponding pixel coordinates of each pixel in the static occluded pixel region in the reference camera image. Based on the intrinsic parameter matrices of the current camera and the reference camera and the baseline distance between them, the disparity of each pixel coordinate and the corresponding pixel coordinate in the static occluded pixel region is calculated to obtain the depth value corresponding to each pixel coordinate. Based on the intrinsic parameter matrix of the current camera and the depth value corresponding to each pixel coordinate, the pixel coordinates of the static occluded pixel area are back-projected into the coordinate system of the BIM model to obtain the three-dimensional point set of the actual occluded area. The spatial extent enclosed by the three-dimensional point set of the actual occlusion area is removed from the initial geometric model to obtain the corrected geometric model, which serves as the current camera's view frustum model.

6. The smart park visualization management method according to claim 2, characterized in that: The spatial inclusion relationship between the spatial coordinates of the alarm point and each view frustum in the view frustum dataset is determined. Cameras whose alarm point coordinates fall within the effective range of the view frustum are selected, resulting in a set of cameras with effective coverage, including: Based on the spatial extent of each view frustum model in the view frustum dataset, construct a corresponding axis-aligned bounding box for each view frustum model; Using the axis-aligned bounding boxes of each view frustum model as leaf nodes, a hierarchical bounding volume tree is constructed by merging them from bottom to top according to their spatial positional relationships; Based on the hierarchical bounding volume tree, the spatial coordinates of the alarm points are determined layer by layer to obtain the set of cameras with effective coverage.

7. The smart park visualization management method according to claim 6, characterized in that: Based on the hierarchical bounding volume tree, the spatial coordinates of the alarm points are spatially contained layer by layer to obtain the set of cameras with effective coverage, including: Obtain the trigger timestamp of the security alarm event, compare the trigger timestamp with the most recent update timestamp of the view frustum model corresponding to each leaf node in the hierarchical bounding volume tree, and select the leaf nodes whose most recent update timestamp is later than the most recent construction timestamp of the hierarchical bounding volume tree as the leaf nodes to be verified. The consistency of the hierarchical bounding volume tree is checked based on the leaf nodes to be checked, and the updated hierarchical bounding volume tree is obtained. Perform a point inclusion check between the spatial coordinates of the alarm point and the bounding box of the root node of the updated hierarchical bounding body tree; if the spatial coordinates of the alarm point fall within the bounding box of the current node, recursively traverse the child nodes downwards and repeat the point inclusion check for the bounding boxes of each child node until the leaf node is reached; if the spatial coordinates of the alarm point do not fall within the bounding box of the current node, prune and skip the corresponding node and all its corresponding child nodes. For the view frustum model determined by the inclusion of the leaf node bounding box, the corresponding camera is added to the set of valid covered cameras using the camera device ID corresponding to the view frustum model as the index. Traverse the updated hierarchical bounding tree and output the set of cameras that are effectively covered.

8. The smart park visualization management method according to claim 7, characterized in that: Based on the leaf nodes to be verified, a consistency check is performed on the hierarchical bounding volume tree to obtain the updated hierarchical bounding volume tree, including: For each leaf node to be verified, recalculate the axis-aligned bounding box of the current spatial range of the corresponding view frustum model; The boundary coordinates of the recalculated axis-aligned bounding box are compared with the existing bounding box of the node to be verified. If the boundary coordinates of the two exceed the preset deviation threshold, the existing bounding box of the node to be verified is replaced with the recalculated axis-aligned bounding box, and the bounding boxes of each node are gradually recalculated upwards from the node to be verified until the root node. Otherwise, the existing bounding box is left unchanged. After verifying all the leaf nodes to be verified, the updated hierarchical bounding volume tree is obtained.

9. The smart park visualization management method according to any one of claims 2 to 8, characterized in that: The real-time video stream and the location markers of alarm points in the BIM model are jointly pushed to the visual cockpit interface, including: Centered on the spatial coordinates of the alarm point, the park scene within the preset radius of the alarm point is rendered according to the three-dimensional spatial structure of the BIM model to obtain a three-dimensional scene view. The real-time video stream is then overlaid on the three-dimensional scene view in a picture-in-picture format and pushed to the visualization cockpit interface.

10. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 9.

Images