Video GIS intelligent analysis method based on deep learning

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a video GIS intelligent analysis method based on deep learning, and the method comprises the steps: detecting a target in a video in real time through a predefined target detection model, and generating a space-time mark which comprises a bounding box and category information; thirdly, associating target tracks of different cameras by using a dynamic graph model and a dynamic graph neural network to form a space-time ID and a track chain; then, combining a visual inertial odometer and a geographic information system to calibrate a homography matrix, and mapping the trajectory chain to a geographic coordinate system to obtain space-time trajectory data; then, constructing a neural network model and a trajectory generation model, and respectively predicting crowd density distribution and pedestrian motion trajectories; and finally, carrying out real-time alarm according to the track and the multi-level geo-fencing rule. According to the invention, intelligent analysis of video data is realized, target tracking and alarm capabilities are improved, and the method is widely applicable to the fields of crowd management and safety monitoring.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a video GIS intelligent analysis method based on deep learning. Background Technology

[0002] Deep learning technology has achieved or even surpassed human accuracy in tasks such as image recognition and object detection, and is widely used, especially in security monitoring and smart logistics, providing efficient technical support for video analytics. GIS, as a geographic information spatial data management system, integrates geographic and statistical data to achieve spatial data visualization and analysis, playing a core role in smart cities, traffic management, and other scenarios. However, traditional GIS has limitations in processing dynamic video data, struggling to correlate geographic coordinates with video content in real time.

[0003] Traditional monitoring systems have the following problems:

[0004] 1. Isolated geographic information: Video data lacks deep integration with geospatial data, making it impossible to support multi-camera collaboration and spatial analysis.

[0005] 2. Lagging dynamic analysis: Lack of real-time prediction and early warning capabilities for population density trends and suspicious person trajectories. Summary of the Invention

[0006] Therefore, this invention provides a video GIS intelligent analysis method based on deep learning to solve the aforementioned problems existing in the prior art.

[0007] To achieve the above objectives, this invention provides a deep learning-based intelligent video GIS analysis method, comprising:

[0008] Step S1: Real-time detection of targets in the video using a predefined target detection model to generate spatiotemporal labels containing bounding boxes and category information;

[0009] Step S2: Construct a dynamic graph model based on the spatiotemporal markers and use a dynamic graph neural network to associate the target trajectories of different cameras to generate spatiotemporal IDs and spatiotemporally continuous trajectory chains;

[0010] Step S3: Based on the visual inertial odometry and the geographic information system calibration homography matrix, the trajectory chain is mapped to the geographic coordinate system in real time to obtain spatiotemporal trajectory data;

[0011] Step S4: Using the number of spatiotemporal trajectories as input parameters, construct a neural network model and a trajectory generation model respectively to predict the crowd density distribution and the movement trajectory of pedestrians to obtain the target density distribution and the target trajectory;

[0012] Step S5: Issue real-time alerts based on the target trajectory and preset multi-level geofencing rules.

[0013] Furthermore, the process of step S2 includes:

[0014] Using targets in the spatiotemporal markers as nodes, edges between nodes are constructed based on the spatiotemporal relationships of the targets to form a dynamic graph model;

[0015] The dynamic graph model is input into the dynamic graph neural network for processing to obtain updated node features and the association weight matrix between nodes;

[0016] Based on the association weight matrix, the Hungarian algorithm is used to calculate the trajectory segments belonging to the same target under different cameras, and a unique spatiotemporal ID is generated for each target;

[0017] The trajectory segments are connected according to time sequence and spatial relationship to form a spatiotemporally continuous trajectory chain.

[0018] Furthermore, the process of inputting the dynamic graph model into the dynamic graph neural network for processing to obtain updated node features and the association weight matrix between nodes includes:

[0019] Each node in the dynamic graph model is used to extract features as the initial input feature vector for the node in the dynamic graph neural network.

[0020] Collect the feature information of neighboring nodes based on the connection relationship between each node and its neighboring nodes;

[0021] The neighbor feature information is combined with the initial input feature vector, and the initial input feature vector is updated using an update function to obtain the target feature vector;

[0022] The correlation weight matrix between nodes is calculated using cosine distance based on the target feature vector.

[0023] Furthermore, the process of step S3 includes:

[0024] Feature points are extracted from continuous image sequences captured by a camera, and combined with angular velocity and acceleration data from an inertial measurement unit, a visual odometry model is constructed using a point-line feature fusion algorithm.

[0025] A robust optimization method is used to dynamically solve for camera motion parameters to generate a real-time updated homography matrix, which maps pixel coordinates to a local world coordinate system to obtain local coordinates.

[0026] The local coordinates are registered with the predefined ground control points in the geographic information system to convert the local coordinates into geographic coordinates.

[0027] The geographic coordinates are subjected to spatiotemporal smoothing and multi-source data are fused using a Kalman filter to generate spatiotemporal trajectory data.

[0028] Furthermore, the process of dynamically solving for camera motion parameters using a robust optimization method to generate a real-time updated homography matrix includes:

[0029] Feature points are extracted from consecutive frames of a video, and feature descriptors are calculated to obtain feature point pairs.

[0030] The initial homography matrix is ​​calculated using the direct linear transformation algorithm based on the matched feature point pairs;

[0031] Construct an optimization objective function that minimizes the reprojection error;

[0032] The homography matrix is ​​obtained by dynamically adjusting the homography matrix using an optimization algorithm based on the objective function and iteratively calculating the updated matrix value.

[0033] Furthermore, the process of step S4 includes:

[0034] To obtain historical spatiotemporal trajectory data, spatiotemporal trajectory data is obtained by acquiring spatiotemporal trajectory data within a past time period.

[0035] The population density distribution information within the divided geographical units and predefined time intervals is calculated based on the historical spatiotemporal trajectory data and used as the population density distribution label.

[0036] The spatiotemporal trajectory data is aggregated and normalized to generate an input tensor containing spatiotemporal information and population distribution characteristics;

[0037] Construct a training dataset using the input tensor and the crowd density distribution labels;

[0038] Long Short-Term Memory (LSTM) networks are used to extract features from the input tensor to learn the spatiotemporal variation patterns of population density distribution;

[0039] The model is trained using the spatiotemporal variation pattern and the training dataset to obtain a trained model;

[0040] The spatiotemporal trajectory data is input into the training model to obtain the target density distribution.

[0041] Furthermore, step S4 also includes:

[0042] The historical spatiotemporal trajectory data is preprocessed to extract the historical motion trajectory features of individuals;

[0043] The spatiotemporal trajectory data and the historical trajectory features are fused to obtain a fusion result;

[0044] The fusion result is used as input to construct the trajectory generation model;

[0045] The trajectory generation model is trained using the spatiotemporal trajectory data and the historical spatiotemporal trajectory data to obtain the target trajectory.

[0046] Furthermore, the process of step S5 includes:

[0047] Define multi-level geofences, each geofence including a unique identifier, geographic shape, spatial location coordinates and corresponding alarm level;

[0048] Extract trajectory points from the target trajectory, each trajectory point including a timestamp and geographic coordinates;

[0049] Determine whether the trajectory point enters or leaves the geofence;

[0050] When the trajectory point enters or leaves the geofence, an alarm of the corresponding level is triggered according to the preset multi-level geofence rules.

[0051] Furthermore, the process of determining whether the trajectory point enters or leaves the geofence includes:

[0052] Convert the multi-level fence into a list of vertex coordinates;

[0053] Transform the geographic coordinates of the trajectory points to the same coordinate system as the geographic fence;

[0054] An algorithm for determining the positional relationship between points and polygons is used to determine whether the trajectory points are located within the geofence.

[0055] Determine whether the timestamps of the trajectory points conform to the time rules of the geofence;

[0056] Record the matching results between the trajectory points and the geofence, update the matching status in real time, and record the corresponding alarm level.

[0057] Furthermore, step S5 also includes:

[0058] The alarm level is evaluated and adjusted in real time based on the dwell time and behavior patterns of the trajectory points within the geofence.

[0059] Alarm information is updated in real time to integrate alarm data from multiple geofences and different time periods. The alarm data and performance are analyzed to obtain a comprehensive alarm evaluation result.

[0060] Execute the corresponding alarm response strategy based on the alarm level and the comprehensive evaluation result.

[0061] Compared with existing technologies, the advantages of this invention are as follows: By constructing a dynamic graph model and applying a dynamic graph neural network, this invention effectively correlates target trajectories under different cameras, achieving spatiotemporally continuous trajectory chain generation and improving the accuracy and reliability of target tracking. Utilizing visual inertial odometry and geographic information system calibration, real-time mapping of trajectory chains to the geographic coordinate system is achieved, providing precise geographic information support for subsequent functions such as geofencing alarms. By predicting crowd density distribution and pedestrian movement trajectories, potential risks such as congestion and collisions can be detected in advance, allowing for timely measures to prevent accidents. Pre-setting multi-level geofencing rules enables real-time monitoring and alarming of different areas and alarm levels, improving adaptability to complex scenarios and management efficiency.

[0062] In particular, by constructing a dynamic graph model and utilizing a dynamic graph neural network, it is possible to more accurately associate target trajectories from different cameras, reducing false and missed associations. Employing the Hungarian algorithm for trajectory segment association can find the optimal solution in polynomial time, improving computational efficiency and meeting real-time requirements. Through feature extraction and updating, the dynamic graph neural network can adapt to changes in the target's appearance and motion state, improving the robustness of trajectory association.

[0063] In particular, by constructing a visual odometry model and employing a robust optimization method to dynamically solve the homography matrix, pixel coordinates can be mapped to the local world coordinate system more accurately, improving the accuracy of coordinate transformation. Registering local coordinates with ground control points in a geographic information system (GIS) enables accurate conversion of local coordinates to geographic coordinates, enhancing the reliability and usability of trajectory data. Spatiotemporal smoothing of geographic coordinates combined with Kalman filtering to fuse multi-source data generates smooth, continuous spatiotemporal trajectory data, reducing the impact of noise and errors and improving the quality of trajectory data.

[0064] In particular, by learning spatiotemporal variation patterns, the model can more accurately predict population density distribution, providing more reliable data support for resource allocation and risk warning. Trained using historical spatiotemporal trajectory data and corresponding labels, the model can learn the patterns of population density changes in different scenarios, improving its generalization ability. Processing and predicting real-time spatiotemporal trajectory data can provide timely population density distribution information, meeting the needs of real-time monitoring and early warning. Based on the predicted population density distribution, data-driven decision support is provided for population management and traffic optimization, improving management efficiency and the scientific nature of decision-making. By adjusting the geographical unit division and time interval, the model can adapt to different scenarios and needs, exhibiting strong flexibility and adaptability.

[0065] In particular, by evaluating the dwell time and behavioral patterns of trajectory points within geofences in real time, alarm levels can be dynamically adjusted, improving the accuracy and timeliness of alarms. Integrating alarm data from multiple geofences and different time periods generates comprehensive alarm assessment results, providing more comprehensive information for decision-making. Based on alarm levels and comprehensive assessment results, corresponding response strategies are executed, improving system flexibility and adaptability. Performance analysis of alarm data optimizes the alarm system, reducing false alarms and missed alarms, and improving system reliability and efficiency. Timely response to high-risk alarms and effective measures enhance security management of specific areas, preventing and reducing the occurrence of security incidents. Attached Figure Description

[0066] Figure 1 A flowchart illustrating the deep learning-based intelligent video GIS analysis method provided by this invention;

[0067] Figure 2 This is a flowchart illustrating step S2 in the deep learning-based intelligent video GIS analysis method provided by the present invention.

[0068] Figure 3 This is a flowchart illustrating step S3 in the deep learning-based intelligent video GIS analysis method provided by the present invention.

[0069] Figure 4 This is a flowchart illustrating step S4 in the deep learning-based intelligent video GIS analysis method provided by the present invention. Detailed Implementation

[0070] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0071] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0072] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0073] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0074] Please see Figure 1 As shown, this invention provides a video GIS intelligent analysis method based on deep learning, comprising:

[0075] Step S1: Real-time detection of targets in the video using a predefined target detection model to generate spatiotemporal labels containing bounding boxes and category information;

[0076] Step S2: Construct a dynamic graph model based on the spatiotemporal markers and use a dynamic graph neural network to associate the target trajectories of different cameras to generate spatiotemporal IDs and spatiotemporally continuous trajectory chains;

[0077] Step S3: Based on the visual inertial odometry and the geographic information system calibration homography matrix, the trajectory chain is mapped to the geographic coordinate system in real time to obtain spatiotemporal trajectory data;

[0078] Step S4: Using the number of spatiotemporal trajectories as input parameters, construct a neural network model and a trajectory generation model respectively to predict the crowd density distribution and the movement trajectory of pedestrians to obtain the target density distribution and the target trajectory;

[0079] Step S5: Issue real-time alerts based on the target trajectory and preset multi-level geofencing rules.

[0080] Specifically, a deep learning object detection model (YOLOv5) is used. Based on the requirements of the object detection model, the resolution of the input video is adjusted to the size required by the model's input layer. Commonly used resolutions include, but are not limited to, 320×240, 640×480, and 1280×720. Image scaling algorithms such as bilinear interpolation can be used to complete the resolution conversion while minimizing image distortion. The pixel values ​​of the video frames are normalized from the original range (e.g., [0,255]) to the numerical range required by the object detection model, commonly [0,1] or [-1,1]. If the original frame rate of the video does not match the processing frame rate of the object detection model, frame rate conversion is required. When the original frame rate is higher than the model's processing frame rate, a uniform frame extraction method can be used to select keyframes for input into the model; conversely, interpolation or other methods can be used to appropriately increase the number of frames, but excessive redundancy should be avoided. The preprocessed video frames are organized and packaged according to the format required by the model (e.g., tensor format). Multiple frames of video data are usually grouped into a batch. Video frames are batched into a predefined target detection model. The model uses its internal deep learning algorithms (such as YOLO, Faster R-CNN, etc.) to identify targets in the image. Taking the YOLO series models as an example, it divides the input image into multiple grid cells. Each cell is responsible for predicting a certain number of bounding boxes and corresponding class probabilities. Image features are extracted through the model's convolutional layers, pooling layers, and other structures, and finally, the bounding box coordinates and class information of the target are output. Based on the detection results output by the model, target detection results with higher confidence are selected according to a set confidence threshold. For example, if the confidence threshold is set to 0.5, only the bounding boxes and class information of targets predicted by the model with a confidence level higher than 0.5 are retained, and other unreliable detection results are discarded to reduce false positives and false negatives. For each selected target detection result, the coordinate information of its bounding box is extracted. This generally includes the coordinates of the top left and bottom right corners of the bounding box (xmin, ymin, xmax, ymax), or the center coordinates (x, y), width w, and height h, etc. These coordinates define the position range of the target in the image. Based on the class probability distribution output by the model, the class information for each target is determined, such as "pedestrian," "vehicle," or "animal." Typically, the class with the highest probability is selected as the final class for the target, and this class label is associated with the corresponding bounding box. The bounding box coordinates and class information for each target are integrated into a spatiotemporal labeled data structure.

[0081] Specifically, by constructing a dynamic graph model and applying a dynamic graph neural network, the target trajectories under different cameras were effectively linked, achieving spatiotemporally continuous trajectory chain generation and improving the accuracy and reliability of target tracking. Utilizing visual inertial odometry and geographic information system calibration, real-time mapping of trajectory chains to the geographic coordinate system was achieved, providing precise geographic information support for subsequent functions such as geofencing alarms. By predicting crowd density distribution and pedestrian movement trajectories, potential risks such as congestion and collisions can be detected in advance, allowing for timely measures to prevent accidents. Pre-setting multi-level geofencing rules enables real-time monitoring and alarming of different areas and alarm levels, improving adaptability to complex scenarios and management efficiency.

[0082] Specifically, such as Figure 2 As shown, the process of step S2 includes:

[0083] Step S21: Using the targets in the spatiotemporal markers as nodes, construct the edges between nodes according to the spatiotemporal relationships of the targets to form a dynamic graph model;

[0084] Specifically, each target in the spatiotemporal marker is treated as a node, and the temporal and spatial relationships between nodes are analyzed to construct a dynamic graph model. If two targets are in adjacent time periods and are geographically close, a connection edge is established. The determination of whether to establish a connection is made by calculating the time difference and spatial distance between the targets. For example, if the time difference is less than a threshold (e.g., 1 second) and the spatial distance is less than a threshold (e.g., 5 meters), the two targets are considered to be related, and an edge connection is established.

[0085] Step S22: Input the dynamic graph model into the dynamic graph neural network for processing to obtain the updated node features and the association weight matrix between nodes;

[0086] Specifically, the process of inputting the dynamic graph model into the dynamic graph neural network for processing to obtain updated node features and the association weight matrix between nodes includes:

[0087] Each node in the dynamic graph model is used to extract features as the initial input feature vector for the node in the dynamic graph neural network.

[0088] Collect the feature information of neighboring nodes based on the connection relationship between each node and its neighboring nodes;

[0089] The neighbor feature information is combined with the initial input feature vector, and the initial input feature vector is updated using an update function to obtain the target feature vector;

[0090] The correlation weight matrix between nodes is calculated using cosine distance based on the target feature vector.

[0091] Specifically, information such as the target category, bounding box position, velocity, and acceleration of each node is extracted to form an initial feature vector. The feature vector can be represented as [category code, bounding box coordinates, velocity, acceleration]. Category coding can use one-hot coding or embedding coding. The neighboring nodes of each node are traversed, and their feature information is collected. The feature vectors of neighboring nodes are collected through the message passing mechanism in the graph neural network. The feature information of neighboring nodes is aggregated (e.g., by averaging, summing, or using an attention mechanism), combined with the initial feature vector, and the feature vector is updated through an update function (e.g., a fully connected layer). The update formula can be expressed as:

[0092]

[0093] in, Let N(i) be the initial feature vector of node i, N(i) be the set of neighboring nodes of node i, Aggregate be the aggregation function, W be the weight matrix, and σ be the activation function.

[0094] The similarity between each node and other nodes is calculated using cosine distance, forming an association weight matrix. Cosine distance measures the similarity between two vectors in a given direction, and its calculation formula is:

[0095]

[0096] The value of Similarity(i,j) ranges from [-1,1]. The closer the value is to 1, the more similar the feature vectors of the two nodes are in the direction, and the stronger the correlation. i h represents the target feature vector of node i. j This represents the target feature vector of node j. ||h i || is the vector h i The magnitude (i.e., the L of the vector) 2 Norm). ||h j || represents vector h j The length of the module.

[0097] Step S23: Based on the association weight matrix, the Hungarian algorithm is used to calculate the trajectory segments belonging to the same target under different cameras, and a unique spatiotemporal ID is generated for each target;

[0098] Specifically, the association weight matrix is ​​treated as a cost matrix, and the Hungarian algorithm is used to find the optimal match, associating trajectory segments from different cameras with the same target and generating a unique time ID. The Hungarian algorithm is used to solve the bipartite graph matching problem and can find the matching scheme with the minimum cost. The trajectory segments from different cameras are treated as two vertex sets of a bipartite graph, and the association weight matrix is ​​used as the edge weights, then the Hungarian algorithm is applied for matching.

[0099] Step S24: Connect the trajectory segments according to time sequence and spatial relationship to form a spatiotemporally continuous trajectory chain.

[0100] Specifically, based on the timestamps and spatial locations of trajectory segments, trajectory segments of the same target are connected in chronological order to form a continuous trajectory chain. Trajectory segments are sorted by timestamp, and the spatial continuity of adjacent segments is checked. If both the time difference and spatial distance are less than a threshold, the segments are connected into a trajectory chain.

[0101] Specifically, by constructing a dynamic graph model and utilizing a dynamic graph neural network, target trajectories from different cameras can be more accurately associated, reducing false and missed associations. The Hungarian algorithm is used for trajectory segment association, finding the optimal solution in polynomial time, improving computational efficiency and meeting real-time requirements. Through feature extraction and updating, the dynamic graph neural network can adapt to changes in target appearance and motion state, improving the robustness of trajectory association.

[0102] Specifically, such as Figure 3 As shown, the process of step S3 includes:

[0103] Step S31: Extract feature points from the continuous image sequence acquired by the camera, combine the angular velocity and acceleration data of the inertial measurement unit, and construct a visual odometry model through a point-line feature fusion algorithm.

[0104] Specifically, feature points are extracted from a continuous sequence of images captured by the camera and combined with angular velocity and acceleration data provided by the inertial measurement unit to construct a visual odometry model. This model can estimate camera pose changes in real time. Feature points are extracted from the images using corner detection algorithms (such as Harris corner detection) or feature point detection algorithms (such as ORB, SIFT, SURF, etc.). Feature matching algorithms (such as FLANN or brute-force matching) are used to match feature points across consecutive frames, obtaining matched feature point pairs. Combining feature points with line segment features in the image improves the estimation accuracy of the odometry. Point-line feature fusion algorithms can utilize the geometric relationships between feature points and line segments to enhance the estimation of camera motion. Angular velocity and acceleration data are fused with visual feature point data, and a more accurate visual odometry model is constructed using sensor fusion algorithms (such as tightly coupled or loosely coupled methods).

[0105] Step S32: A robust optimization method is used to dynamically solve the camera motion parameters to generate a real-time updated homography matrix. The homography matrix maps pixel coordinates to a local world coordinate system to obtain local coordinates.

[0106] Specifically, the process of dynamically solving for camera motion parameters using a robust optimization method to generate a real-time updated homography matrix includes:

[0107] Feature points are extracted from consecutive frames of a video, and feature descriptors are calculated to obtain feature point pairs.

[0108] The initial homography matrix is ​​calculated using the direct linear transformation algorithm based on the matched feature point pairs;

[0109] Construct an optimization objective function that minimizes the reprojection error;

[0110] The homography matrix is ​​obtained by dynamically adjusting the homography matrix using an optimization algorithm based on the objective function and iteratively calculating the updated matrix value.

[0111] Specifically, feature points are extracted from consecutive frames of the video, and feature descriptors are calculated. Matched feature point pairs are obtained using a feature matching algorithm. An initial homography matrix is ​​calculated based on the matched feature point pairs using the Direct Linear Transform (DLT) algorithm. The DLT algorithm estimates the elements of the homography matrix by solving a system of linear equations, requiring at least four pairs of matched feature points to compute the homography matrix. An optimization objective function is constructed to minimize the reprojection error of feature points in the pixel coordinate system compared to the local world coordinate system after mapping through the homography matrix. The optimization objective function can be expressed as:

[0112]

[0113] Where proj represents the projection function, pixel i These are pixel coordinates, world i It is a world coordinate system.

[0114] The homography matrix is ​​dynamically adjusted using optimization algorithms (such as the Levenberg-Marquardt algorithm or gradient descent-based optimization methods) to minimize the objective function. The Levenberg-Marquardt algorithm is a commonly used nonlinear least squares optimization algorithm that can quickly converge to the optimal solution. During the optimization process, the value of the homography matrix is ​​iteratively updated until the convergence condition is met (such as the change in the objective function value being less than a set threshold or reaching the maximum number of iterations).

[0115] Step S33: Register the local coordinates with the predefined ground control points in the geographic information system to convert the local coordinates into geographic coordinates;

[0116] Specifically, the local coordinates obtained through the homography matrix are registered with predefined ground control points in a Geographic Information System (GIS), converting the local coordinates into geographic coordinates. Ground control points are points with known geographic coordinates. By matching and transforming the local coordinates with these control points, the conversion from the local coordinate system to the geographic coordinate system is achieved. Coordinate registration can be performed using methods such as similarity transformation, affine transformation, or polynomial transformation.

[0117] Step S34: Perform spatiotemporal smoothing on the geographic coordinates and fuse multi-source data using a Kalman filter to generate spatiotemporal trajectory data.

[0118] Specifically, the transformed geographic coordinates undergo spatiotemporal smoothing, and multi-source data (such as visual odometry and IMU data) is fused using a Kalman filter to generate spatiotemporal trajectory data. Moving average filtering and Gaussian filtering methods can be used to smooth the geographic coordinates and reduce the impact of noise. The Kalman filter is a recursive filter capable of fusing multi-source data in real time to estimate the optimal state of the target. By designing state equations and observation equations, geographic coordinates, velocity, acceleration, and other information are fused together to generate smooth, continuous spatiotemporal trajectory data.

[0119] Specifically, by constructing a visual odometry model and employing a robust optimization method to dynamically solve the homography matrix, pixel coordinates can be mapped to the local world coordinate system more accurately, improving the accuracy of coordinate transformation. Registering local coordinates with ground control points in a geographic information system (GIS) allows for accurate conversion of local coordinates to geographic coordinates, enhancing the reliability and usability of trajectory data. Spatiotemporal smoothing of geographic coordinates combined with Kalman filtering to fuse multi-source data generates smooth, continuous spatiotemporal trajectory data, reducing the impact of noise and errors and improving the quality of trajectory data.

[0120] Specifically, such as Figure 4 As shown, the process of step S4 includes:

[0121] Step S41: Obtain spatiotemporal trajectory data within past time periods to obtain historical spatiotemporal trajectory data;

[0122] Step S42: Calculate the population density distribution information within the divided geographical units and predefined time intervals based on the historical spatiotemporal trajectory data, and use it as the population density distribution label.

[0123] Step S43: Aggregate and normalize the spatiotemporal trajectory data to generate an input tensor containing spatiotemporal information and population distribution characteristics;

[0124] Step S44: Construct a training dataset using the input tensor and the crowd density distribution labels;

[0125] Step S45: Use a long short-term memory network to extract features from the input tensor to learn the spatiotemporal variation pattern of population density distribution;

[0126] Step S46: Use the spatiotemporal change pattern and the training dataset to train the model to obtain a trained model;

[0127] Step S47: Input the spatiotemporal trajectory data into the training model to obtain the target density distribution.

[0128] Specifically, spatiotemporal trajectory data within a specific time period is collected from a video GIS system. This data can come from historical records of similar or identical scenes. Historical spatiotemporal trajectory data can be obtained through database queries or file reading. The data typically includes trajectory point information of the target, such as timestamps, geographic coordinates, and target IDs. The geographic area is divided into multiple geographic units (e.g., grids), and time intervals (e.g., per minute, per hour) are defined. Within each geographic unit and time interval, the number of people is counted, generating crowd density distribution labels. Grid partitioning methods can be used to divide the geographic area into several equal-sized grid units. Appropriate time intervals, such as per minute or per hour, are selected based on the specific application scenario. Within each geographic unit and time interval, the number of targets is counted, and crowd density is calculated. Methods such as simple counting or Gaussian kernel density estimation can be used. The spatiotemporal trajectory data is aggregated and normalized to generate an input tensor containing spatiotemporal information and crowd distribution characteristics. The spatiotemporal trajectory data is then aggregated along the time and spatial dimensions to form a multidimensional data array. Normalize the data to a specific range (e.g., [0,1]), commonly using methods such as min-max normalization and Z-score normalization. Pair the input tensors with crowd density distribution labels to form a training dataset. Ensure that each input tensor corresponds to a correct crowd density distribution label, forming training sample pairs. Use a Long Short-Term Memory (LSTM) network to extract time-series features from the input tensors to learn the spatiotemporal variation patterns of the crowd density distribution. Construct a multi-layer LSTM network, with each layer containing an appropriate number of hidden units. The LSTM network extracts time-series features from the input tensors through cyclic processing at time steps. Use the training dataset to train the LSTM model, optimize the model parameters, and learn the spatiotemporal variation patterns.

[0129] Choose an appropriate loss function, such as mean squared error (MSE) or mean absolute error (MAE). Train the model using optimization algorithms (such as Adam or RMSprop), adjusting the model parameters to minimize the loss function. Perform multiple training epochs, iteratively training the model on the training data in each epoch and updating the model parameters. Input real-time spatiotemporal trajectory data into the trained model to obtain the predicted target density distribution. Perform the same preprocessing (such as aggregation and normalization) on the real-time data as on the training data, and then input it into the model for prediction to obtain the predicted population density distribution.

[0130] Specifically, by learning spatiotemporal variation patterns, the model can more accurately predict population density distribution, providing more reliable data support for resource allocation and risk warning. Trained using historical spatiotemporal trajectory data and corresponding labels, the model can learn the patterns of population density changes in different scenarios, improving its generalization ability. Processing and predicting real-time spatiotemporal trajectory data can provide timely population density distribution information, meeting the needs of real-time monitoring and early warning. Based on the predicted population density distribution, data-driven decision support is provided for population management and traffic optimization, improving management efficiency and the scientific nature of decision-making. By adjusting the geographical unit division and time interval, the model can adapt to different scenarios and needs, exhibiting strong flexibility and adaptability.

[0131] Specifically, step S4 further includes:

[0132] The historical spatiotemporal trajectory data is preprocessed to extract the historical motion trajectory features of individuals;

[0133] The spatiotemporal trajectory data and the historical trajectory features are fused to obtain a fusion result;

[0134] The fusion result is used as input to construct the trajectory generation model;

[0135] The trajectory generation model is trained using the spatiotemporal trajectory data and the historical spatiotemporal trajectory data to obtain the target trajectory.

[0136] Specifically, historical spatiotemporal trajectory data is preprocessed to extract historical motion trajectory features for each individual, including position, velocity, and acceleration. Noise and outliers are removed to ensure data accuracy and completeness. The velocity and acceleration of each trajectory point are calculated; velocity can be obtained by removing the position difference between adjacent points by time intervals, and acceleration can be calculated by the rate of change of velocity. Long trajectories are segmented into multiple short segments to better capture local motion patterns. Real-time spatiotemporal trajectory data is fused with the extracted historical motion trajectory features to form a fusion result. Real-time and historical data are aligned in the time dimension, which can be achieved through interpolation or time window matching. Real-time data and historical features are concatenated into a comprehensive feature vector in chronological order. Weighted fusion or nonlinear fusion methods are used to combine real-time data and historical features to enhance the model's perception of the current motion state. Sequence prediction models, such as Long Short-Term Memory (LSTM) networks or Social-GAN, are employed. The fusion result is organized into the input format required by the model, such as time series data. Model parameters are initialized to prepare for training. The trajectory generation model is trained using spatiotemporal trajectory data and historical trajectory features to obtain the predicted target trajectory. Historical spatiotemporal trajectory data and extracted features are used as the training set, including the input sequence and the corresponding real trajectory sequence. A loss function, such as mean squared error (MSE) or a trajectory similarity metric, is selected. An optimization algorithm (such as Adam) is used to adjust the model parameters to minimize the error between the predicted trajectory and the real trajectory.

[0137] Specifically, by integrating historical motion trajectory features with real-time data, the model can more comprehensively understand pedestrian movement patterns and improve the accuracy of trajectory prediction. Utilizing historical trajectory features, the model can capture complex motion patterns such as pedestrian acceleration and velocity changes, enhancing the precision of predictions. Combining historical and real-time data allows the model to better adapt to different scenarios and dynamic changes, enhancing robustness. Real-time trajectory prediction: Processing and predicting real-time data enables the system to promptly output the future trajectory of pedestrians, meeting the needs of real-time monitoring and early warning.

[0138] Specifically, step S5 includes the following process:

[0139] Define multi-level geofences, each geofence including a unique identifier, geographic shape, spatial location coordinates and corresponding alarm level;

[0140] Specifically, multiple geofences are predefined in the geographic information system. Each geofence has a unique identifier, geographic shape (such as a polygon or circle), spatial coordinates, and a corresponding alarm level. Geofences are defined using GIS tools or programming interfaces, setting their shape, location, and alarm level. Different alarm levels are set according to the importance and sensitivity of the geofence.

[0141] Extract trajectory points from the target trajectory, each trajectory point including a timestamp and geographic coordinates;

[0142] Specifically, trajectory points are extracted from the generated target trajectory, each containing a timestamp and geographic coordinates. Trajectory point extraction can be performed at fixed time intervals to ensure the temporal continuity of the trajectory points.

[0143] Determine whether the trajectory point enters or leaves the geofence;

[0144] Specifically, the process of determining whether the trajectory point enters or leaves the geofence includes:

[0145] Convert the multi-level fence into a list of vertex coordinates;

[0146] Transform the geographic coordinates of the trajectory points to the same coordinate system as the geographic fence;

[0147] An algorithm for determining the positional relationship between points and polygons is used to determine whether the trajectory points are located within the geofence.

[0148] Determine whether the timestamps of the trajectory points conform to the time rules of the geofence;

[0149] Record the matching results between the trajectory points and the geofence, update the matching status in real time, and record the corresponding alarm level.

[0150] Specifically, for polygonal fences, the vertex coordinates are extracted; for circular fences, the center coordinates and radius are extracted. Coordinate transformation algorithms, such as projection transformation, are used to ensure coordinate system consistency. A point-to-polygon positional relationship judgment algorithm is used to determine whether a trajectory point is within the geofence. Common algorithms include ray casting and area casting. Ray casting determines the positional relationship between a point and a polygon by emitting a ray from the trajectory point in one direction and calculating the number of intersections with the polygon's edges. The timestamps of the trajectory points are checked to ensure they conform to the geofence's time rules. Time rules can be specific time periods (such as working hours, nighttime, etc.), and judgment is made by comparing timestamps. Data structures (such as dictionaries or database tables) are used to record matching results and alarm levels, updating status information in real time. An event-driven mechanism triggers an alarm event when a trajectory point is detected entering or leaving the fence. Alarm information includes alarm time, location, alarm level, and target ID. Alarm information, including alarm status, duration, and frequency, is updated in real time. A time window sliding mechanism and spatial analysis methods are used to fuse multi-source alarm data. By analyzing the fused data, the overall risk level of the alarms is assessed to support decision-making.

[0151] When the trajectory point enters or leaves the geofence, an alarm of the corresponding level is triggered according to the preset multi-level geofence rules.

[0152] Specifically, the alarm rules are as follows: Level 1 Alarm (Alert): The target enters the outer buffer zone (within 50 meters). Level 2 Alarm (Warning): The target approaches the fence boundary (10 meters) and its speed is >5 m / s. Level 3 Alarm (Emergency): The target enters the core area, triggering audible and visual alarms and police dispatch, with a response time of <2 seconds.

[0153] Specifically, it can monitor the relationship between target trajectories and geofences in real time, triggering alarms promptly and improving monitoring efficiency and response speed. It supports multi-level geofences and different alarm levels, adapting to various complex scenarios and needs. Through coordinate transformation and geometric algorithms, it accurately determines whether trajectory points are within the geofence, improving alarm accuracy. Real-time monitoring and alarms for specific areas enable timely detection of abnormal behavior, enhancing security.

[0154] Specifically, step S5 further includes:

[0155] The alarm level is evaluated and adjusted in real time based on the dwell time and behavior patterns of the trajectory points within the geofence.

[0156] Alarm information is updated in real time to integrate alarm data from multiple geofences and different time periods. The alarm data and performance are analyzed to obtain a comprehensive alarm evaluation result.

[0157] Execute the corresponding alarm response strategy based on the alarm level and the comprehensive evaluation result.

[0158] Specifically, the system records the timestamps of trajectory points entering the geofence and calculates dwell time in real time. It calculates the velocity and direction change rate of trajectory points to identify abnormal behavior (such as loitering or acceleration). Rules are set based on dwell time and behavior patterns; for example, the alarm level is increased when the dwell time exceeds a threshold (e.g., 1 minute) or abnormal behavior occurs. Alarm data from multiple geofences are aligned temporally and spatially and integrated into a unified data structure. Alarm frequency, duration, and other metrics are statistically analyzed to evaluate the alarm system's performance. A comprehensive alarm evaluation result is generated by combining alarm level and performance metrics using weighted averaging or other fusion algorithms. Predefined response measures are defined for different alarm levels and comprehensive evaluation results, such as notifying relevant personnel and activating emergency plans. Corresponding response actions are executed through automated systems or manual intervention, such as sending alarm notifications and adjusting monitoring resource allocation.

[0159] Specifically, by evaluating the dwell time and behavioral patterns of trajectory points within geofences in real time, alarm levels can be dynamically adjusted, improving the accuracy and timeliness of alarms. Integrating alarm data from multiple geofences and different time periods generates a comprehensive alarm assessment result, providing more comprehensive information for decision-making. Based on the alarm level and the comprehensive assessment result, corresponding response strategies are executed, improving the system's flexibility and adaptability. Performance analysis of alarm data optimizes the alarm system, reducing false alarms and missed alarms, and improving system reliability and efficiency. Timely response to high-risk alarms and the implementation of effective measures enhance security management in specific areas, preventing and reducing the occurrence of security incidents.

[0160] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0161] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A video GIS intelligent analysis method based on deep learning, characterized in that, include: Step S1: Real-time detection of targets in the video using a predefined target detection model to generate spatiotemporal labels containing bounding boxes and category information; Step S2: Construct a dynamic graph model based on the spatiotemporal markers and use a dynamic graph neural network to associate the target trajectories of different cameras to generate spatiotemporal IDs and spatiotemporally continuous trajectory chains; Step S3: Based on the visual inertial odometry and the geographic information system calibration homography matrix, the trajectory chain is mapped to the geographic coordinate system in real time to obtain spatiotemporal trajectory data; Step S4: Using the number of spatiotemporal trajectories as input parameters, construct a neural network model and a trajectory generation model respectively to predict the crowd density distribution and the movement trajectory of pedestrians to obtain the target density distribution and the target trajectory; Step S5: Issue real-time alerts based on the target trajectory and preset multi-level geofencing rules.

2. The video GIS intelligent analysis method based on deep learning according to claim 1, characterized in that, The process of step S2 includes: Using targets in the spatiotemporal markers as nodes, edges between nodes are constructed based on the spatiotemporal relationships of the targets to form a dynamic graph model; The dynamic graph model is input into the dynamic graph neural network for processing to obtain updated node features and the association weight matrix between nodes; Based on the association weight matrix, the Hungarian algorithm is used to calculate the trajectory segments belonging to the same target under different cameras, and a unique spatiotemporal ID is generated for each target; The trajectory segments are connected according to time sequence and spatial relationship to form a spatiotemporally continuous trajectory chain.

3. The video GIS intelligent analysis method based on deep learning according to claim 2, characterized in that, The process of inputting the dynamic graph model into the dynamic graph neural network for processing to obtain updated node features and the association weight matrix between nodes includes: Each node in the dynamic graph model is used to extract features as the initial input feature vector for the node in the dynamic graph neural network. Collect the feature information of neighboring nodes based on the connection relationship between each node and its neighboring nodes; The neighbor feature information is combined with the initial input feature vector, and the initial input feature vector is updated using an update function to obtain the target feature vector; The correlation weight matrix between nodes is calculated using cosine distance based on the target feature vector.

4. The video GIS intelligent analysis method based on deep learning according to claim 3, characterized in that, The process of step S3 includes: Feature points are extracted from continuous image sequences captured by a camera, and combined with angular velocity and acceleration data from an inertial measurement unit, a visual odometry model is constructed using a point-line feature fusion algorithm. A robust optimization method is used to dynamically solve for camera motion parameters to generate a real-time updated homography matrix, which maps pixel coordinates to a local world coordinate system to obtain local coordinates. The local coordinates are registered with the predefined ground control points in the geographic information system to convert the local coordinates into geographic coordinates. The geographic coordinates are subjected to spatiotemporal smoothing and multi-source data are fused using a Kalman filter to generate spatiotemporal trajectory data.

5. The video GIS intelligent analysis method based on deep learning according to claim 4, characterized in that, The process of dynamically solving for camera motion parameters using a robust optimization method to generate a real-time updated homography matrix includes: Feature points are extracted from consecutive frames of a video, and feature descriptors are calculated to obtain feature point pairs. The initial homography matrix is ​​calculated using the direct linear transformation algorithm based on the matched feature point pairs; Construct an optimization objective function that minimizes the reprojection error; The homography matrix is ​​obtained by dynamically adjusting the homography matrix using an optimization algorithm based on the objective function and iteratively calculating the updated matrix value.

6. The video GIS intelligent analysis method based on deep learning according to claim 5, characterized in that, The process of step S4 includes: To obtain historical spatiotemporal trajectory data, spatiotemporal trajectory data is obtained by acquiring spatiotemporal trajectory data within a past time period. The population density distribution information within the divided geographical units and predefined time intervals is calculated based on the historical spatiotemporal trajectory data and used as the population density distribution label. The spatiotemporal trajectory data is aggregated and normalized to generate an input tensor containing spatiotemporal information and population distribution characteristics; Construct a training dataset using the input tensor and the crowd density distribution labels; Long Short-Term Memory (LSTM) networks are used to extract features from the input tensor to learn the spatiotemporal variation patterns of population density distribution; The model is trained using the spatiotemporal variation pattern and the training dataset to obtain a trained model; The spatiotemporal trajectory data is input into the training model to obtain the target density distribution.

7. The video GIS intelligent analysis method based on deep learning according to claim 6, characterized in that, The process of step S4 also includes: The historical spatiotemporal trajectory data is preprocessed to extract the historical motion trajectory features of individuals; The spatiotemporal trajectory data and the historical trajectory features are fused to obtain a fusion result; The fusion result is used as input to construct the trajectory generation model; The trajectory generation model is trained using the spatiotemporal trajectory data and the historical spatiotemporal trajectory data to obtain the target trajectory.

8. The video GIS intelligent analysis method based on deep learning according to claim 7, characterized in that, The process of step S5 includes: Define multi-level geofences, each geofence including a unique identifier, geographic shape, spatial location coordinates and corresponding alarm level; Extract trajectory points from the target trajectory, each trajectory point including a timestamp and geographic coordinates; Determine whether the trajectory point enters or leaves the geofence; When the trajectory point enters or leaves the geofence, an alarm of the corresponding level is triggered according to the preset multi-level geofence rules.

9. The video GIS intelligent analysis method based on deep learning according to claim 8, characterized in that, The process of determining whether the trajectory point enters or leaves the geofence includes: Convert the multi-level fence into a list of vertex coordinates; Transform the geographic coordinates of the trajectory points to the same coordinate system as the geographic fence; An algorithm for determining the positional relationship between points and polygons is used to determine whether the trajectory points are located within the geofence. Determine whether the timestamps of the trajectory points conform to the time rules of the geofence; Record the matching results between the trajectory points and the geofence, update the matching status in real time, and record the corresponding alarm level.

10. The video GIS intelligent analysis method based on deep learning according to claim 9, characterized in that, The process of step S5 also includes: The alarm level is evaluated and adjusted in real time based on the dwell time and behavior patterns of the trajectory points within the geofence. Alarm information is updated in real time to integrate alarm data from multiple geofences and different time periods. The alarm data and performance are analyzed to obtain a comprehensive alarm evaluation result. Execute the corresponding alarm response strategy based on the alarm level and the comprehensive evaluation result.

Images