An edge-computing-based video monitoring intelligent analysis method and system

Abstract

The application relates to the technical field of data processing, and discloses a video monitoring intelligent analysis method and system based on edge computing, which comprises the following steps: generating a sub-region set for each camera in a monitoring area and a time sequence people flow feature vector of each camera and a people flow anomaly mark of current real-time people flow; constructing a dynamic weighted undirected graph representing the topological relationship of a plurality of sub-regions in the monitoring area; inputting a feature matrix fusing the time sequence people flow feature vector and the people flow anomaly mark into a sub-region key level probability prediction model based on a space-time graph convolution network and the dynamic weighted undirected graph; and obtaining the final key level of each sub-region after prediction and correction, so as to control the edge device to perform frame extraction and video picture analysis at a corresponding frame rate. Therefore, the application can balance the video analysis accuracy and the computing power load of the edge device by acquiring real-time and historical people flow features and anomaly marks and fusing the physical distance and topological connectivity among the sub-regions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a video monitoring intelligent analysis method and system based on edge computing. BACKGROUND

[0002] With the wide application of video monitoring technology in the fields of security and protection, traffic, smart city and the like, the scale of the monitoring area is continuously expanded, the number of deployed cameras is continuously increased, and edge computing becomes the core carrier of video analysis due to the advantages of low delay and high real-time performance.

[0003] However, the current edge box generally adopts fixed frequency for video decoding and inference analysis, without considering the actual differences of different sub-areas in the monitoring area (for example, the monitoring and personnel identification analysis of people flow in each position by deploying cameras in large shopping malls): firstly, the high-frequency analysis is required in the dense people flow area (such as intersections and entrances) to ensure safety, but the high-frequency analysis in the sparse people flow area (such as remote passages) will cause waste of computing power; secondly, there is a spatial topological correlation between sub-areas (such as the linkage of people flow in adjacent passages), and isolated judgment of the importance of a single area is easy to produce misjudgment; thirdly, people flow has obvious time sequence law (such as the difference between morning and evening peak and weekdays and weekends), and the static division strategy cannot adapt to dynamic changes.

[0004] Therefore, how to consider the actual differences of different sub-areas in the monitoring area, and avoid the problems of excessive overall computing power load or insufficient video analysis accuracy caused by the edge box performing video picture analysis on each camera video stream with the same frequency, is a technical problem to be solved. SUMMARY

[0005] The present application provides a video monitoring intelligent analysis method and system based on edge computing, which aims to solve at least one of the above technical problems.

[0006] To achieve the above purpose, the present application provides a video monitoring intelligent analysis method based on edge computing, comprising:

[0007] Based on the geographic information of the monitoring area and the camera frustum parameters, the region is discretized by gridding, and a sub-area set is generated according to the camera coverage range;

[0008] Using the real-time people flow and historical people flow features collected by each camera, a time sequence people flow feature vector of each camera and a people flow anomaly flag of the current real-time people flow are generated;

[0009] According to the monitoring node set of each camera, the edge set determined by the spatial position of each camera in the monitoring area, and the edge weight fused by the spatial distance and the people flow similarity, a dynamic weighted undirected graph representing the topological relationship of a plurality of sub-areas in the monitoring area is constructed;

[0010] inputting the fusion time-series passenger flow feature vector and the feature matrix of the passenger flow anomaly flag into a sub-region key level probability prediction model based on a spatio-temporal graph convolution network and a dynamic weighted undirected graph, to predict a key level probability of each sub-region, and determining an original key level of each sub-region based on the key level probability;

[0011] performing key level correction of each sub-region according to a key level correction rule, and obtaining a final key level for each sub-region;

[0012] generating an analysis strategy of the camera video stream of each sub-region in the next time period based on the final key level of each sub-region in the current time period, and controlling the edge device to perform video monitoring analysis in the next time period according to the analysis strategy.

[0013] Optionally, based on the monitoring area geographic information and the camera view cone parameters, the sub-region set is generated by discretizing the area through gridding, and the specific steps include:

[0014] obtaining the planar geographic information of the monitoring area, uniformly discretizing the monitoring area into a cell grid of a preset size, and forming a grid set , wherein m represents the number of grids;

[0015] According to the projection range of the view cone of each camera on the ground, all cell grids covered by the camera are marked to form a sub-region , wherein i represents the i-th camera, represents the sub-region corresponding to the i-th camera, and k represents the k-th grid in the grid set;

[0016] all sub-regions corresponding to the cameras are summarized to form a sub-region set ; wherein any two and in the sub-region set R allow spatial overlap, n represents the number of sub-regions, and respectively represent the sub-region corresponding to the i-th camera and the sub-region corresponding to the j-th camera.

[0017] Optionally, the time-series passenger flow feature vector of each camera and the passenger flow anomaly flag of the current real-time passenger flow are generated by using real-time passenger flow and historical passenger flow features collected by each camera, and the specific steps include:

[0018] For the video stream of the current time period t, the YOLOv5 target detection algorithm is used to identify pedestrian targets, and then the DeepSORT multi-target tracking algorithm is used to remove repeated counting, to obtain the current passenger flow , represents the number of people flow of the i-th camera at the current time period t;

[0019] extracting the historical people flow numbers of the previous two time periods from the storage unit and , querying the people flow data of the historical time periods, calculating the mean value and the standard deviation , and respectively represent the mean value and the standard deviation of the number of people flow of the i-th camera at the current time period t and the historical time periods t-1 and t-2;

[0020] constructing a time series people flow feature vector containing the current people flow number , the historical people flow number , the mean value and the standard deviation , , represents the time series people flow feature vector of the i-th camera at the current time period t;

[0021] determining whether the current people flow number exceeds the historical mean value by 2 standard deviations, if yes, assigning a value of 1 to the people flow anomaly flag , if not, assigning a value of 0 to the people flow anomaly flag , represents the people flow anomaly flag of the i-th camera at the current time period t.

[0022] Optionally, a step of constructing a dynamic weighted undirected graph representing the topological relationship of a plurality of sub-regions in the monitoring area according to the monitoring node set of each camera, the edge set determined according to the spatial position of each camera in the monitoring area, and the edge weight of the fusion spatial distance and people flow similarity, specifically comprising:

[0023] constructing a monitoring node set , using the Delaunay triangulation algorithm to spatially divide the monitoring nodes, if the Euclidean distance between any two nodes and does not exceed a predetermined distance threshold, then an edge is established between the two, forming an edge set , n represents the number of monitoring nodes corresponding to the cameras, i and j respectively represent the i-th and j-th monitoring nodes, represents the edge between the i-th and j-th monitoring nodes;

[0024] based on the spatial distance and the people flow similarity, calculating the weight of each edge , combining the monitoring node set , the edge set​ with edge weights , forming a dynamic weighted undirected graph , denotes the weight of the edge between the i-th and j-th monitoring nodes.

[0025] Optionally, the weight of each edge is expressed by an expression, specifically:

[0026] ;

[0027] wherein, denotes the planar Euclidean distance between the monitoring nodes and , denotes the distance reference value, denotes the spatial distance decay function between the monitoring nodes and , associated with the spatial distance, denotes the minimum maximum ratio between the monitoring nodes and , associated with the crowd flow similarity, and are weight coefficients. .

[0028] Optionally, the feature matrix of the fused time-series crowd flow feature vector and crowd flow anomaly flag is input into a sub-region key level probability prediction model based on a spatio-temporal graph convolution network and a dynamic weighted undirected graph, to predict the key level probability of each sub-region, and based on the key level probability, determine the original key level of each sub-region, specifically including:

[0029] The time-series crowd flow feature vector and the crowd flow anomaly flag are spliced as the feature matrix of the monitoring nodes ;

[0030] The feature matrix is aggregated by a two-layer spatio-temporal graph convolution network, the feature matrix is taken as the first layer of the spatio-temporal graph convolution network to obtain the feature matrix , and the feature matrix is taken as the second layer of the spatio-temporal graph convolution network to obtain a 32-dimensional feature matrix ;

[0031] wherein, the aggregation expression of each layer in the two-layer spatio-temporal graph convolution network is specifically:

[0032] ;​​

[0033] wherein, represents the feature matrix of the i-th layer, represents the feature matrix of the i-th layer, represents the adjacency matrix of the dynamic weighted undirected graph, represents the identity matrix, represents the degree matrix, represents the symmetric normalized adjacency matrix, represents the trainable weight matrix of the i-th layer, represents the ReLU activation function; The feature matrix is input into a fully connected layer for dimension conversion, and a probability vector of each region is calculated using a Sotfmax function, and the expression is specifically as follows:

[0034] The feature matrix is input into a fully connected layer for dimension conversion, and a probability vector of each region is calculated using a Sotfmax function, and the expression is specifically as follows: , respectively, represent the probabilities corresponding to the L, M, and H levels, wherein L represents a low-frequency level, M represents a medium-frequency level, and H represents a high-frequency level.

[0035] ;

[0036] wherein, , respectively, represent the probabilities corresponding to the L, M, and H levels, wherein L represents a low-frequency level, M represents a medium-frequency level, and H represents a high-frequency level.

[0037] Optionally, the method further comprises:

[0038] In each incremental training period, the time sequence people flow feature vectors generated in the past two incremental training periods, , the dynamic weighted undirected graph of the monitoring area, and the manually annotated key event samples are constructed as training samples.

[0039] The training samples are input into a two-layer spatio-temporal graph convolutional network, an incremental training is performed using a cross-entropy loss function, trainable parameters of the spatio-temporal graph convolutional network are updated, and a time-updated sub-region key level probability prediction model is obtained.

[0040] Optionally, for the original key level of each sub-region and the dynamic weighted undirected graph, key level correction of each sub-region is performed according to a key level correction rule to obtain a final key level step, which specifically comprises:

[0041] The original key level of each sub-region in the current period is smoothed based on historical periods, and the smoothed original key level and the dynamic weighted undirected graph are used to perform key level correction according to a first correction rule and a second correction rule to obtain a final key level.​​​​​​

[0042] The first correction rule is configured to: for a sub-region with an original key level of H, calculate a shortest path hop count of the sub-region in a dynamic weighted undirected graph with other H-level sub-regions, and if the shortest path hop count is greater than a target hop count, downgrade the sub-region to an M key level.

[0043] The second correction rule is configured to: for a sub-region with an original key level of L for a continuous preset number of periods, judge whether a historical average of the sub-region is less than a target low-traffic judgment value, and if so, upgrade the sub-region to an M key level.

[0044] Optionally, based on the final key level of each sub-region in the current period, an analysis strategy of the camera video stream of the edge device for each sub-region in the next period is generated, and according to the analysis strategy, the edge device is controlled to perform the video monitoring analysis step in the next period, specifically including:

[0045] Based on the final key level of each sub-region in the current period, according to a preset mapping relationship between the final key level and the video analysis frequency, a video analysis frequency of the camera video stream of the edge device for each sub-region in the next period is determined.

[0046] According to the video analysis frequency and the video frame rate of each camera, a frame extraction frequency for the camera video stream is generated, and the edge device is controlled to perform frame extraction and video picture analysis in the next period.

[0047] In addition, in order to achieve the above purpose, the application also provides a video monitoring intelligent analysis system based on edge computing, comprising:

[0048] The grid module is configured to generate a sub-region set according to the camera coverage range by discretizing the region through gridding based on the monitoring region geographic information and the camera view cone parameters.

[0049] The generation module is configured to generate a time series traffic feature vector of each camera and a traffic anomaly flag of the current real-time traffic flow by using the real-time traffic flow and the historical traffic flow features collected by each camera.

[0050] The construction module is configured to construct a dynamic weighted undirected graph representing the topological relationship of a plurality of sub-regions in the monitoring region according to the monitoring node set of each camera, the edge set determined by the spatial position of each camera in the monitoring region, and the edge weight fused with the spatial distance and the traffic similarity.

[0051] ​​a prediction module configured to input the fusion time-series passenger flow feature vector and the feature matrix of the passenger flow anomaly flag into a sub-region key level probability prediction model based on a spatio-temporal graph convolution network and a dynamic weighted undirected graph, to predict a key level probability of each sub-region, and to determine an original key level of each sub-region based on the key level probability;

[0052] a correction module configured to perform key level correction of each sub-region according to a key level correction rule based on the original key level of each sub-region and the dynamic weighted undirected graph, and to obtain a final key level;

[0053] an analysis module configured to generate an analysis strategy of a camera video stream of each sub-region of the edge device in a next time period based on the final key level of each sub-region in a current time period, and to control the edge device to perform video monitoring analysis in the next time period according to the analysis strategy.

[0054] The present application has the beneficial effects that: a video monitoring intelligent analysis method and system based on edge computing are provided, a sub-region set for each camera in a monitoring area is generated, real-time passenger flow and historical passenger flow features collected by the camera are used to generate a time-series passenger flow feature vector of each camera and a passenger flow anomaly flag of the current real-time passenger flow, a dynamic weighted undirected graph representing the topological relationship of a plurality of sub-regions in the monitoring area is constructed, a sub-region key level probability prediction model based on a spatio-temporal graph convolution network and the dynamic weighted undirected graph is inputted with the fusion time-series passenger flow feature vector and the feature matrix of the passenger flow anomaly flag, and the final key level of each sub-region is obtained after prediction and correction, and the edge device is controlled to perform frame extraction and video picture analysis at a corresponding frame rate for each camera video stream according to the final key level. Thus, the present application considers the actual differences between different sub-regions in the monitoring area, obtains real-time and historical passenger flow features and anomaly flags, and fuses the physical distance and topological connectivity between each sub-region, so that the key level can accurately respond to scenarios such as morning and evening peak and sudden passenger flow, and the possibility of isolated high-level regions is reduced, balancing video analysis accuracy and edge device computing power load. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 FIG. 1 is a flowchart of a video monitoring intelligent analysis method based on edge computing according to the present application;

[0056] Figure 2 FIG. 2 is a structural diagram of a video monitoring intelligent analysis system based on edge computing according to the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical scheme and advantages of the present application clearer, further detailed description will be made to the present application in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application, and are not used to limit the present application.

[0058] The embodiment of the present application provides a video monitoring intelligent analysis method based on edge computing, referring to Figure 1 , Figure 1 FIG. 1 is a flowchart of a video monitoring intelligent analysis method based on edge computing according to the embodiment of the present application.

[0059] In the embodiment, the video monitoring intelligent analysis method based on edge computing comprises the following steps.

[0060] S1: based on monitoring area geographic information and camera view parameter, a sub-area set is generated according to camera coverage range by grid discretization area;

[0061] S2: a time sequence people flow feature vector of each camera and a people flow anomaly flag of current real-time people flow are generated by using real-time people flow and historical people flow features collected by each camera;

[0062] S3: a dynamic weighted undirected graph representing topological relationship of several sub-areas in the monitoring area is constructed according to monitoring node set of each camera, edge set determined by spatial position of each camera in the monitoring area and edge weight of fusion spatial distance and people flow similarity;

[0063] S4: a feature matrix of fusion time sequence people flow feature vector and people flow anomaly flag is input into a sub-area key level probability prediction model based on spatio-temporal graph convolution network and dynamic weighted undirected graph, key level probability of each sub-area is predicted, and original key level of each sub-area is determined based on the key level probability;

[0064] S5: according to original key level of each sub-area and dynamic weighted undirected graph, key level correction of each sub-area is performed according to key level correction rule, and finally key level is obtained;

[0065] S6: based on finally key level of each sub-area in the current period, an analysis strategy of camera video stream of each sub-area of the edge device in the next period is generated, and the edge device is controlled to perform video monitoring analysis in the next period according to the analysis strategy.

[0066] It should be noted that current edge boxes generally use fixed frequencies for video decoding and inference analysis, without considering the actual differences between different sub-areas within the monitored area (for example, deploying cameras in a large shopping mall to monitor pedestrian flow and identify people at various locations): First, densely populated areas (such as intersections and entrances / exits) require high-frequency analysis to ensure safety, but high-frequency analysis of sparsely populated areas (such as remote passages) would result in a waste of computing power; Second, there are spatial topological relationships between sub-areas (such as the interconnectedness of pedestrian flow in adjacent passages), and judging the importance of a single area in isolation is prone to misjudgment; Third, pedestrian flow has obvious temporal patterns (such as morning and evening rush hours, and differences between weekdays and weekends), and static segmentation strategies cannot adapt to dynamic changes.

[0067] To address the aforementioned issues, this embodiment generates a set of sub-regions for each camera within the monitoring area, along with the temporal pedestrian flow feature vector for each camera and pedestrian flow anomaly indicators for the current real-time pedestrian flow. This constructs a dynamic weighted undirected graph representing the topological relationships of several sub-regions within the monitoring area. Then, by inputting the feature matrix, which fuses the temporal pedestrian flow feature vector and pedestrian flow anomaly indicators, into a sub-region key level probabilistic prediction model based on a spatiotemporal graph convolutional network and the dynamic weighted undirected graph, the final key level of each sub-region is obtained after prediction and correction. This allows for frame extraction and video image analysis at the corresponding frame rate. Therefore, by acquiring real-time and historical pedestrian flow features and anomaly indicators, and fusing the physical distance and topological connectivity between sub-regions, this invention can balance video analysis accuracy with the computing load on edge devices.

[0068] In a preferred embodiment, the step of generating a set of sub-regions based on the geographic information of the monitored area and the camera's view frustum parameters, by rasterizing and discretizing the area, and according to the camera's coverage area, specifically includes:

[0069] S11: Obtain the planar geographic information of the monitored area, and uniformly discretize the plane of the monitored area into a cellular grid of a preset size to form a grid set. m represents the number of grid cells;

[0070] S12: Based on each camera The projection range of the view cone on the ground is marked by all the cell grids covered by the camera, forming a sub-region. , This represents the i-th camera. Let represent the sub-region corresponding to the i-th camera, and k represent the k-th grid in the grid set;

[0071] S13: Summarize the sub-regions corresponding to all cameras to form a sub-region set. ; where any two subregions in the set R and Spatial overlap is allowed, where n represents the number of sub-regions. and Let represent the sub-region corresponding to the i-th camera and the sub-region corresponding to the j-th camera, respectively.

[0072] In this embodiment, the monitored area is uniformly discretized into 0.5m × 0.5m cellular grids. Then, based on the viewing cone parameters (focal length, installation angle, etc.) of each camera, its projection range on the ground is calculated. The covered cellular grids are marked, forming sub-regions corresponding to individual cameras. Finally, the sub-regions of all cameras are aggregated, allowing spatial overlap (adapting to scenarios where multiple cameras overlap to supplement blind spots). This results in a set of analyzable sub-regions. By employing rasterization discretization and camera coverage marking, the physical monitored area is transformed into a structured set of sub-regions, ensuring a balance between spatial accuracy and computational efficiency. This lays the foundation for subsequent pedestrian flow feature extraction and spatial correlation analysis.

[0073] In a preferred embodiment, the step of generating a temporal pedestrian flow feature vector for each camera and a pedestrian flow anomaly flag for the current real-time pedestrian flow, using the real-time and historical pedestrian flow features collected by each camera, specifically includes:

[0074] S21: For the video stream of the current time period t, the YOLOv5 object detection algorithm is used to identify pedestrian targets, and then the DeepSORT multi-object tracking algorithm is used to remove duplicate counts to obtain the current number of people. , This represents the number of people passing by the i-th camera in the current time period t;

[0075] S22: Retrieve historical pedestrian flow data from the storage unit for the previous two time periods. and Query historical pedestrian flow data for the same period and calculate the average. and standard deviation , and Let represent the mean and standard deviation of the number of people passing through the i-th camera in the current time period t and in the historical time periods t-1 and t-2, respectively.

[0076] S23: Construct a system containing the current number of people. Historical visitor numbers and mean and standard deviation Temporal human flow feature vector , This represents the temporal human flow feature vector of the i-th camera in the current time period t;

[0077] S24: Determine the current number of people. If the abortion rate exceeds two standard deviations from the historical average, it is considered an abnormal abortion pattern. Assign a value of 1; otherwise, it serves as a marker for abnormal human flow. Assign the value 0, This indicates an abnormal flow of people at the i-th camera during time period t of the current year.

[0078] In this embodiment, a combined approach of YOLOv5 object detection and DeepSORT multi-object tracking is adopted. YOLOv5 quickly identifies pedestrian targets in the video stream, while DeepSORT removes duplicate counts of the same pedestrian through trajectory correlation to obtain the pedestrian count for the current time period (e.g., 5 minutes). Then, the pedestrian counts for the previous two time periods are extracted from the storage unit, and historical data for the same week and time period over the past 7 days are queried to calculate the mean and standard deviation. By combining real-time and historical pedestrian counts and statistical features, a 5-dimensional temporal pedestrian flow feature vector is formed. Meanwhile, based on the principle of 2 standard deviations (95% of normal data fall within the range of μ±2σ), it serves as a marker of abnormal human abortion. Values ​​are assigned. Thus, by integrating real-time dynamic features with historical patterns, comprehensive data on pedestrian flow is provided for determining the key levels of sub-regions.

[0079] In a preferred embodiment, the step of constructing a dynamic weighted undirected graph representing the topological relationships of several sub-regions within the monitoring area, based on the monitoring node set of each camera, the edge set determined by the spatial location of each camera within the monitoring area, and the edge weights that fuse spatial distance and pedestrian flow similarity, specifically includes:

[0080] S31: Construct a monitoring node set The Delaunay triangulation algorithm is used to spatially partition the monitoring nodes. If any two nodes... and If the Euclidean distance between them does not exceed a preset distance threshold, then an edge is established between them. , forming an edge set n represents the number of monitoring nodes corresponding to the camera, and i and j represent the i-th and j-th monitoring nodes, respectively. This represents the edge between the i-th and j-th monitoring nodes;

[0081] S32: Calculate each edge based on spatial distance and pedestrian flow similarity. weight Combined monitoring node set Edge set With border rights This forms a dynamic weighted undirected graph. , This represents the edge between the i-th and j-th monitoring nodes. The weight.

[0082] Furthermore, each edge weight The expression is as follows:

[0083] ;

[0084] In the formula, Indicates monitoring node and The plane Euclidean distance, Indicates the distance to the reference value. Indicates monitoring node and The spatial distance decay function is related to spatial distance. Indicates monitoring node and The minimum-to-maximum ratio is related to the similarity of pedestrian flow. and These are the weighting coefficients. .

[0085] In this embodiment, a node set is formed by mapping each sub-region to a node in the graph. Then, the Delaunay triangulation algorithm is used to spatially partition the nodes (maximizing the minimum angle and avoiding narrow triangles). If the Euclidean distance between any two nodes is less than or equal to a preset distance threshold, then an edge is established. , forming an edge set Finally, by calculating each edge weight And combine them to form a dynamic weighted undirected graph. This dynamic weighted undirected graph The edge set and edge weights are updated over time (due to changes in pedestrian flow and equipment adjustments). Therefore, the edge set generated through Delaunay triangulation can reasonably reflect the spatial proximity relationships between sub-regions, reduce redundant edges, and filter out unrelated distant nodes using distance thresholds, ensuring the effectiveness of the graph structure. Simultaneously, the dynamic update mechanism allows the graph structure to adapt to scene changes (such as camera position adjustments), preventing topological relationships from becoming outdated and providing a carrier for the spatial feature aggregation of subsequent spatiotemporal graph convolutional networks.

[0086] It should be noted that when calculating each edge weight At that time, it was based on the similarity between spatial distance and pedestrian flow, that is Includes spatial distance attenuation term Similarity to pedestrian flow Among them, the exponential function The closer the distance, the stronger the spatial correlation (the closer the value is to 1). α=0.6 assigns a higher weight to spatial correlation. In this process, the similarity of current pedestrian flow between two sub-regions is calculated using the minimum-maximum ratio (values ​​from 0 to 1). The closer the value is to 1, the higher the similarity. β=0.4 assigns a secondary weight to pedestrian flow linkage to balance the influence of spatial and dynamic pedestrian flow. The edge weight is obtained by adding the two results. (0-1), the larger the value, the stronger the comprehensive correlation between the two sub-regions.

[0087] In a preferred embodiment, the feature matrix fusing temporal pedestrian flow feature vectors and pedestrian flow anomaly markers is input into a sub-region key level probability prediction model based on a spatiotemporal graph convolutional network and a dynamic weighted undirected graph to predict the key level probability of each sub-region. Based on the key level probabilities, the step of determining the original key level of each sub-region specifically includes:

[0088] S41: Transform the temporal human flow feature vector and abnormal flow of people Feature splicing is performed to serve as a monitoring node. Feature matrix ;

[0089] S42: The feature matrix is ​​processed through a two-layer spatiotemporal graph convolutional network. Perform aggregation, combining the feature matrix As the first layer of the spatiotemporal graph convolutional network, the feature matrix is ​​obtained. , the feature matrix As the second layer of the spatiotemporal graph convolutional network, a 32-dimensional feature matrix is ​​obtained. ;

[0090] The aggregation expression for each layer in the two-layer spatiotemporal graph convolutional network is as follows:

[0091] ;

[0092] In the formula, Indicates the first The feature matrix of the layer, Indicates the first The feature matrix of the layer, Represents a dynamic weighted undirected graph The adjacency matrix, Represents the identity matrix. Degree matrix, Represents a symmetric normalized adjacency matrix. Indicates the first The trainable weight matrix of the layer, Represents the ReLU activation function;

[0093] S43: Transform the characteristic matrix The input is transformed into dimensions using a fully connected layer, and the Sotfmax function is used to compute the values ​​for each region. probability vector The expression is as follows:

[0094] ;

[0095] In the formula, , respectively, represent the probabilities of the corresponding L, M, and H levels, where L represents the low-frequency level, M represents the mid-frequency level, and H represents the high-frequency level.

[0096] Based on this, the method further includes:

[0097] S44: In each incremental training cycle, the time-series pedestrian flow feature vectors generated in the previous two incremental training cycles are used as input. Abnormal pedestrian flow signs Dynamic weighted undirected graph of the monitoring area Training samples were constructed using key event samples that were manually labeled;

[0098] S45: Input the training samples into a two-layer spatiotemporal graph convolutional network, perform incremental training using the cross-entropy loss function, and update the trainable parameters of the spatiotemporal graph convolutional network. This allows us to obtain a probabilistic prediction model for the critical level of a sub-region that is updated over time.

[0099] In this embodiment, the temporal pedestrian flow feature vector With abnormal flags Concatenate to form the initial feature matrix of the nodes. It provides a temporal dimension basis, and then aggregates the feature matrix through a two-layer spatiotemporal graph convolutional network (ST-GCN). During the aggregation process, and From dynamic weighted undirected graph This generates the spatial topology, which is then used to generate the symmetric normalized adjacency matrix. To ensure that aggregation is unaffected by node differences, the 32-dimensional feature matrix output from the second layer is finally input into the fully connected layer to transform its dimensions. The Softmax function is then used to obtain probability vectors for three levels: L (low frequency), M (mid frequency), and H (high frequency). The level corresponding to the highest probability is taken as the original key level. Therefore, this embodiment utilizes the spatiotemporal fusion capability of ST-GCN to combine the temporal pedestrian flow characteristics and spatial correlation characteristics of sub-regions to achieve more accurate level prediction.

[0100] It should be noted that during incremental training, in each incremental training cycle (e.g., 24 hours), the temporal pedestrian flow features from the previous two cycles are included. Abnormal pedestrian flow signs and dynamic weighted undirected graph The training samples are constructed by combining manually labeled key event samples (such as congestion and missed detections). The training samples are then input into ST-GCN, and the cross-entropy loss function (which measures the difference between the predicted probability and the labeled probability) is used for training. The trainable parameters are updated through gradient descent. After training, the old model is replaced to obtain a key level prediction model adapted to the latest scene. By continuously learning the features of the new scene, the model can avoid overfitting or prediction failure due to scene changes (such as changes in pedestrian flow patterns or camera adjustments).

[0101] In a preferred embodiment, the step of correcting the key level of each sub-region based on the original key level and the dynamically weighted undirected graph of each sub-region, and obtaining the final key level, specifically includes:

[0102] S51: For each sub-region, perform smoothing processing based on historical time periods on the original key level in the current time period. Using the smoothed original key level and the dynamic weighted undirected graph, perform key level correction according to the first correction rule and the second correction rule to obtain the final key level.

[0103] The first correction rule is configured to: for a sub-region with an original key level of H, calculate its position in the dynamic weighted undirected graph. The shortest path hop count between the middle and other H-level sub-regions is used. If the shortest path hop count is greater than the target hop count, the sub-region is downgraded to M critical level.

[0104] The second correction rule is configured to: determine the historical average pedestrian flow for sub-regions with an original key level of L over a consecutive preset number of time periods. If the number of people is less than the target low-traffic threshold, upgrade the sub-area to M-level criticality.

[0105] In this embodiment, the probability vector of the original key level is first smoothed to balance real-time prediction and historical trends, reducing the impact of instantaneous fluctuations (e.g., by using...). The smoothing formula is used to process the key level (k is 0.65), and then the key level is corrected by the first rule of isolated H downgrading and the second rule of continuous L upgrading. Finally, the smoothing and correction results are integrated to obtain the final key level of each sub-region.

[0106] It should be noted that the first rule mainly considers that isolated H-level sub-regions (such as remote corners misjudged as high criticality levels) will lead to meaningless high-frequency analysis and waste of computing power. In practical applications, the target hop count is set to 3 hops, that is, sub-regions with more than 3 hops usually have no actual linkage and are judged as isolated. The second rule mainly considers that if L-level regions with low traffic are continuously analyzed at low frequency (such as 0.2fps), there may be missed detections due to the low frame rate (such as occasional abnormal events). Upgrading to M (1fps) can improve the reliability of monitoring with a small increase in computing power.

[0107] In a preferred embodiment, based on the final criticality level of each sub-region in the current time period, an analysis strategy for the camera video stream of each sub-region is generated for the edge device in the next time period. According to the analysis strategy, the edge device is controlled to execute video surveillance analysis steps in the next time period, specifically including:

[0108] S61: Based on the final key level of each sub-region in the current time period, and according to the preset mapping relationship between the final key level and the video analysis frequency, determine the video analysis frequency of the camera video stream for each sub-region in the next time period.

[0109] S62: Based on the video analysis frequency and the video frame rate of each camera, generate a frame extraction frequency for the camera video stream, and control the edge device to perform frame extraction and video image analysis in the next time period.

[0110] In this embodiment, based on the mapping relationship between critical level and video analysis frequency (e.g., L: 0.2fps, M: 1fps, H: 4fps), the video analysis frequency for the next time period is determined according to the final critical level of each sub-region. Then, based on the video analysis frequency and the camera's original frame rate (e.g., 25fps), the frame extraction interval is calculated (e.g., one frame is extracted every 125 frames for level L). The controller of the edge device executes video decoding and analysis according to the frequency, ensuring that the computing power allocation matches the importance of the sub-region. While ensuring the accuracy of high-level regions, the amount of data processing in low-level regions is significantly reduced, achieving overall load optimization.

[0111] Reference Figure 2 , Figure 2 This is a schematic diagram of the structure of the intelligent video surveillance analysis system based on edge computing, according to an embodiment of the present invention.

[0112] like Figure 2 As shown, the video surveillance intelligent analysis system based on edge computing proposed in this embodiment of the invention includes:

[0113] Rasterization module 10 is used to generate a set of sub-regions based on the geographic information of the monitored area and the camera frustum parameters by rasterizing the area and discretizing it according to the camera coverage.

[0114] The generation module 20 is used to generate a time-series pedestrian flow feature vector and a pedestrian flow anomaly flag for each camera by utilizing the real-time pedestrian flow and historical pedestrian flow features collected by each camera.

[0115] Module 30 is used to construct a dynamic weighted undirected graph representing the topological relationship of several sub-regions in the monitoring area based on the monitoring node set of each camera, the edge set determined by the spatial position of each camera in the monitoring area, and the edge weights that fuse spatial distance and human flow similarity.

[0116] Prediction module 40 is used to input the feature matrix of the fused temporal pedestrian flow feature vector and pedestrian flow anomaly marker into the sub-region key level probability prediction model based on spatiotemporal graph convolutional network and dynamic weighted undirected graph, predict the key level probability of each sub-region, and determine the original key level of each sub-region based on the key level probability.

[0117] The correction module 50 is used to perform key level correction for each sub-region based on the key level correction rules, and obtain the final key level.

[0118] The analysis module 60 is used to generate an analysis strategy for the camera video stream of each sub-region in the next time period based on the final key level of each sub-region in the current time period, and to control the edge device to perform video surveillance analysis in the next time period according to the analysis strategy.

[0119] Other embodiments or specific implementations of the video surveillance intelligent analysis system based on edge computing of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0120] It is understood that in the description of this specification, references to terms such as "one embodiment," "another embodiment," "other embodiments," or "first embodiment to Nth embodiment," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0121] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0122] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. An edge computing-based intelligent analysis method for video monitoring, characterized in that, The method comprises the following steps: Based on the monitoring area geographic information and the camera view cone parameters, a set of sub-regions is generated according to the camera coverage range by grid discretization of the region; Real-time and historical traffic flow features collected by each camera are used to generate a time-series traffic flow feature vector and a traffic flow anomaly flag of the current real-time traffic flow of each camera; A dynamic weighted undirected graph representing the topological relationship of a plurality of sub-regions in the monitoring area is constructed according to the monitoring node set of each camera, the edge set determined by the spatial position of each camera in the monitoring area, and the edge weight of the fusion spatial distance and the traffic flow similarity, and comprises the following steps: Constructing a monitoring node set , the Delaunay triangulation algorithm is used to divide the space of the monitoring nodes, if the Euclidean distance between any two nodes and is not more than a preset distance threshold, an edge is established between the two nodes, forming an edge set , n represents the number of monitoring nodes corresponding to the camera, i and j represent the i th and j th monitoring nodes respectively, represents the edge between the i th and j th monitoring nodes. Calculate each edge based on spatial distance and pedestrian flow similarity. weight Combined monitoring node set Edge set With border rights This forms a dynamic weighted undirected graph. , This represents the edge between the i-th and j-th monitoring nodes. The weights; wherein each edge has a weight expressed by the expression, in particular: ； In the formula, represents the monitoring node and the plane Euclidean distance, represents the distance reference value, represents the monitoring node and the spatial distance attenuation function, associated with the spatial distance, represents the monitoring node and the minimum maximum ratio, associated with the crowd flow similarity, and is a weight coefficient, ; A feature matrix of the fusion time-series traffic flow feature vector and the traffic flow anomaly flag is input into a sub-region key level probability prediction model based on the spatio-temporal graph convolution network and the dynamic weighted undirected graph, and the key level probability of each sub-region is predicted, and the original key level of each sub-region is determined based on the key level probability; The key level correction of each sub-region is performed according to the key level correction rule, and the final key level is obtained; Based on the final key level of each sub-region in the current period, an analysis strategy of the camera video stream of each sub-region for the edge device in the next period is generated, and the edge device is controlled to perform video monitoring analysis in the next period according to the analysis strategy. 2.The edge computing based video surveillance intelligent analysis method of claim 1, wherein, The method comprises the following steps: The planar geographic information of the monitoring area is acquired, the monitoring area is uniformly discretized into a cell grid with a preset size, and a grid set is formed , m represents the number of grids According to each camera The projection range of the view cone on the ground is marked by all the cell grids covered by the camera, forming a sub-region. , This represents the i-th camera. Let represent the sub-region corresponding to the i-th camera, and k represent the k-th grid in the grid set; Collecting all the sub-regions corresponding to the cameras to form a sub-region set ; wherein any two of the sub-region set R And Allow spatial overlap, n represents the number of sub-regions, And Respectively, the i-th camera corresponds to the sub-region and the j-th camera corresponds to the sub-region. 3.The edge computing based video surveillance intelligent analysis method of claim 1, wherein, Real-time and historical traffic flow features collected by each camera are used to generate a time-series traffic flow feature vector and a traffic flow anomaly flag of the current real-time traffic flow of each camera, and the method comprises the following steps: For the video stream of the current period t, a YOLOv5 target detection algorithm is used to identify pedestrian targets, and then a DeepSORT multi-target tracking algorithm is used to remove repeated counting to obtain the current number of people flow , represents the number of people flow of the i th camera at the current period t Extract the historical flow number of the first two periods from the storage unit With , query the historical flow data of the same period, calculate the mean And standard deviation , And The mean and standard deviation of the flow number of the ith camera at the current period t and the historical period t-1 and t-2 are represented by Construct a system containing the current number of people. Historical visitor numbers and mean and standard deviation Temporal human flow feature vector , This represents the temporal human flow feature vector of the i-th camera in the current time period t; determining whether the current people flow count exceeds the historical average by 2 standard deviations, and if so, assigning a value of 1 to the people flow anomaly flag determining whether the current people flow count exceeds the historical average by 2 standard deviations, and if so, assigning a value of 1 to the people flow anomaly flag determining whether the current people flow count exceeds the historical average by 2 standard deviations, and if so, assigning a value of 1 to the people flow anomaly flag determining whether the current people flow count exceeds the historical average by 2 standard deviations, and if so, assigning a value of 1 to the people flow anomaly flag representing the people 4.The edge computing based video surveillance intelligent analysis method of claim 1, wherein, A feature matrix of the fusion time-series traffic flow feature vector and the traffic flow anomaly flag is input into a sub-region key level probability prediction model based on the spatio-temporal graph convolution network and the dynamic weighted undirected graph, and the key level probability of each sub-region is predicted, and the original key level of each sub-region is determined based on the key level probability; Time-series human flow feature vector and abnormal flow of people Feature splicing is performed to serve as a monitoring node. Feature matrix ; by a two-layer spatio-temporal graph convolutional network, aggregating the feature matrix as a first layer spatio-temporal graph convolutional network, obtaining a feature matrix as a first layer spatio-temporal graph convolutional network, obtaining a feature matrix as a first layer spatio-temporal graph convolutional network, obtaining a feature matrix as a second layer spatio-temporal graph convolutional network, obtaining a 32-dimensional feature matrix ; The aggregation expression of each layer of the two-layer spatio-temporal graph convolution network is specifically as follows: ； wherein, represents the first layer's feature matrix, represents the first layer's feature matrix, represents the adjacency matrix of the dynamic weighted undirected graph , represents the identity matrix, represents the degree matrix, represents the symmetric normalized adjacency matrix, represents the first layer's trainable weight matrix, represents the ReLU activation function; feature matrix The input is transformed into dimensions using a fully connected layer, and the Sotfmax function is used to compute the values ​​for each region. probability vector The expression is as follows: ； wherein respectively represent the probabilities corresponding to the L, M, H levels, L representing the low frequency level, M representing the medium frequency level, H representing the high frequency level. 5.The edge computing based video surveillance intelligent analysis method of claim 4, wherein, The method further comprises the following steps: In each incremental training cycle, the time-series passenger flow feature vectors generated by the past two incremental training cycles , passenger flow anomaly flag and dynamic weighted undirected graph of the monitoring area and the manually annotated key event samples are constructed as training samples; The training sample is input into a two-layer spatio-temporal graph convolution network, an incremental training is performed by using a cross-entropy loss function, and a trainable parameter of the spatio-temporal graph convolution network is updated , to obtain a sub-region key level probability prediction model updated over time. 6.The edge computing based video surveillance intelligent analysis method of claim 1, wherein, The key level correction of each sub-region is performed according to the key level correction rule, and the final key level is obtained; The original key level of each sub-region in the current period is smoothed based on the historical period, and the final key level is obtained by performing key level correction according to the first correction rule and the second correction rule using the smoothed original key level and the dynamic weighted undirected graph; The first correction rule is configured to: for a sub-region with an original key level of H, calculate the shortest path hop count of the sub-region in a dynamic weighted undirected graph with other H-level sub-regions, and if the shortest path hop count is greater than a target hop count, downgrade the sub-region to an M key level. The second correction rule is configured to: determine the historical average pedestrian flow for sub-regions with an original key level of L over a consecutive preset number of time periods. If the number of people is less than the target low-traffic threshold, upgrade the sub-area to M-level criticality. 7.The edge computing based video surveillance intelligent analysis method of claim 1, wherein, Based on the final key level of each sub-region in the current period, an analysis strategy of the camera video stream of each sub-region for the edge device in the next period is generated, and the edge device is controlled to perform video monitoring analysis in the next period according to the analysis strategy. Based on the final key level of each sub-region in the current period, a video analysis frequency of the camera video stream of each sub-region for the edge device in the next period is determined according to a preset mapping relationship between the final key level and the video analysis frequency. According to the video analysis frequency and the video frame rate of each camera, a frame extraction frequency for the camera video stream is generated, and the edge device is controlled to perform frame extraction and video picture analysis in the next period.

8. An edge computing based intelligent video surveillance analysis system, characterized in that, Comprise: The gridding module is configured to discretize the region by gridding based on the geographical information of the monitoring region and the camera view cone parameters, and generate a set of sub-regions according to the coverage of the camera; The generation module is configured to generate a time series flow feature vector of each camera and a flow anomaly flag of the current real-time flow by using real-time flow and historical flow features collected by each camera; The construction module is configured to construct a dynamic weighted undirected graph representing the topological relationship of a plurality of sub-regions in the monitoring region according to the monitoring node set of each camera, the edge set determined by the spatial position of each camera in the monitoring region, and the edge weight fused with the spatial distance and flow similarity; Specifically comprising: Constructing a monitoring node set , the Delaunay triangulation algorithm is used to divide the space of the monitoring nodes, if the Euclidean distance between any two nodes and is not more than a preset distance threshold, an edge is established between the two nodes, forming an edge set , n represents the number of monitoring nodes corresponding to the camera, i and j represent the i th and j th monitoring nodes respectively, represents the edge between the i th and j th monitoring nodes; Calculate each edge based on spatial distance and pedestrian flow similarity. weight Combined monitoring node set Edge set With border rights This forms a dynamic weighted undirected graph. , This represents the edge between the i-th and j-th monitoring nodes. The weights; wherein each edge of the weight of the expression, in particular: ； In the formula, represents the monitoring node with the planar Euclidean distance, represents the distance reference value, represents the monitoring node with the spatial distance decay function, associated with the spatial distance, represents the monitoring node with the minimum maximum ratio, associated with the crowd flow similarity, with is a weight coefficient, ; The prediction module is configured to input the feature matrix fused with the time series flow feature vector and the flow anomaly flag into a sub-region key level probability prediction model based on the spatio-temporal graph convolution network and the dynamic weighted undirected graph, to predict the key level probability of each sub-region, and determine the original key level of each sub-region based on the key level probability. The correction module is configured to perform key level correction of each sub-region according to the key level correction rule based on the original key level of each sub-region and the dynamic weighted undirected graph, to obtain the final key level. The analysis module is configured to generate an analysis strategy of the camera video stream of each sub-region for the edge device in the next period based on the final key level of each sub-region in the current period, and control the edge device to perform video monitoring analysis in the next period according to the analysis strategy.

Images