Video monitoring intelligent analysis method and system based on edge calculation

Abstract

The invention relates to the technical field of data processing, and discloses a video monitoring intelligent analysis method and system based on edge computing, and the method comprises the steps: generating a sub-region set for each camera in a monitoring region, a time sequence people flow feature vector of each camera, and a people flow abnormal sign of the current real-time people flow; a dynamic weighted undirected graph representing a topological relation of a plurality of sub-regions in a monitoring region is constructed, and then a feature matrix fusing time sequence people flow feature vectors and people flow abnormal marks is input into a sub-region key level probability prediction model based on a space-time diagram convolutional network and the dynamic weighted undirected graph. And the final key level of each sub-region is obtained after prediction and correction, so that edge equipment is controlled to execute frame extraction and video picture analysis of a corresponding frame rate. Therefore, the video analysis precision and the computing power load of edge equipment can be balanced by acquiring the real-time and historical people flow characteristics and the abnormal marks and fusing the physical distance and the topological connectivity between the sub-regions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a video monitoring intelligent analysis method and system based on edge computing. BACKGROUND

[0002] With the wide application of video monitoring technology in the fields of security, traffic and smart city, the scale of the monitoring area is continuously expanding, the number of deployed cameras is continuously increasing, and edge computing has become the core carrier of video analysis due to its low delay and high real-time advantage.

[0003] However, the current edge box generally uses a fixed frequency for video decoding and inference analysis, without considering the actual differences of different sub-areas in the monitoring area (for example, the deployment of cameras in large shopping malls for people flow monitoring and personnel identification analysis in each position): first, high-frequency analysis is required in densely populated areas (such as intersections and entrances) to ensure safety, but high-frequency analysis in sparsely populated areas (such as remote passages) will cause waste of computing power; second, there is a spatial topological relationship between sub-areas (such as the linkage of people flow in adjacent passages), and isolated judgment of the importance of a single area is prone to misjudgment; third, people flow has obvious time sequence rules (such as morning and evening peak, weekday and weekend differences), and static division strategies cannot adapt to dynamic changes.

[0004] Therefore, how to consider the actual differences of different sub-areas in the monitoring area and avoid the problems of excessive overall computing power load or insufficient video analysis accuracy caused by the edge box performing video picture analysis on each camera video stream with the same frequency is a technical problem that needs to be solved. SUMMARY

[0005] The present application provides a video monitoring intelligent analysis method and system based on edge computing, which aims to solve at least one of the above technical problems.

[0006] To achieve the above purpose, the present application provides a video monitoring intelligent analysis method based on edge computing, comprising: Based on the geographic information of the monitoring area and the camera frustum parameters, the region is discretized by gridding, and a sub-area set is generated according to the camera coverage range; Using the real-time people flow and historical people flow features collected by each camera, a time sequence people flow feature vector of each camera and a people flow anomaly flag of the current real-time people flow are generated; According to the monitoring node set of each camera, the edge set determined by the spatial position of each camera in the monitoring area, and the edge weight of the fusion spatial distance and people flow similarity, a dynamic weighted undirected graph representing the topological relationship of several sub-areas in the monitoring area is constructed; inputting the fusion time-series passenger flow feature vector and the feature matrix of the passenger flow anomaly flag into a sub-region key level probability prediction model based on a spatio-temporal graph convolution network and a dynamic weighted undirected graph, to predict a key level probability of each sub-region, and determining an original key level of each sub-region based on the key level probability; performing key level correction of each sub-region according to a key level correction rule based on the original key level of each sub-region and the dynamic weighted undirected graph, to obtain a final key level; generating an analysis strategy of the camera video stream of the edge device for each sub-region in the next time period based on the final key level of each sub-region in the current time period, and controlling the edge device to perform video monitoring analysis in the next time period according to the analysis strategy.

[0007] Optionally, the step of generating a sub-region set according to the camera coverage range based on the monitoring area geographic information and the camera view cone parameters comprises: acquiring planar geographic information of the monitoring area, uniformly discretizing the monitoring area planarly into a cell grid of a preset size to form a grid set , wherein m represents the number of grids; according to the projection range of the view cone of each camera on the ground, marking all cell grids covered by the camera to constitute a sub-region , wherein i represents the i-th camera, represents the i-th camera corresponding sub-region, and k represents the k-th grid in the grid set; summarizing all camera corresponding sub-regions to form a sub-region set ; wherein any two and in the sub-region set R allow spatial overlap, n represents the number of sub-regions, and respectively represent the i-th camera corresponding sub-region and the j-th camera corresponding sub-region.

[0008] Optionally, the step of generating a time-series passenger flow feature vector of each camera and a passenger flow anomaly flag of the current real-time passenger flow using real-time passenger flow and historical passenger flow features collected by each camera comprises: using a YOLOv5 target detection algorithm to identify pedestrian targets for the video stream of the current time period t, and then removing repeated counting through a DeepSORT multi-target tracking algorithm to obtain a current passenger flow number , wherein represents the passenger flow number of the i-th camera in the current time period t; extracting historical passenger flow numbers of the previous two time periods from a storage unit, Query historical pedestrian flow data for the same period and calculate the average. and standard deviation , and Let represent the mean and standard deviation of the number of people passing through the i-th camera in the current time period t and in the historical time periods t-1 and t-2, respectively. Construct a system containing the current number of people. Historical visitor numbers and mean and standard deviation Temporal human flow feature vector , This represents the temporal human flow feature vector of the i-th camera in the current time period t; Determine the current number of people If the abortion rate exceeds two standard deviations from the historical average, it is considered an abnormal abortion pattern. Assign a value of 1; otherwise, it serves as a marker for abnormal human flow. Assign the value 0, This indicates an abnormal flow of people at the i-th camera during time period t of the current year.

[0009] Optionally, based on the monitoring node set of each camera, the edge set determined by the spatial location of each camera within the monitoring area, and the edge weights that fuse spatial distance and pedestrian flow similarity, a dynamic weighted undirected graph representing the topological relationships of several sub-regions within the monitoring area is constructed. This specifically includes: Build a monitoring node set The Delaunay triangulation algorithm is used to spatially partition the monitoring nodes. If any two nodes... and If the Euclidean distance between them does not exceed a preset distance threshold, then an edge is established between them. , forming an edge set n represents the number of monitoring nodes corresponding to the camera, and i and j represent the i-th and j-th monitoring nodes, respectively. This represents the edge between the i-th and j-th monitoring nodes; Calculate each edge based on spatial distance and pedestrian flow similarity. weight Combined monitoring node set Edge set With border rights This forms a dynamic weighted undirected graph. , This represents the edge between the i-th and j-th monitoring nodes. The weight.

[0010] Optionally, each edge weight , specifically, ; wherein, denotes the monitoring node and a planar Euclidean distance, denotes a distance reference value, denotes the spatial distance decay function between the monitoring node and , associated with the spatial distance, denotes the minimum maximum ratio between the monitoring node and , associated with the crowd flow similarity, and is a weight coefficient, .

[0011] Optionally, the feature matrix of the fusion time-series crowd flow feature vector and the crowd flow anomaly flag is input into a sub-region key level probability prediction model based on a spatio-temporal graph convolution network and a dynamic weighted undirected graph, to predict the key level probability of each sub-region, and based on the key level probability, determine the original key level of each sub-region, specifically comprising: the time-series crowd flow feature vector and the crowd flow anomaly flag are spliced as the feature matrix of the monitoring node ; the feature matrix is aggregated by a two-layer spatio-temporal graph convolution network, the feature matrix is taken as the first-layer spatio-temporal graph convolution network to obtain the feature matrix , and the feature matrix is taken as the second-layer spatio-temporal graph convolution network to obtain a 32-dimensional feature matrix ; wherein, the aggregation expression of each layer in the two-layer spatio-temporal graph convolution network is specifically: ; wherein, denotes the feature matrix of the th layer, denotes the feature matrix of the th layer, denotes the adjacency matrix of the dynamic weighted undirected graph , denotes the unit matrix, denotes the degree matrix, denotes the symmetric normalized adjacency matrix, denotes the trainable weight matrix of the th layer, Represents the ReLU activation function; feature matrix The input is fully connected layer for dimensionality transformation, and the Sotfmax function is used to compute the dimensionality of each region. probability vector The expression is as follows: ; In the formula, , respectively, represent the probabilities of the corresponding L, M, and H levels, where L represents the low-frequency level, M represents the mid-frequency level, and H represents the high-frequency level.

[0012] Optionally, the method further includes: In each incremental training cycle, the time-series pedestrian flow feature vectors generated in the previous two incremental training cycles are used. Abnormal pedestrian flow signs Dynamic weighted undirected graph of the monitoring area Training samples were constructed using key event samples that were manually labeled; The training samples are input into a two-layer spatiotemporal graph convolutional network, and incremental training is performed using the cross-entropy loss function to update the trainable parameters of the spatiotemporal graph convolutional network. This allows us to obtain a probabilistic prediction model for the critical level of a sub-region that is updated over time.

[0013] Optionally, for each sub-region's original key level and dynamically weighted undirected graph, the key level of each sub-region is corrected according to the key level correction rule to obtain the final key level. This process specifically includes: For each sub-region, the original key level in the current time period is smoothed based on the historical time period. Using the smoothed original key level and the dynamic weighted undirected graph, the key level is corrected according to the first correction rule and the second correction rule to obtain the final key level. The first correction rule is configured to: for a sub-region with an original key level of H, calculate its position in the dynamic weighted undirected graph. The shortest path hop count between the middle and other H-level sub-regions is used. If the shortest path hop count is greater than the target hop count, the sub-region is downgraded to M critical level. The second correction rule is configured to: determine the historical average pedestrian flow for sub-regions with an original key level of L over a consecutive preset number of time periods. If the number of people is less than the target low-traffic threshold, upgrade the sub-area to M-level criticality.

[0014] Optionally, based on the final key level of each sub-region in the current period, an analysis strategy of the camera video stream of the edge device for each sub-region in the next period is generated, according to which the video monitoring analysis step of the edge device in the next period is controlled, and specifically includes: Based on the final key level of each sub-region in the current period, according to a preset mapping relationship between the final key level and the video analysis frequency, the video analysis frequency of the camera video stream of the edge device for each sub-region in the next period is determined. According to the video analysis frequency and the video frame rate of each camera, a frame extraction frequency for the camera video stream is generated, and the edge device is controlled to perform frame extraction and video picture analysis in the next period.

[0015] In addition, in order to achieve the above-mentioned purpose, the application also provides a video monitoring intelligent analysis system based on edge computing, which comprises: The grid module is used for discretizing the region by gridding based on the monitoring region geographic information and the camera view cone parameters, and generating a sub-region set according to the camera coverage range. The generation module is used for generating a time sequence flow feature vector of each camera and a flow anomaly flag of the current real-time flow by using the real-time flow and the historical flow features collected by each camera. The construction module is used for constructing a dynamic weighted undirected graph representing the topological relationship of a plurality of sub-regions in the monitoring region according to the monitoring node set of each camera, the edge set determined by the spatial position of each camera in the monitoring region, and the edge weight fused with the spatial distance and the flow similarity. The prediction module is used for inputting a feature matrix fused with the time sequence flow feature vector and the flow anomaly flag into a sub-region key level probability prediction model based on the spatio-temporal graph convolution network and the dynamic weighted undirected graph, predicting the key level probability of each sub-region, and determining the original key level of each sub-region based on the key level probability. The correction module is used for performing key level correction of each sub-region according to the key level correction rule based on the original key level of each sub-region and the dynamic weighted undirected graph, and obtaining the final key level. The analysis module is used for generating an analysis strategy of the camera video stream of the edge device for each sub-region in the next period based on the final key level of each sub-region in the current period, and controlling the edge device to perform video monitoring analysis in the next period according to the analysis strategy.

[0016] The beneficial effects of this invention are as follows: It proposes an intelligent video surveillance analysis method and system based on edge computing. By generating a set of sub-regions for each camera within the monitoring area, and utilizing real-time and historical pedestrian flow characteristics collected by the cameras, it generates a temporal pedestrian flow feature vector and a current real-time pedestrian flow anomaly marker for each camera. A dynamic weighted undirected graph representing the topological relationships of several sub-regions within the monitoring area is constructed. Then, by inputting the feature matrix fusing the temporal pedestrian flow feature vector and the pedestrian flow anomaly marker into a sub-region key level probability prediction model based on a spatiotemporal graph convolutional network and the dynamic weighted undirected graph, the final key level of each sub-region is obtained after prediction and correction. Based on the final key level, the edge device is controlled to perform frame extraction and video image analysis at the corresponding frame rate for each camera's video stream. Therefore, this invention, by considering the actual differences between different sub-regions within the monitoring area, obtains real-time and historical pedestrian flow characteristics and anomaly markers, and integrates the physical distance and topological connectivity between each sub-region. This enables the key level to accurately respond to scenarios such as morning and evening rush hours and sudden pedestrian flows, while reducing the possibility of isolated high-level areas, thus balancing video analysis accuracy and edge device computing load. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the intelligent video surveillance analysis method based on edge computing of the present invention. Figure 2 This is a schematic diagram of the intelligent video surveillance analysis system based on edge computing according to the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] This invention provides a video surveillance intelligent analysis method based on edge computing, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the intelligent video surveillance analysis method based on edge computing, as described in an embodiment of the present invention.

[0020] In this embodiment, a video surveillance intelligent analysis method based on edge computing includes: S1: Based on the geographic information of the monitored area and the camera frustum parameters, the area is discretized by rasterization, and a set of sub-regions is generated according to the camera coverage area; S2: Utilize the real-time and historical pedestrian flow characteristics collected by each camera to generate a temporal pedestrian flow feature vector for each camera and a pedestrian flow anomaly flag for the current real-time pedestrian flow. S3: constructing a dynamic weighted undirected graph representing the topological relationship of a plurality of sub-regions in the monitoring area according to the monitoring node set of each camera, the edge set determined according to the spatial position of each camera in the monitoring area, and the edge weight of the fusion spatial distance and the crowd flow similarity; S4: inputting the fusion time sequence crowd flow feature vector and the feature matrix of the crowd flow anomaly mark into a sub-region key level probability prediction model based on the spatio-temporal graph convolution network and the dynamic weighted undirected graph, predicting the key level probability of each sub-region, and determining the original key level of each sub-region based on the key level probability; S5: according to the original key level of each sub-region and the dynamic weighted undirected graph, performing key level correction of each sub-region according to the key level correction rule to obtain the final key level; S6: generating an analysis strategy of the edge device for the camera video stream of each sub-region in the next period based on the final key level of each sub-region in the current period, and controlling the edge device to perform video monitoring analysis in the next period according to the analysis strategy.

[0021] It should be noted that the current edge box generally adopts fixed frequency for video decoding and inference analysis, and does not consider the actual differences of different sub-regions in the monitoring area (for example, the deployment of cameras in large shopping malls for people flow monitoring and personnel identification analysis in each position): first, the high-frequency analysis of the dense crowd area (such as the intersection and the entrance and exit) is needed to ensure safety, but the high-frequency analysis of the sparse crowd area (such as the remote passage) will cause waste of computing power; second, there is a spatial topological association between sub-regions (such as the linkage of people flow in adjacent passages), and isolated judgment of the importance of a single region is easy to misjudge; third, the crowd has obvious time sequence rules (such as the morning and evening peak, and the difference between weekdays and weekends), and the static division strategy cannot adapt to dynamic changes.

[0022] In order to solve the above problems, the embodiment generates a sub-region set for each camera in the monitoring area and a time sequence crowd flow feature vector and a crowd flow anomaly mark of the current real-time crowd flow, constructs a dynamic weighted undirected graph representing the topological relationship of a plurality of sub-regions in the monitoring area, and then inputs the fusion time sequence crowd flow feature vector and the feature matrix of the crowd flow anomaly mark into a sub-region key level probability prediction model based on the spatio-temporal graph convolution network and the dynamic weighted undirected graph, and obtains the final key level of each sub-region after prediction and correction, so as to perform frame extraction and video picture analysis at a corresponding frame rate. Thus, by obtaining real-time, historical crowd flow features and anomaly marks, and fusing the physical distance and topological connectivity between each sub-region, the video analysis accuracy and the computing power load of the edge device can be balanced.

[0023] In the preferred embodiment, based on the monitoring area geographic information and the camera cone parameter, the sub-region set is generated according to the camera coverage range by grid discretization of the region, specifically including: S11: Obtain the planar geographic information of the monitored area, and uniformly discretize the plane of the monitored area into a cellular grid of a preset size to form a grid set. m represents the number of grid cells; S12: Based on each camera The projection range of the view cone on the ground is marked by all the cell grids covered by the camera, forming a sub-region. , This represents the i-th camera. Let represent the sub-region corresponding to the i-th camera, and k represent the k-th grid in the grid set; S13: Summarize the sub-regions corresponding to all cameras to form a sub-region set. ; where any two subregions in the set R and Spatial overlap is allowed, where n represents the number of sub-regions. and Let represent the sub-region corresponding to the i-th camera and the sub-region corresponding to the j-th camera, respectively.

[0024] In this embodiment, the monitored area is uniformly discretized into 0.5m × 0.5m cellular grids. Then, based on the viewing cone parameters (focal length, installation angle, etc.) of each camera, its projection range on the ground is calculated. The covered cellular grids are marked, forming sub-regions corresponding to individual cameras. Finally, the sub-regions of all cameras are aggregated, allowing spatial overlap (adapting to scenarios where multiple cameras overlap to supplement blind spots). This results in a set of analyzable sub-regions. By employing rasterization discretization and camera coverage marking, the physical monitored area is transformed into a structured set of sub-regions, ensuring a balance between spatial accuracy and computational efficiency. This lays the foundation for subsequent pedestrian flow feature extraction and spatial correlation analysis. In a preferred embodiment, the step of generating a temporal pedestrian flow feature vector for each camera and a pedestrian flow anomaly flag for the current real-time pedestrian flow, using the real-time and historical pedestrian flow features collected by each camera, specifically includes: S21: For the video stream of the current time period t, the YOLOv5 object detection algorithm is used to identify pedestrian targets, and then the DeepSORT multi-object tracking algorithm is used to remove duplicate counts to obtain the current number of people. , This represents the number of people passing by the i-th camera in the current time period t; S22: Retrieve historical pedestrian flow data from the storage unit for the previous two time periods. and Query historical pedestrian flow data for the same period and calculate the average. and standard deviation , and respectively represent the mean and standard deviation of the people flow of the i-th camera at the current time period t and the historical time periods t-1 and t-2; S23: constructing a time-series people flow feature vector containing the current people flow , the historical people flow , the mean , and the standard deviation ; , denotes the time-series people flow feature vector of the i-th camera at the current time period t; S24: determining whether the current people flow exceeds the historical mean by 2 standard deviations, and if so, assigning a people flow anomaly flag a value of 1, and if not, assigning a people flow anomaly flag a value of 0, denotes the people flow anomaly flag of the i-th camera at the current time period t.

[0025] In this embodiment, a combination scheme of YOLOv5 target detection and DeepSORT multi-target tracking is adopted, that is, YOLOv5 is used to quickly identify pedestrian targets in a video stream, and DeepSORT removes repeated counting of the same pedestrian through trajectory association to obtain the people flow at the current time period (e.g., 5 minutes). The people flow at the previous two time periods is extracted from the storage unit, and the historical data of the same week and time period in the past 7 days is queried to calculate the mean and standard deviation. Through the combination of real-time and historical people flow and statistical features, a 5-dimensional time-series people flow feature vector is formed. At the same time, based on the 2 standard deviation principle (95% normal data falls within μ ± 2σ), the people flow anomaly flag is assigned a value. In this way, by fusing real-time dynamic features and historical regularity features, comprehensive people flow basis is provided for sub-regional key level judgment.

[0026] In a preferred embodiment, the step of constructing a dynamic weighted undirected graph representing the topological relationship of a plurality of sub-regions in the monitoring area according to the monitoring node set of each camera, the edge set determined by the spatial position of each camera in the monitoring area, and the edge weight of the fusion spatial distance and people flow similarity, specifically includes: S31: constructing a monitoring node set , using the Delaunay triangulation algorithm to spatially divide the monitoring nodes, and if the Euclidean distance between any two nodes and does not exceed a preset distance threshold, an edge is established between the two to form an edge set , n represents the number of monitoring nodes corresponding to the cameras, i and j respectively represent the i-th and j-th monitoring nodes,​ This represents the edge between the i-th and j-th monitoring nodes; S32: Calculate each edge based on spatial distance and pedestrian flow similarity. weight Combined monitoring node set Edge set With border rights This forms a dynamic weighted undirected graph. , This represents the edge between the i-th and j-th monitoring nodes. The weight.

[0027] Furthermore, each edge weight The expression is as follows: ; In the formula, Indicates monitoring node and The plane Euclidean distance, Indicates the distance to the reference value. Indicates monitoring node and The spatial distance decay function is related to spatial distance. Indicates monitoring node and The minimum-to-maximum ratio is related to the similarity of pedestrian flow. and These are the weighting coefficients. .

[0028] In this embodiment, a node set is formed by mapping each sub-region to a node in the graph. Then, the Delaunay triangulation algorithm is used to spatially partition the nodes (maximizing the minimum angle and avoiding narrow triangles). If the Euclidean distance between any two nodes is less than or equal to a preset distance threshold, then an edge is established. , forming an edge set Finally, by calculating each edge weight And combine them to form a dynamic weighted undirected graph. This dynamic weighted undirected graph The edge set and edge weights are updated over time (due to changes in pedestrian flow and equipment adjustments). Therefore, the edge set generated through Delaunay triangulation can reasonably reflect the spatial proximity relationships between sub-regions, reduce redundant edges, and filter out unrelated distant nodes using distance thresholds, ensuring the effectiveness of the graph structure. Simultaneously, the dynamic update mechanism allows the graph structure to adapt to scene changes (such as camera position adjustments), preventing topological relationships from becoming outdated and providing a platform for the subsequent spatial feature aggregation of spatiotemporal graph convolutional networks.

[0029] It should be noted that when calculating each edge weight At that time, it was based on the similarity between spatial distance and pedestrian flow, that is Includes spatial distance attenuation term Similarity to pedestrian flow Among them, the exponential function The closer the distance, the stronger the spatial correlation (the closer the value is to 1). α=0.6 assigns a higher weight to spatial correlation. In this process, the similarity of current pedestrian flow between two sub-regions is calculated using the minimum-maximum ratio (values ​​from 0 to 1). The closer the value is to 1, the higher the similarity. β=0.4 assigns a secondary weight to pedestrian flow linkage to balance the influence of spatial and dynamic pedestrian flow. The edge weight is obtained by adding the two results. (0-1), the larger the value, the stronger the comprehensive correlation between the two sub-regions.

[0030] In a preferred embodiment, the feature matrix fusing temporal pedestrian flow feature vectors and pedestrian flow anomaly markers is input into a sub-region key level probability prediction model based on a spatiotemporal graph convolutional network and a dynamic weighted undirected graph to predict the key level probability of each sub-region. Based on the key level probabilities, the step of determining the original key level of each sub-region specifically includes: S41: Transform the temporal human flow feature vector and abnormal flow of people Feature splicing is performed to serve as a monitoring node. Feature matrix ; S42: The feature matrix is ​​processed through a two-layer spatiotemporal graph convolutional network. Perform aggregation, combining the feature matrix As the first layer of the spatiotemporal graph convolutional network, the feature matrix is ​​obtained. , the feature matrix As the second layer of the spatiotemporal graph convolutional network, a 32-dimensional feature matrix is ​​obtained. ; The aggregation expression for each layer in the two-layer spatiotemporal graph convolutional network is as follows: ; In the formula, Indicates the first The feature matrix of the layer, Indicates the first The feature matrix of the layer, Represents a dynamic weighted undirected graph The adjacency matrix, Represents the identity matrix. Degree matrix, Represents a symmetric normalized adjacency matrix. Indicates the first The trainable weight matrix of the layer, Represents the ReLU activation function; S43: Transform the characteristic matrix The input is fully connected layer for dimensionality transformation, and the Sotfmax function is used to compute the dimensionality of each region. probability vector The expression is as follows: ; In the formula, , respectively, represent the probabilities of the corresponding L, M, and H levels, where L represents the low-frequency level, M represents the mid-frequency level, and H represents the high-frequency level.

[0031] Based on this, the method further includes: S44: In each incremental training cycle, the time-series pedestrian flow feature vectors generated in the previous two incremental training cycles are used as input. Abnormal pedestrian flow signs Dynamic weighted undirected graph of the monitoring area Training samples were constructed using key event samples that were manually labeled; S45: Input the training samples into a two-layer spatiotemporal graph convolutional network, perform incremental training using the cross-entropy loss function, and update the trainable parameters of the spatiotemporal graph convolutional network. This allows us to obtain a probabilistic prediction model for the critical level of a sub-region that is updated over time.

[0032] In this embodiment, the temporal human flow feature vector With abnormal flags Concatenate to form the initial feature matrix of the nodes. It provides a temporal dimension basis, and then aggregates the feature matrix through a two-layer spatiotemporal graph convolutional network (ST-GCN). During the aggregation process, and From dynamic weighted undirected graph This generates the spatial topology, which is then used to generate the symmetric normalized adjacency matrix. To ensure that aggregation is unaffected by node differences, the 32-dimensional feature matrix output from the second layer is finally input into the fully connected layer to transform its dimensions. The Softmax function is then used to obtain probability vectors for three levels: L (low frequency), M (mid frequency), and H (high frequency). The level corresponding to the highest probability is taken as the original key level. Therefore, this embodiment utilizes the spatiotemporal fusion capability of ST-GCN to combine the temporal pedestrian flow characteristics and spatial correlation characteristics of sub-regions to achieve more accurate level prediction.

[0033] It should be noted that during incremental training, in each incremental training cycle (e.g., 24 hours), the temporal pedestrian flow features from the previous two cycles are included. , human flow anomaly sign and dynamic weighted undirected graph , and the training sample is input into the ST-GCN, a cross-entropy loss function (measuring the difference between the predicted probability and the label) is used for training, and the trainable parameters are updated through gradient descent After training, replace the old model to obtain a key level prediction model adapted to the latest scene, and continuously learn new scene features to avoid overfitting or prediction failure of the model due to changes in the scene (such as changes in human flow patterns, camera adjustments).

[0034] In a preferred embodiment, the original key level of each sub-region and the dynamic weighted undirected graph are corrected according to the key level correction rule to obtain the final key level step, which specifically includes: S51: Smooth the original key level of each sub-region in the current period based on historical periods, and correct the key level according to the first correction rule and the second correction rule using the smoothed original key level and the dynamic weighted undirected graph to obtain the final key level; The first correction rule is configured to: for a sub-region with an original key level of H, calculate the shortest path hop count of the sub-region in the dynamic weighted undirected graph with other H-level sub-regions, and if the shortest path hop count is greater than the target hop count, the sub-region is downgraded to M key level; The second correction rule is configured to: for a sub-region with an original key level of L for a continuous number of periods, determine whether the historical average human flow is less than the target low human flow judgment value, and if so, the sub-region is upgraded to M key level.

[0035] In this embodiment, first, the probability vector of the original key level is smoothed to balance real-time prediction and historical trends and reduce the influence of transient fluctuations (for example, using a smoothing formula , k is 0.65), then the key level is corrected by the first rule of isolated H downgrading and the second rule of continuous L upgrading, and finally the smoothing and correction results are integrated to obtain the final key level of each sub-region.

[0036] It should be noted that the first rule mainly considers that isolated H-level sub-regions (such as a remote corner misjudged as a high critical level) will lead to meaningless high-frequency analysis and waste computing power. In actual application, the target hop number is set to 3 hops, that is, there is usually no actual linkage between sub-regions exceeding 3 hops, and it is determined to be isolated; the second rule mainly considers that if the L-level region with long-term low passenger flow is continuously analyzed at a low frequency (such as 0.2fps), the missed detection (such as an abnormal event occasionally occurring) caused by the frame rate being down may be upgraded to M (1fps), which can improve the monitoring reliability with a small increase in computing power.

[0037] In a preferred embodiment, based on the final critical level of each sub-region in the current period, an analysis strategy of the camera video stream of the edge device for each sub-region in the next period is generated, and according to the analysis strategy, the video monitoring analysis step of the edge device in the next period is controlled, specifically comprising: S61: Based on the final critical level of each sub-region in the current period, according to a preset mapping relationship between the final critical level and the video analysis frequency, the video analysis frequency of the camera video stream of the edge device for each sub-region in the next period is determined; S62: According to the video analysis frequency and the video frame rate of each camera, the frame extraction frequency for the camera video stream is generated, and the edge device is controlled to perform frame extraction and video picture analysis in the next period.

[0038] In this embodiment, based on the mapping relationship between the critical level and the video analysis frequency (such as L: 0.2fps, M: 1fps, H: 4fps), the video analysis frequency of the next period is determined according to the final critical level of each sub-region, and then the frame extraction interval (such as one frame every 125 frames for L-level) is calculated according to the video analysis frequency and the original frame rate of the camera (such as 25fps). The controller of the edge device performs video decoding and analysis at a frequency to ensure that the computing power allocation matches the importance of the sub-region, while ensuring the accuracy of high-level regions, significantly reducing the data processing amount of low-level regions, and realizing overall load optimization.

[0039] Referring to Figure 2 , Figure 2 FIG. 1 is a structural schematic diagram of an embodiment of the video monitoring intelligent analysis system based on edge computing of the application.

[0040] As Figure 2 shown, the video monitoring intelligent analysis system based on edge computing of the embodiment of the application comprises: The rasterization module 10 is configured to discretize the region based on the geographic information of the monitoring region and the camera frustum parameters, and generate a set of sub-regions according to the camera coverage range. The generating module 20 is configured to generate a time-series flow feature vector of each camera and a flow anomaly flag of the current real-time flow by using the real-time flow and the historical flow features collected by each camera; The constructing module 30 is configured to construct a dynamic weighted undirected graph representing the topological relationship of the sub-regions in the monitoring region according to the monitoring node set of each camera, the edge set determined according to the spatial position of each camera in the monitoring region and the edge weight fused from the spatial distance and the flow similarity. The predicting module 40 is configured to input the feature matrix fused from the time-series flow feature vector and the flow anomaly flag into a sub-region key level probability prediction model based on the spatio-temporal graph convolution network and the dynamic weighted undirected graph, to predict the key level probability of each sub-region, and to determine the original key level of each sub-region based on the key level probability. The correcting module 50 is configured to perform key level correction of each sub-region according to a key level correction rule based on the original key level of each sub-region and the dynamic weighted undirected graph, to obtain a final key level. The analyzing module 60 is configured to generate an analysis strategy of the camera video stream of each sub-region in the next time period for the edge device based on the final key level of each sub-region in the current time period, and to control the edge device to perform video monitoring analysis in the next time period according to the analysis strategy.

[0041] Other embodiments or specific implementations of the video monitoring intelligent analysis system based on edge computing can refer to the above method embodiments, and details are not described herein.

[0042] It can be understood that, in the description of the specification, the description of the terms "an embodiment", "another embodiment", "other embodiments", or "first embodiment to Nth embodiment" means that the specific features, structures, materials or characteristics described in combination with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0043] It should be noted that, in this paper, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or system. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or system including the element.

[0044] The above merely describes the preferred embodiments of the present application, and is not intended to limit the patent scope of the present application, and any equivalent structure or equivalent process conversion, or direct or indirect application in other related technical fields, which are made by using the content of the present application specification and drawings, are also included in the patent protection scope of the present application.

Claims

1. An edge computing-based intelligent analysis method for video monitoring, characterized in that, The method comprises the following steps: Based on the monitoring area geographic information and the camera view cone parameters, a set of sub-regions is generated according to the camera coverage range by grid discretization of the region; Real-time and historical traffic flow features collected by each camera are used to generate a time-series traffic flow feature vector and a traffic flow anomaly flag of the current real-time traffic flow of each camera; According to the monitoring node set of each camera, the edge set determined by the spatial position of each camera in the monitoring area, and the edge weight of the fusion spatial distance and the traffic flow similarity, a dynamic weighted undirected graph representing the topological relationship of a plurality of sub-regions in the monitoring area is constructed; A feature matrix of the fusion time-series traffic flow feature vector and the traffic flow anomaly flag is input into a sub-region key level probability prediction model based on the spatio-temporal graph convolution network and the dynamic weighted undirected graph, and the key level probability of each sub-region is predicted, and based on the key level probability, the original key level of each sub-region is determined; According to the original key level of each sub-region and the dynamic weighted undirected graph, the key level correction of each sub-region is performed according to the key level correction rule, and the final key level is obtained; Based on the final key level of each sub-region in the current period, an analysis strategy of the camera video stream of each sub-region of the edge device in the next period is generated, and according to the analysis strategy, the edge device is controlled to perform video monitoring analysis in the next period. 2.The edge computing based video surveillance intelligent analysis method of claim 1, wherein, The method comprises the following steps: The planar geographic information of the monitoring area is acquired, the monitoring area is uniformly discretized into a cell grid with a preset size, and a grid set is formed , m represents the number of grids According to the projection range of the view cone of each camera on the ground, mark all the cell grids covered by the camera to form a sub-region , , denotes the i-th camera, denotes the sub-region corresponding to the i-th camera, and k denotes the k-th grid in the grid set. Collecting all the sub-regions corresponding to the cameras to form a sub-region set ; wherein any two of the sub-region set R and allow spatial overlap, n represents the number of sub-regions, and respectively represent the sub-region corresponding to the i-th camera and the sub-region corresponding to the j-th camera. 3.The edge computing based video surveillance intelligent analysis method of claim 1, wherein, Real-time and historical traffic flow features collected by each camera are used to generate a time-series traffic flow feature vector and a traffic flow anomaly flag of the current real-time traffic flow of each camera; For the video stream of the current period t, a YOLOv5 target detection algorithm is used to identify pedestrian targets, and then a DeepSORT multi-target tracking algorithm is used to remove repeated counting to obtain the current number of people flow ; represents the number of people flow of the i-th camera at the current period t Extract the historical flow number of the first two periods from the storage unit With , query the historical flow data of the same period, calculate the mean and standard deviation , and respectively represent the mean and standard deviation of the flow number of the ith camera in the current period t and the historical period t-1 and t-2; constructing a time-series passenger flow feature vector including current passenger flow number , historical passenger flow number , average value , and standard deviation , , , denotes the time-series passenger flow feature vector of the i-th camera at the current time period t; judging whether the current people flow number exceeds the historical average by 2 standard deviations if yes, setting the people flow abnormality flag to 1 if no, setting the people flow abnormality flag to 0 representing the people flow abnormality flag of the i-th camera in the time period t of the current year.​ 4.The edge computing based video surveillance intelligent analysis method of claim 1, wherein, According to the monitoring node set of each camera, the edge set determined by the spatial position of each camera in the monitoring area, and the edge weight of the fusion spatial distance and the traffic flow similarity, a dynamic weighted undirected graph representing the topological relationship of a plurality of sub-regions in the monitoring area is constructed; Constructing a monitoring node set , the Delaunay triangulation algorithm is used to divide the space of the monitoring nodes, if the Euclidean distance between any two nodes and is not more than a preset distance threshold, an edge is established between the two nodes, forming an edge set , n represents the number of monitoring nodes corresponding to the camera, i and j represent the i th and j th monitoring nodes respectively, represents the edge between the i th and j th monitoring nodes. Calculate each edge based on spatial distance and pedestrian flow similarity. weight Combined monitoring node set Edge set With border rights This forms a dynamic weighted undirected graph. , This represents the edge between the i-th and j-th monitoring nodes. The weight. 5.The edge computing based video surveillance intelligent analysis method of claim 4, wherein, each edge weight of expression of, in particular ； wherein, represents a monitoring node and a planar Euclidean distance of represents a distance reference value, represents a monitoring node and a spatial distance decay function of represents a monitoring node and a minimum maximum ratio of and is a weight coefficient, . 6.The edge computing based video surveillance intelligent analysis method of claim 1, wherein, A feature matrix of the fusion time-series traffic flow feature vector and the traffic flow anomaly flag is input into a sub-region key level probability prediction model based on the spatio-temporal graph convolution network and the dynamic weighted undirected graph, and the key level probability of each sub-region is predicted, and based on the key level probability, the original key level of each sub-region is determined; the time-series people flow feature vector and the people flow anomaly flag perform feature splicing as a feature matrix of the monitoring node ; by a two-layer spatio-temporal graph convolutional network, aggregating the feature matrix as a first layer spatio-temporal graph convolutional network, obtaining a feature matrix as a first layer spatio-temporal graph convolutional network, obtaining a feature matrix as a first layer spatio-temporal graph convolutional network, obtaining a feature matrix as a second layer spatio-temporal graph convolutional network, obtaining a 32-dimensional feature matrix ; The aggregation expression of each layer of the two-layer spatio-temporal graph convolution network is as follows: ； wherein, represents the first layer's feature matrix, represents the first layer's feature matrix, represents the adjacency matrix of the dynamic weighted undirected graph represents the identity matrix, represents the identity matrix, represents the degree matrix, represents the symmetric normalized adjacency matrix, represents the first layer's trainable weight matrix, represents the ReLU activation function; feature matrix The input is fully connected layer for dimensionality transformation, and the Sotfmax function is used to compute the dimensionality of each region. probability vector The expression is as follows: ； wherein respectively represent the probabilities corresponding to the L, M, H levels, L representing the low frequency level, M representing the medium frequency level, H representing the high frequency level. 7.The edge computing based video surveillance intelligent analysis method of claim 6, wherein, The method further comprises the following steps: In each incremental training cycle, the time-series passenger flow feature vectors generated by the past two incremental training cycles , passenger flow anomaly flag and dynamic weighted undirected graph of the monitoring area and the manually annotated key event samples are constructed as training samples; The training sample is input into a two-layer spatio-temporal graph convolution network, an incremental training is performed by using a cross-entropy loss function, and the trainable parameters of the spatio-temporal graph convolution network are updated , to obtain a sub-region key level probability prediction model updated over time. 8.The edge computing based video surveillance intelligent analysis method of claim 1, wherein, According to the original key level of each sub-region and the dynamic weighted undirected graph, the key level correction of each sub-region is performed according to the key level correction rule, and the final key level is obtained; The original key level of each sub-region in the current period is smoothed based on historical periods, and the smoothed original key level and the dynamic weighted undirected graph are used to perform key level correction according to the first correction rule and the second correction rule, and the final key level is obtained. The first correction rule is configured to: for a sub-region with an original key level of H, calculate the shortest path hop count of the sub-region in a dynamic weighted undirected graph with other H-level sub-regions, and if the shortest path hop count is greater than a target hop count, downgrade the sub-region to an M key level. The second correction rule is configured to: determine the historical average pedestrian flow for sub-regions with an original key level of L over a consecutive preset number of time periods. If the number of people is less than the target low-traffic threshold, upgrade the sub-area to M-level criticality. 9.The edge computing based video surveillance intelligent analysis method of claim 1, wherein, Generate an analysis strategy of the camera video stream of the edge device for each sub-region in the next period based on the final critical level of each sub-region in the current period, and control the edge device to perform the video monitoring analysis step in the next period according to the analysis strategy, specifically comprising: Determine the video analysis frequency of the camera video stream of the edge device for each sub-region in the next period according to a preset mapping relationship between the final critical level and the video analysis frequency based on the final critical level of each sub-region in the current period; According to the video analysis frequency and the video frame rate of each camera, generate the frame extraction frequency of the camera video stream, and control the edge device to perform frame extraction and video picture analysis in the next period.

10. An edge computing based intelligent video surveillance analysis system, characterized in that, Comprise: The gridding module is used for discretizing the region by gridding based on the monitoring region geographic information and the camera view cone parameters, and generating a sub-region set according to the camera coverage range; The generation module is used for generating a time series flow feature vector of each camera and a flow anomaly flag of the current real-time flow by using the real-time flow and the historical flow features collected by each camera; The construction module is used for constructing a dynamic weighted undirected graph representing the topological relationship of a plurality of sub-regions in the monitoring region according to the monitoring node set of each camera, the edge set determined by the spatial position of each camera in the monitoring region, and the edge weight fused with the spatial distance and the flow similarity; The prediction module is used for inputting a feature matrix fused with the time series flow feature vector and the flow anomaly flag into a sub-region critical level probability prediction model based on the spatio-temporal graph convolution network and the dynamic weighted undirected graph, predicting the critical level probability of each sub-region, and determining the original critical level of each sub-region based on the critical level probability; The correction module is used for performing critical level correction of each sub-region according to the critical level correction rule based on the original critical level of each sub-region and the dynamic weighted undirected graph, and obtaining the final critical level; The analysis module is used for generating an analysis strategy of the camera video stream of the edge device for each sub-region in the next period based on the final critical level of each sub-region in the current period, and controlling the edge device to perform the video monitoring analysis in the next period according to the analysis strategy.

Images