A big data-based intelligent park video monitoring management system

Abstract

The application relates to the field of image data processing and discloses a smart park video monitoring management system based on big data, which comprises the following modules: a first module, which processes fixed-time length video to obtain frame images and carries out gray processing on the frame images to obtain video frame images; a third module, which generates vertex maps based on the video frame images and area images; a fourth module, which generates a training set; a reconstruction error is taken as a loss value to train and update parameters of an abnormality evaluation model; a fifth module, which inputs all training samples in the training set into the abnormality evaluation model to obtain a reconstruction error; a sixth module, which judges whether abnormal behavior exists in a video to be predicted; if abnormal behavior exists, an alarm operation is carried out; feature missing caused by factors such as crowd shielding is avoided, and the accuracy of intelligently judging abnormal behavior by video monitoring is improved.

Description

A Smart Park Video Surveillance Management System Based on Big Data Technical Field

[0001] This invention relates to the field of image data processing, and more specifically, to a smart park video surveillance management system based on big data. Background Technology

[0002] Video anomaly detection falls under the category of intelligent video surveillance. Most methods generally involve two parts: during the training phase, unsupervised learning is used to learn the appearance and motion features of the training data to build one or more models that are associated with normal data; then, the video is judged to be abnormal based on whether the video data matches the model. Motion features are generated by tracking the trajectories of moving targets in the video. However, crowd occlusion can lead to a loss of motion features, resulting in excessively large errors in the model. Summary of the Invention

[0003] This invention provides a smart park video surveillance management system based on big data, which solves the technical problem of excessive model error caused by feature loss due to crowd occlusion in related technologies.

[0004] This invention provides a smart park video surveillance management system based on big data, comprising: a first module, which processes a fixed-duration video to obtain frame images, and performs grayscale processing on the frame images to obtain video frame images;

[0005] The second module performs image segmentation on the video frame images to obtain region images;

[0006] The third module generates a vertex graph based on video frame images and region images. The vertex graph includes vertices and edges connecting the vertices. The vertices include a first vertex and a second vertex. The first vertex corresponds to the video frame image, and the second vertex corresponds to the region image after the video frame image is segmented. There is an edge between the first vertex of the video frame image in frame t and the first vertex of the video frame image in frame t+1. The second vertices of the region images in the same video frame image are fully connected by edges. There is an edge between the second vertices of the overlapping region images in the video frame images in frames t and t+1. The region images and video frame images are characterized to obtain the vertex vectors of the corresponding vertices.

[0007] The fourth module generates a training set. The videos in the training set do not contain any abnormal behavior. The vertex graphs and vertex vectors obtained from processing the videos in the training set are input into the anomaly evaluation model to train the anomaly evaluation model. The anomaly evaluation model includes a GNN layer, a logical layer, a first hidden layer, a deconvolution layer, and a reconstruction error calculation layer. The GNN layer updates the vertex vectors corresponding to the vertices and outputs the second vector of the vertex to the logical layer. The logical layer performs matrix processing on the second vector of the first vertex to obtain vertex features. The vertex features of the first vertex are sorted according to the time of the corresponding video frame image to generate a vertex feature sequence, which is then input into the first hidden layer and outputs the vertex output feature sequence. The deconvolution layer deconvolves the sequence units of the vertex output feature sequence to the same size as the video frame image to obtain the reconstructed image sequence. The reconstruction error calculation layer calculates the reconstruction error between the reconstructed image sequence and the video frame image.

[0008] The reconstruction error is used as the loss value to train and update the anomaly evaluation model. Training stops when the maximum number of updates is reached, or when the change in reconstruction error compared to the previous training is less than 0.1.

[0009] The fifth module inputs all training samples in the training set into the anomaly evaluation model to obtain the reconstruction error, and calculates the maximum value of the reconstruction error of all training samples as the error threshold.

[0010] The sixth module inputs the vertex graph and vertex vectors obtained from the video to be predicted into the anomaly evaluation model and outputs the reconstruction error of the video to be predicted. If the reconstruction error exceeds the error threshold, it is determined that there is abnormal behavior in the video to be predicted; otherwise, it is determined that there is no abnormal behavior in the video to be predicted.

[0011] In the seventh module, if the reconstruction error does not exceed the error threshold, it is determined that there is no abnormal behavior in the video and no processing is performed. If the error exceeds the error threshold, an alarm is triggered, alerting the person in charge to pay attention to the video in the alarm area and requiring manual investigation and handling of the alarm area.

[0012] Furthermore, image segmentation can employ image segmentation models such as SegNet, DeepLab, Mask R-CNN, U-Net, and Gated SCNN.

[0013] Furthermore, the feature processing method is convolution processing. When performing convolution processing on a region image, edge padding can be applied to obtain image features in matrix form.

[0014] Furthermore, the calculation formula for the reconstruction error calculation layer is as follows: Where D x,y Indicates y k With x k The distance between them, y kLet x represent the k-th sequence unit of the reconstructed image sequence. k Let N represent the k-th video frame image, and N*M represent the size of the video frame image.

[0015] Furthermore, D x,y The distance is either Euclidean or cosine, with one pixel value as a dimension.

[0016] Furthermore, the calculation formula for the GNN layer is as follows:

[0017]

[0018] This represents the second vector of the vertex of the i-th node. and Let represent the vertex vectors of the i-th and j-th nodes, respectively, and W represent the weight parameters of the GNN layer. N represents the weight vector of the GNN layer. i This represents the set of nodes that are directly connected to the i-th node.

[0019] Furthermore, the logical layer performs matrix processing on the second vector of the first vertex to obtain vertex features by: extracting S equal-length vectors from the second vector of the vertex and using them as S row vectors of the vertex features.

[0020] Furthermore, the first hidden layer consists of a CNN layer, an RNN layer, and a tensor layer. The CNN layer takes a vertex feature as input and outputs a first intermediate feature. The row vectors of the first intermediate feature are concatenated to generate a second intermediate feature. The RNN layer takes the t-th sequence unit of the vertex feature sequence as input at the t-th time step and generates the second intermediate feature, and outputs the corresponding third intermediate feature. The third intermediate feature is input into the tensor layer and then matrixed to obtain the vertex output feature.

[0021] Furthermore, the deconvolution layer is calculated by padding the input feature map and performing a convolution operation using weights.

[0022] The beneficial effects of this invention are: it avoids feature loss caused by factors such as crowd obstruction, and improves the accuracy of intelligent identification of abnormal behavior in video surveillance. Attached Figure Description

[0023] Figure 1 is a schematic diagram of the modules of a smart park video surveillance management system based on big data according to the present invention;

[0024] Figure 2 is a schematic diagram of the modules of the anomaly assessment model of the present invention;

[0025] Figure 3 is a flowchart of the processing of the first hidden layer of the present invention.

[0026] In the diagram: Module 101, Module 202, Module 303, Module 404, Module 505, Module 606, Module 707. Detailed Implementation

[0027] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0028] As shown in Figures 1-3, a smart park video surveillance management system based on big data includes:

[0029] The first module 101 processes a fixed-duration video to obtain frame images, and performs grayscale processing on the frame images to obtain video frame images.

[0030] The second module 102 performs image segmentation on the video frame images to obtain region images;

[0031] Image segmentation can employ image segmentation models such as SegNet, DeepLab, Mask R-CNN, U-Net, and Gated SCNN.

[0032] The third module 103 generates a vertex graph based on video frame images and region images. The vertex graph includes vertices and edges connecting the vertices. The vertices include a first vertex and a second vertex. The first vertex corresponds to the video frame image, and the second vertex corresponds to the region image after the video frame image is segmented. There is an edge between the first vertex of the video frame image in frame t and the first vertex of the video frame image in frame t+1. The second vertices of the region images in the same video frame image are fully connected by edges. There is an edge between the second vertices of the overlapping region images in the video frame images in frames t and t+1.

[0033] Feature processing is performed on the region image and video frame image to obtain the vertex vectors of the corresponding vertices;

[0034] The feature processing method is convolution. When performing convolution processing on a region image, edge padding can be applied to obtain image features in matrix form.

[0035] Module 4 (104) generates a training set. The videos in the training set do not contain any abnormal behavior. The vertex graphs and vertex vectors obtained from processing the videos in the training set are input into the anomaly evaluation model to train the model. The anomaly evaluation model includes a GNN layer, a logical layer, a first hidden layer, a deconvolution layer, and a reconstruction error calculation layer. The GNN layer updates the vertex vectors corresponding to vertices and outputs the second vertex vector to the logical layer. The logical layer performs matrix processing on the second vertex vector of the first vertex to obtain vertex features. The vertex features of the first vertex are sorted according to the time of its corresponding video frame image to generate a vertex feature sequence, which is then input into the first hidden layer, outputting a vertex output feature sequence. The deconvolution layer deconvolves the sequence units of the vertex output feature sequence to the same size as the video frame image, obtaining a reconstructed image sequence. The reconstruction error calculation layer calculates the reconstruction error between the reconstructed image sequence and the video frame image.

[0036] The calculation formula for the reconstruction error calculation layer is as follows: Where D x,y Represent y and x k The distance between them, y k Let x represent the k-th sequence unit of the reconstructed image sequence. k Let N represent the k-th video frame image, and N*M represent the size of the video frame image.

[0037] D x,y The distance is either Euclidean or cosine, with one pixel value as a dimension.

[0038] The reconstruction error is used as the loss value to train and update the anomaly evaluation model. Training stops when the maximum number of updates is reached, or when the change in reconstruction error compared to the previous training is less than 0.1.

[0039] The formula for calculating the GNN layer is as follows:

[0040]

[0041] This represents the second vector of the vertex of the i-th node. and Let represent the vertex vectors of the i-th and j-th nodes, respectively, and W represent the weight parameters of the GNN layer. N represents the weight vector of the GNN layer. i This represents the set of nodes that are directly connected to the i-th node.

[0042] In one embodiment of the present invention, the method by which the logic layer performs matrix processing on the second vector of the first vertex to obtain vertex features is: to extract S equal-length vectors from the second vector of the vertex and use them as S row vectors of the vertex features.

[0043] In one embodiment of the present invention, the first hidden layer comprises a CNN layer, an RNN layer, and a tensor layer. The CNN layer takes a vertex feature as input and outputs a first intermediate feature. The row vectors of the first intermediate feature are concatenated sequentially to generate a second intermediate feature. The RNN layer takes the t-th sequence unit of the vertex feature sequence as input at the t-th time step to generate the second intermediate feature and outputs the corresponding third intermediate feature. The third intermediate feature is input into the tensor layer and then matrixed to obtain the vertex output feature.

[0044] In one embodiment of the present invention, the calculation process of the deconvolution layer involves filling the input feature map and performing a convolution operation using weights.

[0045] The fifth module 105 inputs all training samples in the training set into the anomaly evaluation model to obtain the reconstruction error, and calculates the maximum value of the reconstruction error of all training samples as the error threshold.

[0046] The sixth module 106 inputs the vertex graph and vertex vector obtained from the video to be predicted into the anomaly evaluation model and outputs the reconstruction error of the video to be predicted. If the reconstruction error exceeds the error threshold, it is determined that there is abnormal behavior in the video to be predicted; otherwise, it is determined that there is no abnormal behavior in the video to be predicted.

[0047] In Module 7, Section 107, if the reconstruction error does not exceed the error threshold, it is determined that there is no abnormal behavior in the video, and no processing is performed. If the error exceeds the threshold, an alarm is triggered, alerting the person in charge to pay attention to the video in the alarm area, and manual investigation and handling of the alarm area are required.

[0048] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A smart park video surveillance management system based on big data, characterized in that, include: The first module processes a fixed-length video to obtain frame images, and then performs grayscale processing on the frame images to obtain video frame images. The second module performs image segmentation on the video frame images to obtain region images; The third module generates a vertex graph based on video frame images and region images. The vertex graph includes vertices and edges connecting the vertices. Vertices include a first vertex and a second vertex. The first vertex corresponds to a video frame image, and the second vertex corresponds to a region image after segmentation of the video frame image. There is an edge between the first vertex of the video frame image in frame t and the first vertex of the video frame image in frame t+1. The second vertices of the region images in the same video frame image are fully connected by edges. There is an edge between the second vertices of the overlapping region images in the video frame images in frames t and t+1. Feature processing is performed on the region images and video frame images to obtain the vertex vectors of the corresponding vertices. The fourth module generates a training set. The videos in the training set do not contain abnormal behavior. The vertex graph and vertex vectors obtained from the video processing in the training set are input into the abnormal behavior set. The evaluation model trains an anomaly evaluation model, which includes a GNN layer, a logical layer, a first hidden layer, a deconvolution layer, and a reconstruction error calculation layer. The GNN layer updates the vertex vectors corresponding to vertices and outputs the second vertex vector to the logical layer. The logical layer performs matrix processing on the second vertex vector of the first vertex to obtain vertex features. These features are then sorted according to the time of their corresponding video frame images to generate a vertex feature sequence, which is then input into the first hidden layer, outputting a vertex output feature sequence. The deconvolution layer deconvolves the sequence units of the vertex output feature sequence to a size consistent with the video frame image, obtaining a reconstructed image sequence. The reconstruction error calculation layer calculates the reconstruction error between the reconstructed image sequence and the video frame image; this reconstruction error is used as the loss value to train and update the parameters of the anomaly evaluation model. The fifth module inputs all training samples in the training set into the anomaly evaluation model to obtain the reconstruction error, and calculates the maximum value of the reconstruction error of all training samples as the error threshold. The sixth module inputs the vertex graph and vertex vectors obtained from the video to be predicted into the anomaly evaluation model and outputs the reconstruction error of the video to be predicted. If the reconstruction error exceeds the error threshold, it is determined that there is abnormal behavior in the video to be predicted; otherwise, it is determined that there is no abnormal behavior in the video to be predicted. The seventh module does not process the video if there is no abnormal behavior. If there is abnormal behavior in the video, an alarm is triggered to remind the person in charge to pay attention to the video in the alarm area and to manually investigate and handle the alarm area.

2. The smart park video surveillance management system based on big data according to claim 1, characterized in that, Image segmentation was performed using SegNet, DeepLab, Mask R-CNN, U-Net, and Gated SCNN.

3. The smart park video surveillance management system based on big data according to claim 1, characterized in that, The feature processing method is convolution processing. When performing convolution processing on the region image, edge padding is applied to obtain image features in matrix form.

4. The smart park video surveillance management system based on big data according to claim 1, characterized in that, The calculation formula for the reconstruction error calculation layer is as follows: Where D x,y Indicates y k With x k The distance between them, y k Let x represent the k-th sequence unit of the reconstructed image sequence. k Let N represent the k-th video frame image, and N*M represent the size of the video frame image.

5. The smart park video surveillance management system based on big data according to claim 4, characterized in that, D x,y The distance is either Euclidean or cosine, with one pixel value as a dimension.

6. The smart park video surveillance management system based on big data according to claim 1, characterized in that, The formula for calculating the GNN layer is as follows: This represents the second vector of the vertex of the i-th node. and Let represent the vertex vectors of the i-th and j-th nodes, respectively, and W represent the weight parameters of the GNN layer. N represents the weight vector of the GNN layer. i This represents the set of nodes that are directly connected to the i-th node.

7. The smart park video surveillance management system based on big data according to claim 1, characterized in that, The logical layer performs matrix processing on the second vector of the first vertex to obtain vertex features by: extracting S equal-length vectors from the second vector of the vertex and using them as S row vectors of the vertex features.

8. The smart park video surveillance management system based on big data according to claim 1, characterized in that, The first hidden layer consists of a CNN layer, an RNN layer, and a tensor layer. The CNN layer takes a vertex feature as input and outputs a first intermediate feature. The row vectors of the first intermediate feature are concatenated to generate a second intermediate feature. The RNN layer takes the t-th sequence unit of the vertex feature sequence as input at time step t and generates the second intermediate feature, and outputs the corresponding third intermediate feature. The third intermediate feature is input into the tensor layer and then matrixed to obtain the vertex output feature.

9. A smart park video surveillance management system based on big data as described in claim 1, characterized in that, The deconvolutional layer is calculated by filling the input feature map and then performing a convolution operation using weights.

Images