A crowd abnormal event detection method, computer program product, device and medium

Abstract

The application discloses a kind of crowd abnormal event detection method, computer program product, equipment and medium, it is related to computer technical field, including: based on the detection information of individual target of each frame monitoring picture, the actual crowd relationship feature map of corresponding frame is constructed, the actual crowd relationship feature map sequence of each frame is determined based on;The crowd relationship feature map sequence is input into preset graph prediction network, and the predicted crowd relationship feature map of future frame is determined by preset graph prediction network;Determine the peak signal-to-noise ratio between the predicted crowd relationship feature map of future frame and the actual crowd relationship feature map of future frame, and generate target abnormal score according to peak signal-to-noise ratio;According to target abnormal score, it is judged whether crowd abnormal event occurs, to be early warning processing in time.The application improves the accuracy of crowd abnormal event detection based on crowd relationship.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a group abnormal event detection method, computer program product, device and medium. BACKGROUND

[0002] Video abnormal event detection task can be roughly divided into group abnormal event detection and individual abnormal event detection. Most of the existing group abnormal event detection methods adopt the appearance information and motion information of video frames for feature extraction modeling, and then determine whether the abnormal event occurs by calculating the abnormal value score. However, most of the existing abnormal detection methods have the problem of inaccurate detection, which is urgent to be solved by the technical personnel in the field. SUMMARY

[0003] Therefore, the purpose of the present application is to provide a group abnormal event detection method, computer program product, device and medium, which improves the accuracy of group abnormal event detection. The specific scheme is as follows:

[0004] In the first aspect, the present application discloses a group abnormal event detection method, comprising:

[0005] Based on the detection information of individual targets of each frame of monitoring picture, the actual group relationship feature map of the corresponding frame is constructed, and the group relationship feature map sequence is determined based on the actual group relationship feature map of each frame;

[0006] The group relationship feature map sequence is input into the preset graph prediction network, and the prediction group relationship feature map of the future frame is determined through the preset graph prediction network;

[0007] The peak signal-to-noise ratio between the prediction group relationship feature map of the future frame and the actual group relationship feature map of the future frame is determined, and the target abnormal score is generated according to the peak signal-to-noise ratio;

[0008] According to the target abnormal score, it is judged whether a group abnormal event occurs, so as to carry out early warning processing when a group abnormal event occurs.

[0009] Optionally, based on the detection information of individual targets of each frame of monitoring picture, the actual group relationship feature map of the corresponding frame is constructed, comprising:

[0010] Identify individual targets in each frame of monitoring picture, and obtain the detection information of individual targets;

[0011] The individual target is taken as a node in the actual group relationship feature map, and the detection information of the individual target is taken as the node attribute of the node, so as to obtain a node set;

[0012] Based on the interaction intensity between any two individual targets, an edge set is obtained;

[0013] Construct an actual group relationship feature graph based on the node set and the edge set.

[0014] Optionally, the detection information includes confidence, position information, appearance feature and motion feature of the individual target.

[0015] Optionally, the individual target in each frame of the monitoring picture is identified, and the detection information of the individual target is obtained, including:

[0016] The monitoring picture is obtained from the monitoring video.

[0017] The monitoring picture is subjected to multi-scale scaling processing to generate input images of multiple scales, and the input images of multiple scales are input in parallel to the multi-scale target detection model.

[0018] The individual target in the monitoring picture is identified by the multi-scale target detection model, and the confidence and the position information of the individual target are obtained.

[0019] Optionally, the individual target in each frame of the monitoring picture is identified, and the detection information of the individual target is obtained, including:

[0020] Based on the position information of the individual target, a corresponding region image is extracted from the monitoring picture.

[0021] The region image is preprocessed, and the preprocessed region image is input into a pre-trained convolutional neural network model, so as to extract the appearance feature of the individual target by the convolutional neural network model; wherein the appearance feature represents the visual appearance feature of the individual target.

[0022] Optionally, the individual target in each frame of the monitoring picture is identified, and the detection information of the individual target is obtained, including:

[0023] Based on the position information of the individual target, a corresponding region image is extracted from the monitoring picture.

[0024] The optical flow information at the region image is extracted by an optical flow estimation network, and the motion feature of the individual target is determined according to the optical flow information at the region image.

[0025] Optionally, the group abnormal event detection method further includes:

[0026] Based on the position information of any two individual targets, the spatial distance between any two individual targets is determined, and the edge weight attribute between any two individual targets is calculated according to the spatial distance; wherein the edge weight attribute represents the interaction intensity between any two individual targets.

[0027] Optionally, the edge set is obtained based on the interaction intensity between any two individual targets, including:

[0028] An edge weight attribute between any two individual targets is taken as a weight value between the any two individual targets, and a target matrix is constructed according to the weight value between the any two individual targets;

[0029] The edge set is determined based on the target matrix.

[0030] Optionally, whether a group abnormal event occurs is determined according to the target abnormal score, including:

[0031] Whether the target abnormal score exceeds a preset score threshold is determined.

[0032] If the target abnormal score exceeds the preset score threshold, it is determined that the group abnormal event occurs.

[0033] Optionally, the preset graph prediction network includes a time domain convolution block and a space domain convolution block; wherein the time domain convolution block processes time domain dimension features of the group relationship feature map sequence, and the space domain convolution block processes node topology features of a single frame of actual group relationship feature map in the group relationship feature map sequence.

[0034] Optionally, a peak signal-to-noise ratio between the predicted group relationship feature map of the future frame and the actual group relationship feature map of the future frame is determined, and a target abnormal score is generated according to the peak signal-to-noise ratio, including:

[0035] A target peak signal-to-noise ratio between the predicted group relationship feature map of the future frame and the actual group relationship feature map of the future frame is determined.

[0036] The target abnormal score is generated according to the target peak signal-to-noise ratio, a historical peak signal-to-noise ratio minimum value and a historical peak signal-to-noise ratio maximum value.

[0037] Optionally, the target abnormal score is generated according to the target peak signal-to-noise ratio, the historical peak signal-to-noise ratio minimum value and the historical peak signal-to-noise ratio maximum value, including:

[0038] A first difference value of the target peak signal-to-noise ratio and the historical peak signal-to-noise ratio minimum value is calculated.

[0039] A second difference value of the historical peak signal-to-noise ratio maximum value and the historical peak signal-to-noise ratio minimum value is calculated.

[0040] The ratio of the first difference value and the second difference value is normalized to obtain the target abnormal score.

[0041] In a second aspect, the present application discloses a computer program product, and the computer program is executed by a processor to realize the steps of the group abnormal event detection method disclosed above.

[0042] In a third aspect, the present application discloses an electronic device, including:

[0043] A memory for saving a computer program;

[0044] a processor configured to execute a computer program to implement the group abnormal event detection method disclosed above.

[0045] In a fourth aspect, the present application discloses a computer readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the group abnormal event detection method disclosed above.

[0046] It can be seen that the present application proposes a group abnormal event detection method, which comprises: constructing an actual group relationship feature map corresponding to each frame based on the detection information of individual targets in the frame of monitoring pictures, and determining a group relationship feature map sequence based on the actual group relationship feature maps of each frame; inputting the group relationship feature map sequence into a preset graph prediction network, and determining a predicted group relationship feature map of a future frame through the preset graph prediction network; determining the peak signal-to-noise ratio between the predicted group relationship feature map of the future frame and the actual group relationship feature map of the future frame, and generating a target abnormal score according to the peak signal-to-noise ratio; and judging whether a group abnormal event occurs according to the target abnormal score, so as to perform early warning processing when the group abnormal event occurs.

[0047] Beneficial effects: the present application constructs an actual group relationship feature map corresponding to each frame based on the detection information of individual targets in the frame of monitoring pictures, and determines a group relationship feature map sequence based on the actual group relationship feature maps of each frame, so that the group relationship can be quantitatively represented in the form of a graph structure. Compared with the traditional method of detecting abnormal events based only on apparent information and motion information, the present application more comprehensively captures the key feature of group relationship, provides more abundant feature input for abnormal detection, and thus improves the accuracy of abnormal detection. Further, the present application inputs the group relationship feature map sequence into a preset graph prediction network, so as to determine a predicted group relationship feature map of a future frame through the preset graph prediction network, realize modeling and prediction of group behavior, and generate a target abnormal score by calculating the peak signal-to-noise ratio between the predicted group relationship feature map of the future frame and the actual group relationship feature map of the future frame. In this way, the difference degree between the predicted group relationship feature map and the actual group relationship feature map can be quantified, so as to accurately judge whether a group abnormal event occurs, avoid the problem of inaccurate detection caused by insufficient consideration of group relationship features in the traditional method, and realize effective early warning processing of group abnormal events. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only belong to the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on the provided drawings.

[0049] Figure 1 A flow chart of a population abnormal event detection method disclosed in the present application;

[0050] Figure 2 A flow chart of a construction of an actual population relationship feature map disclosed in the present application;

[0051] Figure 3 A schematic diagram of a graph prediction encoding and decoding flow disclosed in the present application;

[0052] Figure 4 A schematic diagram of a space-time convolution module disclosed in the present application;

[0053] Figure 5 A flow chart of a specific population abnormal event detection method disclosed in the present application;

[0054] Figure 6 A schematic diagram of a population abnormal event detection device structure disclosed in the present application;

[0055] Figure 7 A schematic diagram of an electronic device structure disclosed in the present application. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0057] Video abnormal event detection tasks can be roughly divided into population abnormal event detection and individual abnormal event detection. Most of the existing population abnormal event detection methods use the appearance information and motion information of video frames for feature extraction modeling, and then determine whether an abnormal event occurs by calculating the abnormal value score. However, most of the existing abnormal detection methods have the problem of inaccurate detection, which needs to be solved by those skilled in the art.

[0058] Therefore, the embodiments of the present application propose a population abnormal event detection scheme, which can improve the accuracy of population abnormal event detection.

[0059] The embodiments of the present application disclose a population abnormal event detection method, referring to Figure 1 The method comprises the following steps:

[0060] Step S11: Based on the detection information of individual targets of each frame of monitoring picture, an actual population relationship feature map corresponding to the frame is constructed, and a population relationship feature map sequence is determined based on the actual population relationship feature maps of each frame.

[0061] In this embodiment, individual targets in each frame of the monitoring footage are identified, and their detection information is obtained. These individual targets are treated as nodes in the actual group relationship feature graph, and their detection information is used as node attributes to obtain a node set. An edge set is obtained based on the interaction strength between any two individual targets. The actual group relationship feature graph is constructed based on the node set and the edge set. The detection information includes the confidence level, location information, appearance features, and motion features of the individual targets.

[0062] The following is a more detailed explanation; see [link / reference]. Figure 2 First, we analyze how to obtain detection information for individual targets.

[0063] In the first specific implementation, surveillance footage is acquired from the video feed and subjected to multi-scale scaling to generate input images at multiple scales. These input images are then fed in parallel into a multi-scale object detection model (You Only Look Once). Further, the multi-scale object detection model identifies individual targets within the surveillance footage and obtains their confidence level and location information (confidence level and location information are collectively referred to as...). Figure 2 The detection results in the image are shown. Specifically, the YOLO detection model focuses on the global information of the entire image during the training and inference phases. Compared to object detection methods based on region proposals (such as Fast Region Convolutional Neural Networks, which only focus on local image information within candidate boxes during detection), the YOLO detection model can effectively reduce the false detection rate of the background. Assuming a single frame image of the surveillance footage is... The scaled image is input into the YOLO detection model. Assume that after passing through the YOLO detection model, the result is... Individual goals At the same time, Confidence level of individual goals and location information The confidence score of an individual target is used to assess its reliability. Assuming a confidence threshold of 0.5, a confidence score of 0.3 indicates low reliability of the YOLO detection model in identifying that target, suggesting a possible false detection that needs to be removed. Conversely, a confidence score of 0.9 indicates high reliability and will be included in subsequent processing. Furthermore, the location information of the individual target includes its center point coordinates, height, and width.

[0064] In the second specific embodiment, based on the position information of the individual target, a corresponding region image is extracted from the monitoring picture, and the region image is preprocessed, and then the preprocessed region image is input into a pre-trained convolutional neural network model, so as to extract the apparent feature of the individual target through the convolutional neural network model , represent the apparent feature of the first individual target. The apparent feature represents the visual appearance feature of the individual target. The apparent feature includes but is not limited to color, texture, etc. The preprocessing includes image size processing, image enhancement processing, etc. Compared with the traditional feature extraction method relying on artificial design feature extraction rules, the convolutional neural network model can automatically learn the most effective apparent feature extraction method from data through layer-by-layer training of multiple neurons, greatly reducing the cost of artificial intervention.

[0065] In the third specific embodiment, based on the position information of the individual target, a corresponding region image is extracted from the monitoring picture, and the optical flow information at the region image is extracted through an optical flow estimation network, and then the motion feature of the individual target is determined according to the optical flow information at the region image , represent the motion feature of the first individual target. In the monitoring scene, it is assumed that the same object has a small brightness difference in adjacent frames. Therefore, the motion feature of the individual target is extracted through the optical flow method by using the time dimension information of the monitoring video sequence. This method does not require prior scene information and is suitable for group motion analysis in complex monitoring environments.

[0066] The detection information of the individual target is taken as the node attribute of the node , represent the apparent feature, the motion feature, the confidence and the position information of the first individual target, respectively. The matrix is the node attribute of all nodes, is the number of nodes, is the dimension of the feature vector of the node, and the node set is represented by . The following analyzes how to obtain the edge set based on the interaction strength between any two individual targets.

[0067] Based on the position information of any two individual targets, the spatial distance between any two individual targets is determined, and the edge weight attribute between any two individual targets is calculated according to the spatial distance; wherein the edge weight attribute represents the interaction strength between any two individual targets. Further, the edge weight attribute between any two individual targets is taken as the weight value between any two individual targets, and a target matrix is constructed according to the weight value between any two individual targets, and then the edge set is determined based on the target matrix.

[0068] For example, the weight values ​​are calculated as follows:

[0069] ;

[0070] The target matrix is ​​calculated as follows:

[0071] ;

[0072] For the first Individual goals and the first The boundary weight attributes between individual goals For the first Individual goals and the first Spatial distance between individual targets and Adjust the distribution and sparsity of the target matrix.

[0073] Finally, a feature graph of actual group relationships is constructed based on the set of nodes and the set of edges. .

[0074] It is evident that group relationship feature graphs possess powerful data understanding and cognitive capabilities. In these graphs, nodes are interconnected by edges, effectively representing the relationships between different individual targets. By linking individual targets within a monitoring feed using group relationship feature graphs, effective relationship reasoning is facilitated, thereby enhancing the understanding of group anomalies and ultimately improving the accuracy of anomaly detection.

[0075] Step S12: Input the group relationship feature map sequence into the preset map prediction network, and determine the predicted group relationship feature map of future frames through the preset map prediction network.

[0076] In this embodiment, the preset graph prediction network includes temporal convolutional blocks and spatial convolutional blocks. The temporal convolutional blocks process the temporal dimension features of the group relationship feature map sequence, while the spatial convolutional blocks process the node topology features of the actual group relationship feature map in a single frame of the group relationship feature map sequence. Specifically, the preset graph prediction network is constructed around the collaborative processing of temporal and spatial features, and its operating logic is similar to... Figure 3 In line with this, the network is based on an autoencoder framework, taking as input a sequence of group relationship feature maps containing M time steps and a target matrix. The whole can be abstracted as , Indicates the first The input feature dimensions of each individual target Representing the real number domain, the encoder integrates temporal and spatial convolutional blocks, with "spatiotemporal convolutional blocks" as the basic unit (each spatiotemporal convolutional block consists of two temporal convolutional blocks and one spatial convolutional block). The temporal convolutional blocks focus on the temporal dimension features of the group relationship feature map sequence, leveraging causal convolution to follow temporal causality and utilizing historical time step features. Relationship characteristics According to the chain rule of probability Predict the relationship features at the current time step Spatial convolutional blocks extract node topological features within the graph structure of a single time step (not between time steps), ensuring that spatial dimension analysis focuses on the current frame. After encoding by the spatiotemporal convolutional network, the hidden layer features are output. , This represents the output feature dimension. After obtaining the hidden layer features, it is necessary to further adapt them to the decoder input requirements through latent vector transformation (corresponding to 4 in the figure). This involves fine-tuning the feature dimensions or distribution. The adjusted... The input flows into the decoder, utilizing the "internal product decoding" mechanism: first, the... Rather than transpose Perform the inner product operation, and then pass the activation function. Map the result of the inner product operation to the [0,1] interval to generate a predicted group relationship feature map. (correspond Figure 3 In This allows for the deduction of future group relationships from historical spatiotemporal characteristics. To constrain the accuracy of encoding and decoding, a loss function is introduced. Using real group relationship feature maps To supervise and continuously optimize the hidden layer features and decoding logic, ensuring the prediction graph... The evolutionary trend that approximates actual group relations. and These represent the hidden layer feature vectors of the corresponding nodes after the encoder encodes the input sequence.

[0077] See Figure 4 As shown, Figure 4 This is a schematic diagram of the spatiotemporal convolution module, used as a preset graph prediction network (corresponding to...). Figure 3 The core component of the architecture, its left half of the process connection Figure 3 The coding logic, Figure 4 The input of the l-th layer features a l for Figure 3 The intermediate features of the encoding process. First, a l The input is fed into a temporally gated convolution, which extracts the dynamics across time. The result of the temporally gated convolution is then compared with the target matrix. The input is fed into a spatial graph convolution, which extracts the spatial topology of the current time step. Then, it is fused by a temporally gated convolution, outputting the l+1 layer feature a. l+1 . Figure 4 The right half is the detailed structure of the temporally gated convolution, with the input temporal feature sequence (a l t-（M-1） , ...a l t The algorithm undergoes two parallel causal convolutions. One causal convolution output is used to generate gating weights via an activation function, while the other causal convolution output is fused with information introduced by residual connections. Adaptive feature selection is then achieved through element-wise multiplication based on the gating weights. This module, through the synergy of temporally gated convolutions and spatial graph convolutions, provides core spatiotemporal feature extraction capabilities for the pre-defined graph prediction network. Its output 'a'... l+1 After multiple iterations, it was gradually refined into Figure 3 The hidden layer feature Z is used to process the temporal dimension features and single-frame node topological features of the group relationship feature map sequence.

[0078] Step S13: Determine the peak signal-to-noise ratio (PSNR) between the predicted group relationship feature map of the future frame and the actual group relationship feature map of the future frame, and generate a target anomaly score based on the PSNR.

[0079] In this embodiment, a target peak signal-to-noise ratio (PSNR) is determined between the predicted group relationship feature map of the future frame and the actual group relationship feature map of the future frame. A target anomaly score is generated based on the target PSNR, the historical minimum PSNR, and the historical maximum PSNR. Specifically, a first difference between the target PSNR and the historical minimum PSNR is calculated; a second difference between the historical maximum PSNR and the historical minimum PSNR is calculated; and the ratio of the first difference to the second difference is normalized to obtain the target anomaly score. The calculation process described above is as follows:

[0080] ;

[0081] ;

[0082] in, Indicates the target peak signal-to-noise ratio. This represents the historical peak signal-to-noise ratio minimum. This represents the highest historical peak signal-to-noise ratio. This indicates the target's abnormal score.

[0083] The higher the peak signal-to-noise ratio, the better the predicted group relationship feature map matches the real feature map, the better the prediction effect, and the greater the probability that the corresponding scenario is a normal event.

[0084] Step S14: judging whether a crowd abnormal event occurs according to the target abnormal score, so as to perform early warning processing when the crowd abnormal event occurs.

[0085] In this embodiment, whether a crowd abnormal event occurs is judged according to the target abnormal score, including: judging whether the target abnormal score exceeds a preset score threshold, and if yes, determining that the crowd abnormal event occurs.

[0086] Referring to FIG. 1, Figure 5 As shown in the figure, the figure embodies the complete flow of the crowd abnormality detection scheme, including two stages of offline learning and abnormality detection: (1) Offline learning: taking a normal scene training sample set as input, first converting video frames into a graph structure containing node features (i.e. the aforementioned crowd relationship feature graph) through graph relationship construction; then inputting a network model (i.e. the aforementioned preset graph prediction network) to minimize the loss function, so that the network model learns the graph pattern of normal crowd relationship, and after training, the network parameters are fixed, and a model that can accurately predict normal crowd relationship is output. (2) Abnormality detection: input test samples (containing abnormal / normal scenes), also perform graph relationship construction to generate a graph structure; input the trained fixed parameter model, output the predicted crowd relationship feature graph; compare the differences between the test sample generated graph and the model output prediction graph through similarity measurement (i.e. calculate the peak signal-to-noise ratio), and finally calculate the abnormal value score to determine whether the crowd behavior is abnormal (the greater the difference, the higher the abnormal probability). The advantages of the above scheme are: (1) Detection framework based on normal mode pre-learning + abnormality difference comparison: breaking through the limitations of traditional frame-by-frame detection, first learning the graph pattern of normal crowd relationship offline, and then comparing the graph differences of the test scene online, adapting to the scene requirements of "complex correlation and occasional abnormality" of crowd behavior, making the abnormality detection more consistent with the crowd interaction logic. (2) Graph structure modeling of crowd relationship: replacing the traditional feature vector with a graph (node + edge) to accurately capture the spatial correlation and behavior interaction (such as personnel gathering and motion coordination) of individuals in the crowd, so that the model can learn more fine-grained normal patterns, improving the accuracy and robustness of abnormality detection. In short, through "graph structure modeling + offline-online staged detection", efficient identification of crowd abnormal events in complex scenes is realized, solving the problem that traditional methods are difficult to depict crowd correlation and have high abnormality missed detection rate.

[0087] Further, the application also proposes a dynamic and adaptive group relationship graph feature weight adjustment mechanism. The adjustment mechanism can analyze the contribution of each feature to group relationship modeling in real time by designing an attention mechanism, for example, automatically increasing the weight of position features and motion features in a crowded scene, and enhancing the influence of appearance features in a low-density scene, thereby adaptively optimizing the representation ability of the graph structure. In addition, a cross-modal feature fusion strategy can be introduced to dynamically weight the optical flow information and appearance features through an attention gate mechanism, avoiding the feature redundancy or missing problem caused by traditional fixed weight, and further improving the generalization and accuracy of anomaly detection, especially for group behavior analysis in complex environments.

[0088] It can be seen that the application proposes a group abnormal event detection method, which comprises: based on the detection information of individual targets of each frame of monitoring picture, constructing the actual group relationship feature graph of the corresponding frame, and determining the group relationship feature graph sequence based on the actual group relationship feature graph of each frame; inputting the group relationship feature graph sequence into a preset graph prediction network, and determining the predicted group relationship feature graph of the future frame through the preset graph prediction network; determining the peak signal-to-noise ratio between the predicted group relationship feature graph of the future frame and the actual group relationship feature graph of the future frame, and generating a target anomaly score according to the peak signal-to-noise ratio; determining whether a group abnormal event occurs according to the target anomaly score, so as to perform early warning processing when a group abnormal event occurs.

[0089] Beneficial effects: based on the detection information of individual targets of each frame of monitoring picture, the application constructs the actual group relationship feature graph of the corresponding frame, and determines the group relationship feature graph sequence based on the actual group relationship feature graph of each frame, so that the group relationship can be quantitatively represented in the form of graph structure. Compared with the traditional method of detecting abnormal events based on only appearance information and motion information, the application more comprehensively captures the key feature of group relationship, provides more rich feature input for anomaly detection, and thus improves the accuracy of anomaly detection. Further, the application inputs the group relationship feature graph sequence into a preset graph prediction network, so as to determine the predicted group relationship feature graph of the future frame through the preset graph prediction network, realize the modeling and prediction of group behavior, and generate a target anomaly score by calculating the peak signal-to-noise ratio between the predicted group relationship feature graph of the future frame and the actual group relationship feature graph of the future frame. In this way, the difference between the predicted group relationship feature graph and the actual group relationship feature graph can be quantified, so as to accurately determine whether a group abnormal event occurs, avoid the inaccurate detection problem caused by not fully considering the group relationship feature in the traditional method, and realize effective early warning processing of group abnormal events.

[0090] Correspondingly, the application also discloses a group abnormal event detection device, as shown in Figure 6 The device comprises:

[0091] The group relationship feature map construction module 11: based on the detection information of individual targets in each frame of monitoring picture, constructs the actual group relationship feature map corresponding to the frame, and determines the group relationship feature map sequence based on the actual group relationship feature maps of each frame;

[0092] The graph prediction network processing module 12: inputs the group relationship feature map sequence into a preset graph prediction network, and determines the predicted group relationship feature map of the future frame through the preset graph prediction network;

[0093] The anomaly score calculation module 13: determines the peak signal-to-noise ratio between the predicted group relationship feature map of the future frame and the actual group relationship feature map of the future frame, and generates a target anomaly score according to the peak signal-to-noise ratio;

[0094] The abnormal event determination and early warning module 14: determines whether a group abnormal event occurs according to the target anomaly score, so as to perform early warning processing when the group abnormal event occurs.

[0095] Among them, the more specific working process of each module described above can refer to the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0096] It can be seen that the present application proposes a group abnormal event detection method, which includes: based on the detection information of individual targets in each frame of monitoring picture, constructing the actual group relationship feature map corresponding to the frame, and determining the group relationship feature map sequence based on the actual group relationship feature maps of each frame; inputting the group relationship feature map sequence into a preset graph prediction network, and determining the predicted group relationship feature map of the future frame through the preset graph prediction network; determining the peak signal-to-noise ratio between the predicted group relationship feature map of the future frame and the actual group relationship feature map of the future frame, and generating a target anomaly score according to the peak signal-to-noise ratio; determining whether a group abnormal event occurs according to the target anomaly score, so as to perform early warning processing when the group abnormal event occurs.

[0097] Beneficial effects: based on the detection information of individual targets of each frame of monitoring picture, the actual group relationship feature map corresponding to the frame is constructed, and the group relationship feature map sequence is determined based on the actual group relationship feature maps of each frame, so as to quantitatively represent the group relationship in the form of graph structure. Compared with the traditional method of detecting abnormal events based on only apparent information and motion information, the application more comprehensively captures the key feature of group relationship, provides more rich feature input for abnormal detection, and improves the accuracy of abnormal detection. Further, the group relationship feature map sequence is input into the preset graph prediction network, so as to determine the predicted group relationship feature map of the future frame through the preset graph prediction network, realize the modeling and prediction of group behavior, and generate the target abnormal score by calculating the peak signal-to-noise ratio between the predicted group relationship feature map of the future frame and the actual group relationship feature map of the future frame. In this way, the difference between the predicted group relationship feature map and the actual group relationship feature map can be quantified, so as to accurately judge whether a group abnormal event occurs, avoid the inaccurate detection problem caused by not fully considering the group relationship feature in the traditional method, and realize the effective early warning processing of the group abnormal event.

[0098] Further, the embodiment of the application also provides a computer program product, including a computer program, the computer program is executed by the processor to realize the steps of the foregoing group abnormal event detection method.

[0099] Further, the embodiment of the application also provides an electronic device. Figure 7 It is an electronic device 20 structure diagram shown according to an exemplary embodiment, the contents in the figure cannot be considered as any limitation on the use range of the application.

[0100] Figure 7 It is an electronic device 20 structure diagram shown according to an exemplary embodiment, the contents in the figure cannot be considered as any limitation on the use range of the application.

[0101] In this embodiment, the power supply 26 is configured to provide operating voltage for each hardware device on the electronic device 20; the communication interface 25 is configured to create a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 25 can be any communication protocol applicable to the technical solution of the present application, which will not be limited herein; the input / output interface 24 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which will not be limited herein.

[0102] In addition, the memory 22 as a carrier for storing resources can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon can include a computer program 221, and the storage mode can be temporary storage or permanent storage. In addition to the computer program 221 capable of completing the group abnormal event detection method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 221 can further include a computer program capable of completing other specific work.

[0103] Further, the present application further discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by a processor to implement the group abnormal event detection method disclosed above.

[0104] The specific steps of the method can refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0105] The embodiments in the present application are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0106] The skilled person can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of the two. In order to clearly show the interchangeability of hardware and software, the components and steps of each example have been described in the above description. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0107] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM, flash memory, ROM, electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. The storage medium can be coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can reside in an ASIC.

[0108] Finally, it should be noted that the terms "first" and "second", etc. are used herein solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0109] The above provides a kind of population abnormal event detection method, computer program product, equipment and medium provided by the application, detailed introduction is carried out in this paper, the principle and implementation mode of the application are described by specific examples;The above example is only used to help understand the method and its core idea of the application;For those skilled in the art, according to the idea of the application, there will be changes in specific implementation mode and application range;In view of the above, the content of the specification should not be understood as the limitation of the application.

Claims

1. A method for detecting abnormal group events, characterized in that, include: Based on the detection information of individual targets in each frame of the monitoring screen, the actual group relationship feature map of the corresponding frame is constructed, and the group relationship feature map sequence is determined based on the actual group relationship feature map of each frame. The group relationship feature map sequence is input into a preset graph prediction network, and the preset graph prediction network is used to determine the predicted group relationship feature map for future frames. Determine the peak signal-to-noise ratio (PSNR) between the predicted group relationship feature map of the future frame and the actual group relationship feature map of the future frame, and generate a target anomaly score based on the PSNR; Based on the target anomaly score, it is determined whether a group anomaly event has occurred, so as to provide early warning processing when the group anomaly event occurs. The step of constructing the actual group relationship feature map for each frame based on the detection information of individual targets in each monitoring frame includes: Identify individual targets in each frame of the monitoring footage and obtain detection information for those individual targets; The individual target is used as a node in the actual group relationship feature graph, and the detection information of the individual target is used as the node attribute of the node to obtain a node set; Based on the interaction strength between any two individual targets, an edge set is obtained; The actual group relationship feature graph is constructed based on the set of nodes and the set of edges.

2. The method for detecting abnormal group events according to claim 1, characterized in that, The detection information includes the confidence level, location information, appearance features, and motion features of the individual target.

3. The method for detecting abnormal group events according to claim 2, characterized in that, The process of identifying individual targets in each frame of the surveillance footage and obtaining detection information for those individual targets includes: The surveillance footage is obtained from the surveillance video; The monitoring image is subjected to multi-scale scaling processing to generate input images of multiple scales, and the input images of multiple scales are input in parallel to the multi-scale target detection model; The multi-scale target detection model is used to identify individual targets in the monitoring screen and obtain the confidence level and location information of the individual targets.

4. The method for detecting abnormal group events according to claim 2, characterized in that, The process of identifying individual targets in each frame of the surveillance footage and obtaining detection information for those individual targets includes: Based on the location information of the individual target, the corresponding area image is extracted from the monitoring screen; The region image is preprocessed, and the preprocessed region image is input into a pre-trained convolutional neural network model so that the appearance features of the individual target can be extracted by the convolutional neural network model; wherein, the appearance features represent the visual appearance features of the individual target.

5. The method for detecting abnormal group events according to claim 2, characterized in that, The process of identifying individual targets in each frame of the surveillance footage and obtaining detection information for those individual targets includes: Based on the location information of the individual target, the corresponding area image is extracted from the monitoring screen; Optical flow information at the region image is extracted using an optical flow estimation network, and the motion characteristics of the individual target are determined based on the optical flow information at the region image.

6. The method for detecting abnormal group events according to claim 2, characterized in that, Also includes: Based on the location information of any two individual targets, the spatial distance between the two individual targets is determined, and the edge weight attribute between the two individual targets is calculated based on the spatial distance; wherein, the edge weight attribute represents the interaction strength between any two individual targets.

7. The method for detecting abnormal group events according to claim 6, characterized in that, The edge set obtained based on the interaction strength between any two individual targets includes: The edge weight attribute between any two individual targets is used as the weight value between any two individual targets, and a target matrix is ​​constructed based on the weight value between any two individual targets; The edge set is determined based on the target matrix.

8. The method for detecting abnormal group events according to claim 1, characterized in that, The step of determining whether a group anomaly event has occurred based on the target anomaly score includes: Determine whether the target abnormal score exceeds a preset score threshold; If the target anomaly score exceeds the preset score threshold, a group anomaly event is determined to have occurred.

9. The method for detecting abnormal group events according to claim 1, characterized in that, The preset graph prediction network includes temporal convolutional blocks and spatial convolutional blocks; wherein, the temporal convolutional blocks process the temporal dimension features of the group relationship feature map sequence, and the spatial convolutional blocks process the node topology features of the actual group relationship feature map in a single frame of the group relationship feature map sequence.

10. The method for detecting abnormal group events according to any one of claims 1 to 9, characterized in that, The step of determining the peak signal-to-noise ratio (PSNR) between the predicted group relationship feature map of the future frame and the actual group relationship feature map of the future frame, and generating a target anomaly score based on the PSNR, includes: Determine the target peak signal-to-noise ratio between the predicted group relationship feature map of the future frame and the actual group relationship feature map of the future frame; The target anomaly score is generated based on the target peak signal-to-noise ratio, the minimum historical peak signal-to-noise ratio, and the maximum historical peak signal-to-noise ratio.

11. The method for detecting abnormal group events according to claim 10, characterized in that, The step of generating the target anomaly score based on the target peak signal-to-noise ratio, the minimum historical peak signal-to-noise ratio, and the maximum historical peak signal-to-noise ratio includes: Calculate the first difference between the target peak signal-to-noise ratio and the minimum historical peak signal-to-noise ratio; Calculate the second difference between the maximum historical peak signal-to-noise ratio and the minimum historical peak signal-to-noise ratio; The ratio of the first difference to the second difference is normalized to obtain the target anomaly score.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the group abnormal event detection method as described in any one of claims 1 to 11.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the group anomaly event detection method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the group abnormal event detection method as described in any one of claims 1 to 11.

Images