Video abnormal event detection method and device, equipment and medium
The video features are extracted through sliding windows and principal component analysis network algorithms, and the problem of video abnormal event detection relying on complex models and manual annotations in the prior art is solved, and simplified unsupervised detection and resource-saving abnormal event location is achieved.
Patent Information
- Application Number
- CN202310583677.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-19
- Publication Date
- 2024-11-22
AI Technical Summary
The existing video anomaly event detection technology relies on convolutional neural network models with complex structures, huge scales and numerous parameters. It requires a large number of manual annotation of normal event samples, making it difficult to effectively detect under unsupervised settings.
The video sequence is preprocessed using a sliding window, and the gradient feature cube and optical flow feature cube are extracted. Block sampling and differential expansion operations are performed through the principal component analysis network algorithm, and the standard deviation of the feature distribution and motion information outliers are calculated, and weighted fusion is performed. To determine the abnormal chunking.
The detection of video abnormal events under unsupervised settings is realized, the convolutional neural network structure is simplified, the computing resource consumption is reduced, and the need for manual labeling is avoided.
Smart Images

Figure CN119007061A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing, and in particular to a method, device, equipment and medium for detecting abnormal video events. Background Art
[0002] In the process of detecting abnormal events in videos, on the one hand, since the same event is located in different spatiotemporal context scenarios, the attribute definitions of abnormal events may be different, and it is necessary to define the relevance of the scenarios for abnormal events; on the other hand, since the frequency of abnormal events is much lower than that of normal events, and the continuous process is extremely difficult to record, there is a lack of available large-capacity abnormal event samples.
[0003] Among the related technologies of video abnormal event detection, video abnormal event detection through semi-supervised learning mostly relies on convolutional neural network models with complex structure, large scale and numerous parameters to ensure high detection accuracy. It is necessary to collect all types of normal event samples and manually label them to build a perfect normal event model. Summary of the invention
[0004] In view of this, the present invention proposes a method, device, equipment and medium for video abnormal event detection, which at least solves the problem that in the above-mentioned related technologies of video abnormal event detection, video abnormal event detection is performed through semi-supervised learning, mostly relying on convolutional neural network models with complex structure, large scale and numerous parameters to ensure high detection accuracy, and it is necessary to collect all types of normal event samples and perform manual labeling to build a perfect normal event model.
[0005] Based on the above purpose, one aspect of an embodiment of the present invention provides a method for data exchange, including: preprocessing a video sequence based on a sliding window to obtain video blocks; extracting a gradient feature cube for expressing appearance information and an optical flow feature cube for expressing motion information from the video blocks; performing block sampling on the gradient feature cube based on a principal component analysis network algorithm and then performing a difference expansion operation to extract a feature map corresponding to each block from the gradient feature cube; calculating the standard deviation of the feature distribution of the feature map corresponding to each block to obtain the corresponding appearance information anomaly value, and calculating the motion information anomaly value of the optical flow feature cube based on the optical flow feature; performing weighted fusion on the appearance information anomaly value and the motion information anomaly value of each video block and comparing them with a preset anomaly threshold to determine whether it is an abnormal block, so as to locate the position of the abnormal event.
[0006] In some embodiments, the step of performing a difference expansion operation after performing block sampling on the gradient feature cube based on a principal component analysis network algorithm to extract a feature map corresponding to each block from the gradient feature cube includes: obtaining overlapping sampling blocks after performing block sampling on any gradient unit in the gradient feature cube according to the pixel size; calculating the maximum gradient value and the minimum gradient value of the overlapping sampling blocks in the corresponding gradient unit, and increasing the difference between different sampling blocks based on the maximum gradient value and the minimum gradient value to extract the feature map corresponding to each block from the gradient feature cube.
[0007] In some embodiments, the step of preprocessing the video sequence based on the sliding window to obtain video blocks includes: setting the width, height and time depth of the sliding window used to preprocess the video sequence; dividing each frame of the video image into a three-dimensional space-time cube according to the sliding window to obtain video blocks.
[0008] In some embodiments, the step of calculating the motion information anomaly value of the optical flow feature cube based on the optical flow feature includes: summing the optical flow amplitudes of all pixels included in the optical flow feature cube to obtain the corresponding motion information anomaly value.
[0009] In some embodiments, the step of performing weighted fusion of the appearance information anomaly value and the motion information anomaly value of each video block and comparing them with a preset abnormal threshold to determine whether it is an abnormal block to locate the position of the abnormal event includes: in response to the result of weighted fusion of the appearance information anomaly value and the motion information anomaly value of each video block being greater than the preset abnormal threshold, confirming that the video block is an abnormal block.
[0010] In some embodiments, the step of weighted fusion of the appearance information anomaly value and the motion information anomaly value of each video block and comparing them with a preset anomaly threshold to determine whether it is an abnormal block to locate the position of the abnormal event also includes: determining whether the video blocks adjacent to the abnormal block are also abnormal blocks; in response to at least one video block adjacent to the abnormal block being an abnormal block at the same time, locating the abnormal event position.
[0011] In some embodiments, the step of performing weighted fusion of the appearance information anomaly values and the motion information anomaly values of each video block and comparing them with a preset abnormal threshold to determine whether it is an abnormal block to locate the position of the abnormal event also includes: in response to the result of weighted fusion of the appearance information anomaly values and the motion information anomaly values of each video block being not greater than the preset abnormal threshold, confirming that the video block is a normal block.
[0012] In another aspect of an embodiment of the present invention, a video abnormal event detection device is provided, comprising: a preprocessing module, used to preprocess a video sequence based on a sliding window to obtain video blocks; a feature cube division module, used to extract a gradient feature cube for expressing appearance information and an optical flow feature cube for expressing motion information in the video blocks; a feature extraction module, used to perform a difference expansion operation on the gradient feature cube after block sampling based on a principal component analysis network algorithm, so as to extract a feature map corresponding to each block from the gradient feature cube; an abnormal value calculation module, used to calculate the standard deviation of the feature distribution of the feature map corresponding to each block, obtain the corresponding appearance information abnormal value, and calculate the motion information abnormal value of the optical flow feature cube based on the optical flow feature; and an abnormal judgment module, used to perform weighted fusion of the appearance information abnormal value and the motion information abnormal value of each video block and compare them with a preset abnormal threshold to determine whether it is an abnormal block, so as to locate the position of the abnormal event.
[0013] Another aspect of an embodiment of the present invention provides a computer device, including at least one processor; and a memory, wherein the memory stores computer instructions that can be executed on the processor, and the instructions implement the steps of the above method when executed by the processor.
[0014] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program that implements the above method steps when executed by a processor.
[0015] The present invention has at least the following beneficial effects: a method for detecting abnormal events in videos proposed by the present invention calculates abnormal values for abnormal discrimination based on the feature distribution of feature maps corresponding to each local video block in each video frame, does not rely on training samples manually labeled in advance, and can perform abnormal event detection in an unsupervised setting. At the same time, the principal component analysis network algorithm used is a simplified convolutional neural network with a small number of convolution layers and a small number of parameters, and does not require a large amount of computing resources to be consumed in the feature extraction process. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.
[0017] Figure 1 The flowchart of a method for detecting abnormal video events provided by an embodiment of the present invention is shown;
[0018] Figure 2The diagram shows an application scenario of the method for detecting abnormal video events provided by an embodiment of the present invention;
[0019] Figure 3 A schematic diagram showing the structure of a video abnormal event detection device provided by an embodiment of the present invention is shown;
[0020] Figure 4 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention is shown;
[0021] Figure 5 A schematic diagram of the structure of a computer-readable storage medium provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0022] The following describes embodiments of the present invention. However, it should be understood that the disclosed embodiments are merely examples and other embodiments can take various alternative forms.
[0023] Furthermore, it should be noted that the terms "includes," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a series of elements includes not only those elements, but may also include elements not explicitly listed or inherent to such processes, methods, articles, or apparatuses.
[0024] One or more embodiments of the present application will be described below with reference to the accompanying drawings.
[0025] Based on the above objectives, a first aspect of an embodiment of the present invention provides an embodiment of a method for detecting abnormal events in a video. Figure 1 The flowchart of a method for detecting abnormal video events provided by an embodiment of the present application is shown in FIG. Figure 1 As shown, a method for detecting abnormal events in a video includes:
[0026] S1, preprocessing the video sequence based on the sliding window to obtain video blocks;
[0027] S2, extracting the gradient feature cube for expressing appearance information and the optical flow feature cube for expressing motion information in the video block;
[0028] S3, performing block sampling on the gradient feature cube based on the principal component analysis network algorithm and then performing a difference expansion operation to extract a feature map corresponding to each block from the gradient feature cube;
[0029] S4, calculating the standard deviation of the feature distribution of the feature graph corresponding to each block, obtaining the corresponding appearance information anomaly value, and calculating the motion information anomaly value of the optical flow feature cube based on the optical flow feature;
[0030] S5. Perform weighted fusion on the appearance information abnormal value and the motion information abnormal value of each video block and compare the result with a preset abnormal threshold to determine whether it is an abnormal block, so as to locate the abnormal event position.
[0031] According to several embodiments of the present invention, the step of performing a difference expansion operation after performing block sampling on a gradient feature cube based on a principal component analysis network algorithm to extract a feature map corresponding to each block from the gradient feature cube includes: obtaining overlapping sampling blocks after performing block sampling on any gradient unit in the gradient feature cube according to the pixel size; calculating the maximum gradient value and the minimum gradient value of the overlapping sampling blocks in the gradient unit to which they correspond, and increasing the difference between different sampling blocks based on the maximum gradient value and the minimum gradient value to extract the feature map corresponding to each block from the gradient feature cube.
[0032] According to several embodiments of the present invention, the step of preprocessing a video sequence based on a sliding window to obtain video blocks includes: setting the width, height and time depth of the sliding window used to preprocess the video sequence; dividing each frame of the video image into a three-dimensional space-time cube according to the sliding window to obtain video blocks.
[0033] According to some embodiments of the present invention, the step of calculating the motion information anomaly value of the optical flow feature cube based on the optical flow feature includes: summing the optical flow amplitudes of all pixels included in the optical flow feature cube to obtain the corresponding motion information anomaly value.
[0034] According to some embodiments of the present invention, the steps of weighted fusion of the appearance information anomaly value and the motion information anomaly value of each video block and comparing them with a preset abnormal threshold to determine whether it is an abnormal block to locate the position of the abnormal event include: in response to the result of weighted fusion of the appearance information anomaly value and the motion information anomaly value of each video block being greater than the preset abnormal threshold, confirming that the video block is an abnormal block.
[0035] According to several embodiments of the present invention, the step of weighted fusion of the appearance information anomaly value and the motion information anomaly value of each video block and comparing them with a preset anomaly threshold to determine whether it is an abnormal block to locate the position of the abnormal event also includes: determining whether the video blocks adjacent to the abnormal block are also abnormal blocks; and locating the abnormal event in response to at least one video block adjacent to the abnormal block being an abnormal block at the same time.
[0036] According to some embodiments of the present invention, the step of performing weighted fusion of the appearance information anomaly value and the motion information anomaly value of each video block and comparing them with a preset abnormal threshold to determine whether it is an abnormal block to locate the position of the abnormal event also includes: in response to the result of weighted fusion of the appearance information anomaly value and the motion information anomaly value of each video block being not greater than the preset abnormal threshold, confirming that the video block is a normal block.
[0037] The following is another embodiment of a method for detecting abnormal video events provided by the present invention.
[0038] Figure 2 The application scenario diagram of the method for detecting abnormal video events provided by the embodiment of the present invention is shown as follows: Figure 2 As shown in the figure, in order to realize the local abnormal event area positioning in the video, the video sequence is preprocessed and the basic unit for analysis and processing is extracted from it. Specifically, a sliding window of size W×H×T is set, where W and H are the width and height of the sliding window, and T is the time depth. Each frame of the video image is divided into two-dimensional image units (Patch) of size W×H and non-overlapping with each other using the sliding window; then the image units of the same spatial coordinates in the consecutive T-frame video images with adjacent temporal dimensions are stacked together to obtain a three-dimensional spatial-temporal cuboid as the basic unit of abnormality detection analysis.
[0039] For each 3D space-time cube, its gradient feature cube and optical flow feature cube are extracted as the carriers of appearance information and motion information for joint detection of appearance and motion abnormal events. The specific calculation process for the gradient feature cube and the optical flow feature cube is as follows:
[0040] (1) Calculate the gradient feature cube
[0041] Calculate the video frame F based on formula (1) t The spatiotemporal gradient of each pixel in the image is obtained by the spatiotemporal gradient map, and p represents F t The pixel points in the space-time gradient Contains three elements: the first two elements F p,x and F p,y are the gradient values in the horizontal and vertical directions of the image, respectively, which are used to describe the posture and shape of the target; the third element F p,t is the gradient value in the time direction, which is used to characterize the change of the target appearance characteristics over time. Each spatiotemporal gradient map contains three channels. Then, the above sliding window is used to sample the sequence composed of multiple spatiotemporal gradient maps to obtain a gradient feature cube, where formula (1) is as follows:
[0042]
[0043] (2) Calculate the optical flow feature cube
[0044] For the optical flow feature cube, the Horn–Schunck optical flow method is first used to calculate the horizontal optical flow value I of each pixel p,x The vertical optical flow value I p,y As the first two channels of the optical flow map. Then use formula (2) to calculate the synthetic optical flow amplitude I of each pixel p As the third channel of the optical flow map. Finally, the optical flow map sequence is processed using a sliding window to obtain an optical flow feature cube. Formula (2) is as follows:
[0045]
[0046] The gradient feature cube is sampled in blocks based on the principal component analysis network algorithm, and then a difference expansion operation is performed to extract the feature map corresponding to each block from the gradient feature cube. The steps of block sampling based on the principal component analysis network algorithm include: principal component analysis PCA (Principal Component Analysis); binary hash coding (Binary Hashing); block histograms (Block-wise Histograms), which correspond to the convolution, nonlinear processing and downsampling operations in traditional neural networks respectively. First, the principal component analysis algorithm is used to learn the multi-layer filter (convolution kernel), then the binary hash coding is used for nonlinear processing, and finally the block histogram is used for resampling to output the block histogram features.
[0047] Each gradient cube used for processing is represented as G = {G1, G2, ... G T}∈ WH×T , for any gradient unit G t , perform pixel-by-pixel sampling of blocks of size k1×k1 around it, and collect all overlapping sampling blocks. The set is represented as where x t,s is the vectorized sampling block, and The next step is to use a patch-difference expansion operation to replace the traditional de-averaging operation to optimize the feature extraction process. This operation increases the difference between different sampling blocks, ensuring that the subsequent principal component analysis filter can more easily capture feature changes, which is conducive to identifying abnormal data. The difference expansion operation is expressed as:
[0048]
[0049] I is a block with samples x t,s Unit vectors of uniform dimensions, is the processed sampling block, is the maximum gradient value in the current gradient cube, is the minimum gradient value. G t The corresponding sampling block set is Use the same process to stack all sampling blocks to form a matrix after processing all gradient units
[0050] Matrix-based Introducing PCA algorithm to solve The eigenvectors corresponding to the first L1 largest eigenvalues of the covariance matrix are used as PCA filters, where the number of filters required corresponding to L1 is expressed as:
[0051]
[0052] Represents the feature vector Mapping to filter matrix The function of express The lth eigenvalue corresponds to the eigenvector.
[0053] Output of PCA filter The most important information in the input gradient feature cube is stored in .
[0054] The principal component analysis network can extract higher semantic features by increasing the number of network layers. For this purpose, the present invention sets two layers, and the processing flow of the second layer is basically the same as that of the first layer. The input is the convolution output of the PCA filter of the first layer, which is expressed as The * is the convolution mapping operation, and appropriate zero-border padding needs to be introduced during the operation to ensure that the output feature map size is consistent with the original input. After the same block sampling, difference expansion and PCA process, the second layer filter is obtained. The number of filters n is set to L2.
[0055] For each gradient unit G t , the first layer outputs L1 convolutional feature maps P t l (l=1,...L1), then in the second layer, for each feature map P t l , using convolutional filters Generate L2 convolution feature maps, expressed as Therefore, the two-layer principal component analysis network outputs a total of L1×L2 feature maps. The input is processed by the binarization function H(z) and then binarized and hashed. The number of bits of the code is the same as The same number is expressed as:
[0056]
[0057] Each pixel feature value of the output feature map is encoded as an interval The integer in .
[0058] Block feature extraction stage: first, each It is further divided into multiple local blocks of size n×n. The local block with spatial position (p, q) is represented as B t (p, q); secondly, calculate and count the histogram information of the pixel feature values of each local block (set interval), indicating Finally, for each local block B t (p,q), and put it in the L1 feature map The histogram features of the corresponding positions in are added as the final histogram feature vector. Therefore, the block feature map corresponding to the gradient unit is expressed as After obtaining the block feature maps corresponding to the gradient units at different spatial positions of the entire gradient map, the entire feature map is represented as and Represent the width and height of the original gradient map respectively.
[0059] For local block B t The apparent anomaly score of (i, j) is calculated based on its histogram features. Generally speaking, the histogram feature distribution of the local block containing only normal events is relatively concentrated, and all pixel feature values are mainly distributed in the front value interval; while for abnormal local blocks, due to the presence of abnormal pixels, the histogram feature distribution is more uniform. For normal blocks, since their pixel feature values are concentrated in the front interval of the histogram and not distributed in the subsequent value intervals, the height values of different intervals of the histogram features are more discrete, which will produce a higher standard deviation; on the contrary, for abnormal blocks, the height values of different intervals of the histogram features are more continuous, which will produce a lower standard deviation. Block B t The standard deviation of the (i,j) histogram feature is calculated as follows:
[0060]
[0061] Among them, s app (i,j) represents B t The apparent anomaly score of (i,j), v{i,j}(δ) represents the height value corresponding to the δth interval of its histogram feature.
[0062] For local block B t The motion anomaly score of (i, j) is obtained by summing the optical flow amplitudes of all pixels:
[0063]
[0064] Where N f For B t The number of pixels in (i,j).
[0065] Generally speaking, m o t The larger (i,j) is, the better B t The faster the pixel movement speed in (i, j), the higher the probability that it contains an unconventional motion event (such as a vehicle suddenly entering a normal walking crowd). app The smaller (i,j) is, the greater the t The higher the probability that (i, j) contains an abnormal appearance target (such as a bicycle slowly entering a normal walking crowd). Therefore, the fusion process of motion anomaly score and appearance anomaly score is as follows:
[0066] s fus =αs mot +β(1-s app ) Formula (8)
[0067] Among them, α and β are weighted fusion weights.
[0068] Finally, based on the abnormal event detection of the single-class classifier, the detection threshold θ is set, and according to the fusion abnormality score, the single-class classifier is used to judge the abnormal local block to achieve abnormal event detection and positioning:
[0069]
[0070] In order to further optimize the abnormal reasoning results and eliminate the intermittent false alarms caused by outliers, a spatial-temporal noise removal process is proposed. If the video frame F t The local block B in t (i, j) is judged as abnormal, and then its local blocks adjacent to it in the temporal dimension and the spatial dimension (i.e., the local blocks adjacent to B in terms of spatiotemporal context information) are checked. t (i,j) has an associated local block), if B t If at least two adjacent blocks of (i, j) are judged as abnormal in any dimension of time domain or space domain, then B is considered t (i,j) is effectively detected as an outlier and is not an outlier.
[0071] A second aspect of the embodiments of the present invention provides a video abnormal event detection device. Figure 3 A schematic diagram of the structure of a firmware loading device provided by an embodiment of the present invention is shown. Figure 3As shown, it includes: a preprocessing module, which is used to preprocess the video sequence based on a sliding window to obtain video blocks; a feature cube division module, which is used to extract the gradient feature cube for expressing the appearance information and the optical flow feature cube for expressing the motion information in the video block; a feature extraction module, which is used to perform a difference expansion operation on the gradient feature cube after block sampling based on the principal component analysis network algorithm, so as to extract the feature map corresponding to each block from the gradient feature cube; an abnormal value calculation module, which is used to calculate the standard deviation of the feature distribution of the feature map corresponding to each block, obtain the corresponding appearance information abnormal value, and calculate the motion information abnormal value of the optical flow feature cube based on the optical flow feature; and an abnormal judgment module, which is used to perform weighted fusion on the appearance information abnormal value and the motion information abnormal value of each video block and compare them with a preset abnormal threshold to determine whether it is an abnormal block, so as to locate the position of the abnormal event.
[0072] A third aspect of an embodiment of the present invention provides a computer device, Figure 4 FIG. 1 is a schematic diagram showing the structure of a computer device provided by an embodiment of the present invention. Figure 4 As shown, a computer device provided by an embodiment of the present invention includes the following modules: at least one processor 021; and a memory 022, wherein the memory 022 stores computer instructions 023 that can be executed on the processor 021, and the computer instructions 023 implement the steps of the method described above when executed by the processor 021.
[0073] The present invention also provides a computer-readable storage medium. Figure 5 FIG. 1 is a schematic diagram of a computer-readable storage medium provided by an embodiment of the present invention. Figure 5 As shown, the computer-readable storage medium 031 stores a computer program 032 that performs the steps of the method described above when executed by a processor.
[0074] Finally, it should be noted that a person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program of the method for setting system parameters can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium of the program can be a disk, an optical disk, a read-only storage memory (ROM) or a random access memory (RAM), etc. The above-mentioned computer program embodiments can achieve the same or similar effects as the corresponding above-mentioned arbitrary method embodiments.
[0075] In addition, the method disclosed in the embodiment of the present invention can also be implemented as a computer program executed by a processor, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the above functions defined in the method disclosed in the embodiment of the present invention are performed.
[0076] In addition, the above method steps and system units may also be implemented using a controller and a computer-readable storage medium for storing a computer program that enables the controller to implement the above steps or unit functions.
[0077] It will also be appreciated by those skilled in the art that various exemplary logic blocks, modules, circuits and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given to the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.
[0078] In one or more exemplary designs, the function can be implemented in hardware, software, firmware or any combination thereof. If implemented in software, the function can be stored on a computer-readable medium or transmitted by a computer-readable medium as one or more instructions or codes. Computer-readable media include computer storage media and communication media, and the communication media include any media that helps to transmit a computer program from one location to another. The storage medium can be any available medium that can be accessed by a general or special computer. As an example and not limiting, the computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store the required program code in the form of an instruction or data structure and can be accessed by a general or special computer or a general or special processor. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technologies such as infrared, radio and microwaves are used to send software from a website, server or other remote source, the above-mentioned coaxial cable, optical fiber cable, twisted pair, DOL or wireless technologies such as infrared, radio and microwaves are all included in the definition of the medium. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, Blu-ray disc, wherein disks usually reproduce data magnetically, while optical discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
[0079] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.
[0080] It should be understood that, as used herein, the singular forms "a", "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations including one or more of the associated listed items.
[0081] The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0082] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0083] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the embodiments of the present invention as above, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.
Claims
1. A method for detecting abnormal events in a video, characterized in that: include: Preprocess the video sequence based on the sliding window to obtain video blocks; Extracting a gradient feature cube for expressing appearance information and an optical flow feature cube for expressing motion information from the video block; The gradient feature cube is subjected to block sampling based on a principal component analysis network algorithm, and then a difference expansion operation is performed to extract a feature map corresponding to each block from the gradient feature cube; Calculate the standard deviation of the feature distribution of the feature map corresponding to each block to obtain the corresponding appearance information anomaly value, and calculate the motion information anomaly value of the optical flow feature cube based on the optical flow feature; The appearance information abnormal value and the motion information abnormal value of each video block are weighted and fused, and then compared with a preset abnormal threshold to determine whether it is an abnormal block, so as to locate the position of the abnormal event.
2. The method according to claim 1, characterized in that The step of performing block sampling on the gradient feature cube based on the principal component analysis network algorithm and then performing a difference expansion operation to extract a feature map corresponding to each block from the gradient feature cube includes: Obtain overlapping sampling blocks after performing block sampling on any gradient unit in the gradient feature cube according to the pixel size; The maximum gradient value and the minimum gradient value of the overlapping sampling blocks in the corresponding gradient units are calculated, and the difference between different sampling blocks is increased based on the maximum gradient value and the minimum gradient value to extract a feature map corresponding to each block from the gradient feature cube.
3. The method according to claim 1, characterized in that The step of preprocessing the video sequence based on the sliding window to obtain video blocks includes: Set the width, height and temporal depth of the sliding window used to preprocess the video sequence; Each frame of video image is divided into a three-dimensional space-time cube according to the sliding window to obtain video blocks.
4. The method according to claim 1, characterized in that The step of calculating the motion information anomaly value of the optical flow feature cube based on the optical flow feature comprises: The optical flow amplitudes of all pixels included in the optical flow feature cube are summed to obtain the corresponding motion information anomaly value.
5. The method according to claim 1, characterized in that The step of weighted fusion of the appearance information abnormal value and the motion information abnormal value of each video block and comparing them with a preset abnormal threshold to determine whether it is an abnormal block to locate the position of the abnormal event includes: In response to the result of weighted fusion of the appearance information abnormal value and the motion information abnormal value of each video block being greater than the preset abnormal threshold, the video block is confirmed to be an abnormal block.
6. The method according to claim 5, characterized in that The step of weighted fusion of the appearance information abnormal value and the motion information abnormal value of each video block and comparing them with a preset abnormal threshold to determine whether it is an abnormal block to locate the abnormal event position also includes: Determine whether the video blocks adjacent to the abnormal block are also abnormal blocks; In response to at least one video block adjacent to the abnormal block being an abnormal block at the same time, the abnormal event position is located.
7. The method according to claim 1, characterized in that The step of weighted fusion of the appearance information abnormal value and the motion information abnormal value of each video block and comparing them with a preset abnormal threshold to determine whether it is an abnormal block to locate the abnormal event position also includes: In response to the result of weighted fusion of the appearance information abnormal value and the motion information abnormal value of each video block being not greater than the preset abnormal threshold, the video block is confirmed to be a normal block.
8. A video abnormal event detection device, characterized in that: include: A preprocessing module, used for preprocessing the video sequence based on a sliding window to obtain video blocks; A feature cube partitioning module, used to extract a gradient feature cube for expressing appearance information and an optical flow feature cube for expressing motion information from the video blocks; A feature extraction module, used for performing block sampling on the gradient feature cube based on a principal component analysis network algorithm and then performing a difference expansion operation to extract a feature map corresponding to each block from the gradient feature cube; An outlier calculation module, used to calculate the standard deviation of the feature distribution of the feature map corresponding to each block, obtain the corresponding appearance information outlier, and calculate the motion information outlier of the optical flow feature cube based on the optical flow feature; as well as The abnormality judgment module is used to perform weighted fusion on the appearance information abnormal value and the motion information abnormal value of each video block and compare them with a preset abnormal threshold to determine whether it is an abnormal block, so as to locate the position of the abnormal event.
9. A computer device, characterized in that: include: at least one processor; as well as A memory storing computer instructions executable on the processor, wherein the instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that implements the steps of the method according to any one of claims 1 to 7 when executed by a processor.
