Scene image anomaly detection method and system based on frame difference probability grammar dictionary

This video anomaly detection method, which utilizes a frame difference probability grammar dictionary and a Gaussian mixture model, solves the challenge of identifying unlabeled or simply labeled video anomalies in video surveillance. It achieves efficient and flexible anomaly detection, is applicable to various scenarios, and reduces hardware and labeling costs.

CN119007067APending Publication Date: 2024-11-22DMAI (GUANGZHOU) CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411020360.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Existing technologies in video surveillance require extensive manual feature extraction or data-dependent deep learning methods, making it difficult to achieve video anomaly recognition without annotation or with simple annotation.

Method used

A frame difference probability grammar dictionary-based approach is adopted. By acquiring background images and scene video streams, performing frame difference grayscale and binarized difference map processing, a frame difference probability grammar dictionary is established, and abnormal frames are identified using Gaussian mixture model and information content value.

Benefits of technology

It enables video anomaly recognition that requires no or only simple annotation, is applicable to various scenarios, reduces the workload of manual annotation, improves the applicability and flexibility of the method, has low hardware configuration requirements, and low deployment costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007067A_ABST
    Figure CN119007067A_ABST
Patent Text Reader

Abstract

The invention discloses a scene image anomaly detection method based on a frame difference probability grammar dictionary. The method comprises the following steps: acquiring a scene video stream to be detected and a background image corresponding to the scene video stream; obtaining a background frame difference gray difference image, a background frame difference binary difference image, a sequence frame difference gray difference image and a sequence frame difference binary difference image based on the background image and the scene video stream; setting a grid range and a grid number; obtaining a frame difference probability grammar dictionary and an attribute matrix of each frame image in the scene video stream based on the grid range, the grid number, the background frame difference gray difference image, the background frame difference binarization difference image, the sequence frame difference gray difference image and the sequence frame difference binarization difference image; substituting the attribute matrix into a frame difference probability grammar dictionary to obtain a probability matrix; obtaining an information amount value based on the probability matrix; and comparing the information amount value with a set threshold value, and when the information amount value is greater than the set threshold value, determining that the image is an abnormal frame image. And video anomaly identification can be simply realized without labeling or simple labeling.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of video surveillance technology, and more particularly to a scene image anomaly detection method and system based on a frame difference probability grammar dictionary. Background Art

[0002] Currently, in the field of video surveillance, real-time analysis of dynamic scenes is crucial to the effectiveness of security monitoring systems. Frame difference image technology, as a commonly used image processing method, effectively extracts moving objects in the scene by comparing the differences between consecutive frames in a video sequence. This method is widely used in front-end image representation of various neural networks due to its simple implementation and high computational efficiency, especially in the fields of dynamic scene analysis and motion detection, showing significant application value.

[0003] However, traditional frame difference methods require a large number of manual feature extraction methods to perform some form of "parameter adjustment" to obtain professional knowledge, and can only be used in one or two scenarios. For example, the prior art CN202410421092.4 requires expert knowledge to construct features to identify mice; some deep learning methods also often stop at frame difference images for end-to-end neural network methods. This method requires a large amount of data for model training and is heavily dependent on both data and computing power. For example, the prior art CN202410534723.3 uses the original video image and video difference frame image as the input of the deep neural network DNN model for training and reasoning.

[0004] Therefore, how to simply realize video anomaly recognition without labeling or with simple labeling is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a scene image anomaly detection method and system based on a frame difference probability grammar dictionary, which can easily realize video anomaly recognition without labeling or with simple labeling.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A scene image anomaly detection method based on a frame difference probability grammar dictionary, comprising:

[0008] Obtaining a video stream of a scene to be detected and its corresponding background image;

[0009] Based on the background image and the scene video stream, a background frame difference grayscale difference map, a background frame difference binary difference map, a sequence frame difference grayscale difference map and a sequence frame difference binary difference map are obtained;

[0010] Set the grid range and grid number;

[0011] Obtaining a frame difference probability grammar dictionary and an attribute matrix of each frame image in the scene video stream based on the grid range, the number of grids, the background frame difference grayscale difference map, the background frame difference binary difference map, the sequence frame difference grayscale difference map and the sequence frame difference binary difference map;

[0012] Substituting the attribute matrix into the frame difference probability grammar dictionary to obtain a probability matrix;

[0013] Obtaining an information value based on the probability matrix;

[0014] The information amount value is compared with a set threshold value, and when it is greater than the set threshold value, it is determined to be an abnormal frame image.

[0015] Preferably, the background image acquisition method is:

[0016] Obtain a video stream of an empty scene and intercept a preset time period to obtain an intercepted video stream;

[0017] Preprocessing is performed based on the intercepted video stream to obtain the background image.

[0018] Preferably, the pretreatment comprises:

[0019] Calculate the pixel values ​​of each channel of all frame images in the intercepted video stream;

[0020] The background image is obtained based on all the pixel values.

[0021] Preferably, obtaining the background frame difference grayscale difference image and the background frame difference binary difference image specifically includes:

[0022] Based on the current frame image in the scene video stream and the background image, the absolute value of the difference is calculated pixel by pixel and channel by channel to obtain a background difference map;

[0023] Converting the background difference map into the background frame difference grayscale difference map based on the background difference map;

[0024] Setting a first threshold to perform binarization processing on the background grayscale difference image to obtain a first binarized difference image;

[0025] An expansion and corrosion operation is performed based on the first binary difference image to obtain the background frame difference binary difference image.

[0026] Preferably, obtaining the sequence frame difference grayscale difference image and the sequence frame difference binary difference image specifically includes:

[0027] Based on the absolute value calculation of the difference between the current frame image and the previous interval frame image in the scene video stream, pixel by pixel and channel by channel, a sequence difference map is obtained;

[0028] Converting the sequence difference map into an intermediate grayscale difference map;

[0029] Performing edge detection based on the intermediate grayscale difference image to obtain an edge mask image;

[0030] Performing binarization processing based on the intermediate grayscale difference image to obtain a second binarized difference image;

[0031] Performing an expansion and corrosion operation based on the second binary difference image to obtain the intermediate binary difference image;

[0032] Based on the edge mask image, the intermediate binary difference image is subtracted, and all negative values ​​are set to 0 to obtain the sequence frame difference binary difference image;

[0033] The grayscale value of the position marked by the edge mask image in the intermediate grayscale difference image is set to 0 to obtain the sequence frame difference grayscale difference image.

[0034] Preferably, the method for acquiring the frame difference probability grammar dictionary is:

[0035] Based on the grid range, the background frame difference grayscale difference map, the background frame difference binary difference map, the sequence frame difference grayscale difference map and the sequence frame difference binary difference map, four attribute values ​​of the number of background frame difference dynamic blocks, the background frame difference activation strength, the number of sequence frame difference dynamic blocks and the sequence frame difference activation strength are obtained;

[0036] Based on the four attribute values ​​in each grid in all frame images in the scene video stream, four one-dimensional vectors w are combined i , i=1,2,3,4;

[0037] Based on all the one-dimensional vectors w i Perform statistical modeling, and obtain a Gaussian mixture model corresponding to the four attributes for each grid;

[0038] The frame difference probability grammar dictionary is obtained based on the Gaussian mixture models corresponding to the four attributes in all grids.

[0039] Preferably, the method for obtaining the number of background frame difference dynamic blocks and the background frame difference activation strength is:

[0040] Counting the number of all non-zero values ​​within the grid range in the background frame difference binary difference image as the number of background frame difference dynamic blocks;

[0041] Counting the average grayscale intensity of all non-zero pixels within the grid range in the background frame difference binary difference image as the background frame difference activation intensity;

[0042] The average grayscale intensity is obtained based on the determined non-zero pixel points in the background frame difference grayscale difference map.

[0043] Preferably, the method for obtaining the number of sequence frame difference dynamic blocks and the sequence frame difference activation strength is:

[0044] Counting the number of all non-zero values ​​within the grid range in the sequence frame difference binary difference image as the number of sequence frame difference dynamic blocks;

[0045] Counting the average grayscale intensity of all non-zero pixels within the grid range in the sequence frame difference binary difference image as the sequence frame difference activation intensity;

[0046] The average grayscale intensity is obtained based on the determined non-zero pixel points in the sequence frame difference grayscale difference map.

[0047] Preferably, the attribute matrix acquisition method is:

[0048] Based on the grid range, the background frame difference grayscale difference map, the background frame difference binary difference map, the sequence frame difference grayscale difference map and the sequence frame difference binary difference map, four attribute values ​​of the background frame difference dynamic block number, the background frame difference activation strength, the sequence frame difference dynamic block number and the sequence frame difference activation strength are obtained accordingly;

[0049] Based on the four attribute values ​​and the number of grids n, an attribute matrix of size 4*n is obtained, where 4 represents four attribute values.

[0050] Preferably, the information value acquisition method is:

[0051] Converting the probability matrix into an information matrix;

[0052] Set the first weight of each mesh;

[0053] Calculating the weighted sum of each information vector based on the first dimension axis direction of the information matrix and the first weight to obtain the information vectors corresponding to the four attribute values;

[0054] Setting the acquisition probability of the background frame difference dynamic block quantity and the background frame difference activation intensity as a second weight;

[0055] The number of the sequence frame difference dynamic blocks and the acquisition probability of the sequence frame difference activation intensity are both set as a third weight;

[0056] The information value is obtained by performing weighted summation based on the second weight, the third weight, and the information vectors corresponding to the four attribute values.

[0057] A scene image anomaly detection system based on a frame difference probability grammar dictionary comprises: a data acquisition module, a difference map acquisition module, a grid setting module, an image processing module, an information value acquisition module and a result determination module;

[0058] The data acquisition module is used to acquire the scene video stream to be detected and its corresponding background image;

[0059] The difference image acquisition module is used to obtain a background frame difference grayscale difference image, a background frame difference binary difference image, a sequence frame difference grayscale difference image and a sequence frame difference binary difference image based on the background image and the scene video stream;

[0060] The grid setting module is used to set the grid range and the number of grids;

[0061] The image processing module is used to obtain a frame difference probability grammar dictionary and an attribute matrix of each frame image based on the grid range, the number of grids, the background frame difference grayscale difference map, the background frame difference binary difference map, the sequence frame difference grayscale difference map and the sequence frame difference binary difference map;

[0062] The information value acquisition module is used to bring the attribute matrix into the frame difference probability grammar dictionary to obtain a probability matrix; and obtain the information value based on the probability matrix;

[0063] The result determination module is used to compare the information amount value with a set threshold value, and when the information amount value is greater than the set threshold value, determine it as an abnormal frame image.

[0064] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a scene image anomaly detection method and system based on a frame difference probability grammar dictionary, which captures dynamic changes in image sequences through inter-frame difference technology, and uses a probability grammar model to characterize and analyze these changes, thereby identifying abnormal behaviors, and has the following beneficial effects:

[0065] 1. The inter-frame difference technology combined with the probabilistic grammar model can be applied to various scene anomaly detection fields, including but not limited to conference rooms, classrooms, monitoring, etc., and the proposed probabilistic representation method of the image has positive significance for subsequent tasks.

[0066] 2. The present invention can perform efficient rough screening of abnormality identification without labeling or with only simple labeling, which significantly reduces the workload and complexity of manual labeling.

[0067] 3. Different from the traditional method that requires precise definition of abnormal categories, the present invention does not need to make excessive definitions of abnormal categories during the abnormality identification process, thereby improving the applicability and flexibility of the method.

[0068] 4. The hardware configuration requirements are not high and the deployment cost is low.

[0069] 5. The method of the present invention is not only applicable to anomaly detection, but also has positive significance for subsequent image processing and analysis tasks, thus enhancing the application value of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0071] Figure 1 A flow chart of a scene image anomaly detection method based on a frame difference probability grammar dictionary provided by the present invention.

[0072] Figure 2 A flow chart of a method for obtaining a background frame difference grayscale difference image and a background frame difference binarization difference image provided by the present invention.

[0073] Figure 3 A flow chart of a method for obtaining a sequence frame difference grayscale difference image and a sequence frame difference binarization difference image provided by the present invention.

[0074] Figure 4 A schematic diagram of the structure of a scene image anomaly detection system based on a frame difference probability grammar dictionary provided by the present invention.

[0075] Figure 5 This is a visualized thermal diagram after operation provided by the present invention.

[0076] Figure 6 This is a visualized thermal diagram of the present invention without any operation.

[0077] Figure 7 This is a schematic diagram of the frequency distribution of the normal sample information provided by the present invention.

[0078] Figure 8 This is a schematic diagram of the frequency distribution of the abnormal sample information provided by the present invention. DETAILED DESCRIPTION

[0079] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0080] Example 1

[0081] like Figure 1 As shown, the embodiment of the present invention discloses a scene image anomaly detection method based on a frame difference probability grammar dictionary, comprising:

[0082] Obtaining a video stream of a scene to be detected and its corresponding background image;

[0083] Based on the background image and the scene video stream, a background frame difference grayscale difference map, a background frame difference binary difference map, a sequence frame difference grayscale difference map and a sequence frame difference binary difference map are obtained;

[0084] Set the grid range and grid number;

[0085] Based on the grid range, the number of grids, the background frame difference grayscale difference map, the background frame difference binary difference map, the sequence frame difference grayscale difference map and the sequence frame difference binary difference map, a frame difference probability grammar dictionary and an attribute matrix of each frame image in the scene video stream are obtained;

[0086] Substitute the attribute matrix into the frame difference probability grammar dictionary to obtain a probability matrix;

[0087] Get the information value based on the probability matrix;

[0088] The information value is compared with the set threshold. When it is greater than the set threshold, it is determined to be an abnormal frame image.

[0089] Example 2

[0090] The embodiment of the present invention discloses a scene image anomaly detection method based on a frame difference probability grammar dictionary, comprising:

[0091] Training phase:

[0092] The training phase requires artificially setting a time period. The main purpose is to obtain a frame difference probability grammar dictionary, which can characterize the average image probability within the time period, so as to obtain a probabilistic representation of the image change relative to the average image in this time period. The specific process is:

[0093] Get the scene video stream to be detected and its corresponding background image:

[0094] Preferably, the present invention can be extended to N video streams. This embodiment takes the processing flow of one video stream as an example and is applicable to various scenarios, including but not limited to conference rooms, classrooms, monitoring, etc.

[0095] Preferably, the background image acquisition method is:

[0096] Obtain a video stream of an empty scene and intercept a preset time period to obtain an intercepted video stream;

[0097] The background image is obtained by preprocessing the intercepted video stream.

[0098] Preferably, an unoccupied scene is manually configured to ensure that the scene contains neither unnecessary personnel nor redundant items, and the scene layout remains normal, as a vacant scene.

[0099] Preferably, the pretreatment comprises:

[0100] Calculate the pixel values ​​of each channel of all frame images in the intercepted video stream;

[0101] Get the background image based on all pixel values.

[0102] Preferably, assuming that the intercepted video stream contains t frame images, the specifications of each frame image are: height h, width w and 3 color channels (RGB);

[0103] For any pixel in any frame image, its pixel value on a single color channel is expressed as P(x,i,j,z), where x represents the frame number, and the value range of x is [0,t]; i and j represent the vertical and horizontal positions of the pixel in the frame image, and the value ranges are [0,h] and [0,w] respectively; z represents the color channel, and the value range is [1,3], corresponding to the three RGB color channels. Therefore, the pixel value of each channel in the average image is calculated by dividing the sum of all corresponding pixel values ​​by the number of frames T, that is:

[0104] Based on the background image and the scene video stream, the background frame difference grayscale difference map, the background frame difference binary difference map, the sequence frame difference grayscale difference map and the sequence frame difference binary difference map are obtained:

[0105] Preferably, Figure 2 As shown, the background frame difference grayscale difference map and the background frame difference binary difference map are obtained, which specifically include:

[0106] Based on the absolute value calculation of the difference between the current frame image and the background image in the scene video stream, pixel by pixel and channel by channel, a background difference map is obtained;

[0107] Based on the background difference map, it is converted into a background frame difference grayscale difference map;

[0108] Setting a first threshold to perform binarization processing on the background grayscale difference image to obtain a first binarized difference image;

[0109] A dilation and corrosion operation is performed based on the first binary difference image to obtain a background frame difference binary difference image.

[0110] Preferably, in this embodiment, the first threshold is set to 70.

[0111] Preferably, this embodiment performs m dilation operations based on the first binary difference image to expand the connected area; and then performs m+1 erosion operations to reduce noise points.

[0112] Preferably, Figure 3 As shown, a sequence frame difference grayscale difference map and a sequence frame difference binary difference map are obtained, which specifically include:

[0113] Based on the absolute value calculation of the difference between the current frame image and the previous interval frame image in the scene video stream, pixel by pixel and channel by channel, a sequence difference map is obtained;

[0114] Convert the sequence difference map into an intermediate grayscale difference map;

[0115] Perform edge detection based on the intermediate grayscale difference image to obtain an edge mask image;

[0116] Performing binarization processing based on the intermediate grayscale difference image to obtain a second binarized difference image;

[0117] Performing an expansion and corrosion operation based on the second binary difference image to obtain an intermediate binary difference image;

[0118] Based on the edge mask image, the intermediate binary difference image is subtracted, and all negative values ​​are set to 0 to obtain the sequence frame difference binary difference image;

[0119] The grayscale values ​​of the positions marked by the edge mask image in the intermediate grayscale difference image are set to 0 to obtain a sequence frame difference grayscale difference image.

[0120] Preferably, this embodiment performs m dilation operations based on the first binary difference image to expand the connected area; and then performs m+1 erosion operations to reduce noise points.

[0121] Preferably, in this embodiment, the interval of the previous interval frame image is set to 5 frames per second, that is, the interval between each frame is 0.2 seconds, so as to accurately capture the dynamic changes in the time series.

[0122] Preferably, edge detection is performed based on the intermediate grayscale difference map, and the detected edge map is dilated to enhance the edge and generate an edge mask map. Assuming that the dimension of the original image is (1080, 1920, 3), the dimension of the edge mask map is (1080, 1920), the edge part is 1, and the rest is 0.

[0123] Preferably, based on the edge mask image minus the intermediate binary difference image, all negative values ​​are set to 0 to obtain a sequence frame difference binary difference image, in order to reduce the impact caused by slight jitter.

[0124] Preferably, the generated background frame difference grayscale difference map, background frame difference binary difference map, sequence frame difference grayscale difference map and sequence frame difference binary difference map can provide an accurate basis for in-depth understanding and analysis of dynamic changes in the scene.

[0125] Preferably, the background frame difference map is to quantify the absolute changes in the scene by comparing the difference between the current frame and the background image, aiming to capture the overall change from the background state to the current state.

[0126] Preferably, the sequence frame difference image is used to evaluate the relative change in the scene by comparing the current frame image with the previous frame image at a specific interval.

[0127] Preferably, two specific difference images, namely, background frame difference image and sequence frame difference image, are used to capture and analyze dynamic changes in the scene.

[0128] Set the grid range and number of grids:

[0129] Preferably, by carefully setting the grid range and the number of grids, the specific dimensions of the final image representation can be clearly determined. In theory, the size of the grid has the potential to be refined to a level equal to that of a single pixel. Although this high-precision setting is technically feasible, considering the robustness requirements for probability representation in practical applications and the rational use of computing resources, a more practical grid size of 50×100 is selected in the specific embodiment of the present invention. In this embodiment, the grid range is set to 50×100; this size selection takes into account both the ability to capture details and the computational efficiency, providing a solid foundation for subsequent statistical analysis. In terms of processed video stream data, it can be applied to pictures with a specification of 1080×1920×3, that is, it can process high-definition video streams to ensure that the quality and details of the image are fully preserved when extracting probability representations. Through such a design, the present invention aims to provide an efficient and robust starting point for complex probabilistic statistical analysis to support subsequent more in-depth data processing and analysis.

[0130] The frame difference probability grammar dictionary is obtained based on the grid range, grid number, background frame difference grayscale difference map, background frame difference binary difference map, sequence frame difference grayscale difference map and sequence frame difference binary difference map:

[0131] Preferably, the method for obtaining the frame difference probability grammar dictionary is:

[0132] Based on the grid range, background frame difference grayscale difference map, background frame difference binary difference map, sequence frame difference grayscale difference map and sequence frame difference binary difference map, four attribute values, namely the number of background frame difference dynamic blocks, the background frame difference activation intensity, the number of sequence frame difference dynamic blocks and the sequence frame difference activation intensity, are obtained;

[0133] Based on the four attribute values ​​in each grid in all frame images in the scene video stream, four one-dimensional vectors w are combined i , i=1,2,3,4;

[0134] Based on all one-dimensional vectors w i Statistical modeling is performed to obtain a Gaussian mixture model corresponding to four attributes for each grid, and its probability density function is modeled;

[0135] Based on the Gaussian mixture models corresponding to the four attributes in all grids, a frame difference probability grammar dictionary is obtained.

[0136] Preferably, the frame difference probability grammar dictionary contains 4*n Gaussian mixture models, where n represents the number of grids, and n is 500 in this embodiment.

[0137] Preferably, the dictionary provides a powerful probabilistic tool for image analysis by describing the specific properties of each cell in the grid partitioning module. For any cell i in the grid, the value range of i is [0,499].

[0138] Preferably, the method for obtaining the number of background frame difference dynamic blocks and the background frame difference activation intensity is:

[0139] Count the number of all non-zero values ​​within the grid range of the background frame difference binary difference image as the number of background frame difference dynamic blocks;

[0140] The average grayscale intensity of all non-zero pixels in the grid range within the background frame difference binary difference image is counted as the background frame difference activation intensity;

[0141] The average gray intensity is obtained based on the determined non-zero pixel points in the background frame difference grayscale difference map.

[0142] Preferably, the non-zero value represents the non-zero value in the background frame difference binary difference image, and the value range of the number of background frame difference dynamic blocks is [0,4147], denoted as w 1 .

[0143] Preferably, the non-zero pixel point refers to: assuming that the size of the original image is 1080*1920, the pixel at the coordinate [10,10] belongs to the 0th grid. If it is 1 in the background frame difference binary difference image, then it is a dynamic block, and the number of background frame difference dynamic blocks of the grid is increased by 1. At this time, the pixel point at the coordinate [10,10] is regarded as a non-zero pixel point.

[0144] Preferably, the background frame difference activation intensity has a value range of [0,255], denoted as w 2 .

[0145] Preferably, the method for obtaining the number of sequence frame difference dynamic blocks and the sequence frame difference activation strength is:

[0146] Count the number of all non-zero values ​​within the grid range of the binary difference image of the sequence frame difference as the number of dynamic blocks of the sequence frame difference, whose value range is [0,4147], denoted as w 3 .

[0147] The average grayscale intensity of all non-zero pixels in the grid range of the binary difference image of the sequence frame difference is counted as the sequence frame difference activation intensity, whose value range is [0,255], denoted as w 4 .

[0148] The average gray intensity is obtained based on the determined non-zero pixel points in the gray difference image of the sequence frame difference.

[0149] Preferably, based on the above four attributes, four probabilities can be defined, denoted as P i (w i ), where i ranges from [1,4]. In the present invention, a typical probability in the probability grammar model is selected, namely, the probability of occurrence of the attribute value.

[0150] Preferably, the frame difference probability grammar dictionary provides a sophisticated and efficient probability description for the dynamic changes of the image, making the image analysis more accurate and efficient.

[0151] Operation phase:

[0152] The running phase depends on the completion of the training phase and requires the use of the frame difference probability grammar dictionary built in the training phase. This dictionary provides the necessary probability model foundation for anomaly detection, enabling anomaly detection to identify abnormal behavior based on the learned normal behavior patterns. The specific process is as follows:

[0153] Based on the grid range, grid number, background frame difference grayscale difference map, background frame difference binary difference map, sequence frame difference grayscale difference map and sequence frame difference binary difference map, the attribute matrix of each frame image in the scene video stream is obtained:

[0154] Preferably, the attribute matrix acquisition method is:

[0155] Based on the grid range, background frame difference grayscale difference map, background frame difference binary difference map, sequence frame difference grayscale difference map and sequence frame difference binary difference map, four attribute values ​​are obtained, namely the number of background frame difference dynamic blocks, the background frame difference activation intensity, the number of sequence frame difference dynamic blocks and the sequence frame difference activation intensity; the method for obtaining the four attribute values ​​is the same as the above method and will not be repeated here.

[0156] Based on the four attribute values ​​and the number of grids n, a 4*n attribute matrix is ​​obtained, where 4 represents the four attribute values.

[0157] Preferably, this embodiment forms a 4*500 attribute matrix.

[0158] Substitute the attribute matrix into the frame difference probability grammar dictionary to obtain the probability matrix:

[0159] Preferably, each attribute value in the attribute matrix is ​​input into the Gaussian mixture model corresponding to the frame difference probability grammar dictionary, and its prior probability is calculated to obtain a probability matrix of size 4*500. This probability matrix represents the image difference probability representation of the sample.

[0160] Get the information value based on the probability matrix:

[0161] Preferably, the information value acquisition method is:

[0162] Convert the probability matrix into an information matrix;

[0163] Set the first weight of each mesh;

[0164] Calculate the weighted sum of each information vector based on the first dimension axis direction and the first weight of the information matrix to obtain the information vectors corresponding to the four attribute values;

[0165] The acquisition probabilities of the number of background frame difference dynamic blocks and the background frame difference activation intensity are both set as the second weight;

[0166] The acquisition probabilities of the number of sequence frame difference dynamic blocks and the activation intensity of sequence frame difference are both set as the third weight;

[0167] The information value is obtained by performing weighted summation based on the second weight, the third weight and the information vectors corresponding to the four attribute values.

[0168] Preferably, in this embodiment, the first weight of each grid is set to 1. For areas that do not need attention, the weight can be appropriately reduced. In this embodiment, the weight of areas that do not need attention (such as several time display areas at the edge) is set to 0.3.

[0169] Preferably, the second weight and the third weight are set according to actual needs. When the absolute difference is emphasized, the second weight is greater than the third weight; when the dynamic difference is emphasized, the third weight is greater than the second weight. In this embodiment, the second weight and the third weight are set to 0.2 and 0.8 respectively.

[0170] Preferably, the probability matrix of a single frame image of size 4*500 is converted into information quantity I: I=-1*log2(p(x)), where p(x) represents the probability matrix, and a 4*500 information quantity matrix is ​​obtained.

[0171] Preferably, the amount of information I is solved in any manner that is inversely proportional to the probability.

[0172] The information value is compared with the set threshold. When it is greater than the set threshold, it is determined to be an abnormal frame image.

[0173] Preferably, the information value is compared with a set threshold value, and when the information value is greater than the set threshold value, it is determined to be an abnormal frame image, and when the information value is less than or equal to the set threshold value, it is determined to be a normal frame image. The above judgment process is repeated until all frame images in the video stream of the scene to be detected are judged, and all abnormal frame images in the video stream of the scene to be detected are identified.

[0174] Preferably, an end-to-end neural network, a traditional machine learning model or an information volume threshold is used for discrimination. In the embodiment of the present invention, abnormal discrimination is performed by using information volume. The simplest discriminator can also identify some obvious abnormal behaviors.

[0175] Preferably, a deep learning or machine learning classifier may be used to classify the frame difference probability representation vector of the image, but this requires more annotation costs.

[0176] Example 3

[0177] like Figure 4 As shown, a scene image anomaly detection system based on a frame difference probability grammar dictionary includes: a data acquisition module, a difference map acquisition module, a grid setting module, an image processing module, an information value acquisition module and a result determination module;

[0178] A data acquisition module is used to acquire a video stream of a scene to be detected and its corresponding background image;

[0179] A difference image acquisition module is used to obtain a background frame difference grayscale difference image, a background frame difference binary difference image, a sequence frame difference grayscale difference image and a sequence frame difference binary difference image based on the background image and the scene video stream;

[0180] Grid setting module, used to set the grid range and grid quantity;

[0181] An image processing module, used for obtaining a frame difference probability grammar dictionary and an attribute matrix of each frame image based on a grid range, a grid number, a background frame difference grayscale difference map, a background frame difference binary difference map, a sequence frame difference grayscale difference map, and a sequence frame difference binary difference map;

[0182] An information value acquisition module is used to bring the attribute matrix into the frame difference probability grammar dictionary to obtain a probability matrix; and obtain the information value based on the probability matrix;

[0183] The result determination module is used to compare the information value with a set threshold value, and when the information value is greater than the set threshold value, it is determined to be an abnormal frame image.

[0184] Example 5

[0185] The embodiment of the present invention is based on a conference room scene video stream. Due to data sensitivity issues, desensitization processing is performed here and the original image is not displayed.

[0186] In the present invention, an additional processing step is performed on the sequence frame difference map: an edge mask map is generated, and the intermediate binary difference map is subtracted from the edge mask map, and all negative values ​​are set to 0 to obtain the sequence frame difference binary difference map. The visualized heat map after this operation is shown as follows: Figure 5 As shown in the figure, the heat map visualized without this operation is as follows Figure 6 As shown, the comparison results are as follows: the heat will not be too concentrated, and will be more evenly distributed throughout the image, which can weaken the impact caused by slight jitter.

[0187] like Figure 7-Figure 8 As shown, it can be seen that normal samples and abnormal samples can achieve a certain accuracy under the information discriminator, because there is a significant difference in the probability density functions of the two. This difference shows that by analyzing the information distribution of the samples, we can effectively distinguish normal behavior from abnormal behavior.

[0188] The difference in the probability density function between normal samples and abnormal samples provides a reliable quantitative indicator for anomaly detection. In practical applications, this means that this difference can be used to design a discriminator or classifier to automatically identify abnormal events in surveillance videos or other monitoring data. This method based on the frequency distribution of information can not only improve the accuracy of anomaly detection, but also reduce false positives and false negatives to a certain extent.

[0189] In addition, the significant difference in the probability density function between normal samples and abnormal samples also means that by selecting an appropriate threshold, normal behavior can be effectively separated from abnormal behavior. This provides an important theoretical basis and practical guidance for designing efficient anomaly detection systems, especially in security-sensitive situations that require fast and accurate responses.

[0190] Based on the analysis of experimental data, we can conclude that even the simplest information discriminator can achieve a preliminary distinction between normal and abnormal samples. This finding has important practical application value, indicating that even in resource-constrained or simple conditions, effective anomaly detection mechanisms can still be deployed.

[0191] By using end-to-end neural network models or other advanced machine learning models, we have reason to believe that more accurate classification results can be achieved with a limited number of samples. These models can better capture the subtle differences between normal and abnormal by learning complex patterns and relationships in sample data, thereby improving classification accuracy and robustness.

[0192] In summary, the significant distribution difference between normal samples and abnormal samples under the information volume discriminator not only verifies the effectiveness of information volume analysis in anomaly detection, but also opens up new avenues for further research and application.

[0193] It can be seen from the above technical solutions that the present invention discloses a scene image anomaly detection method and system based on a frame difference probability grammar dictionary, which captures dynamic changes in image sequences through inter-frame difference technology and uses a probability grammar model to characterize and analyze these changes, thereby identifying abnormal behaviors, and has the following beneficial effects:

[0194] 1. The inter-frame difference technology combined with the probabilistic grammar model can be applied to various scene anomaly detection fields, including but not limited to conference rooms, classrooms, monitoring, etc., and the proposed probabilistic representation method of the image has positive significance for subsequent tasks.

[0195] 2. The present invention can perform efficient rough screening of abnormality identification without labeling or with only simple labeling, which significantly reduces the workload and complexity of manual labeling.

[0196] 3. Different from the traditional method that requires precise definition of abnormal categories, the present invention does not need to make excessive definitions of abnormal categories during the abnormality identification process, thereby improving the applicability and flexibility of the method.

[0197] 4. The hardware configuration requirements are not high and the deployment cost is low.

[0198] 5. The method of the present invention is not only applicable to anomaly detection, but also has positive significance for subsequent image processing and analysis tasks, thus enhancing the application value of the method.

[0199] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0200] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A scene image anomaly detection method based on frame difference probability grammar dictionary, It is characterized in that include: Obtaining a video stream of a scene to be detected and its corresponding background image; Based on the background image and the scene video stream, a background frame difference grayscale difference map, a background frame difference binary difference map, a sequence frame difference grayscale difference map and a sequence frame difference binary difference map are obtained; Set the grid range and grid number; Obtaining a frame difference probability grammar dictionary and an attribute matrix of each frame image in the scene video stream based on the grid range, the number of grids, the background frame difference grayscale difference map, the background frame difference binary difference map, the sequence frame difference grayscale difference map and the sequence frame difference binary difference map; Substituting the attribute matrix into the frame difference probability grammar dictionary to obtain a probability matrix; Obtaining an information value based on the probability matrix; The information amount value is compared with a set threshold value, and when it is greater than the set threshold value, it is determined to be an abnormal frame image.

2. The scene image anomaly detection method based on the frame difference probability grammar dictionary according to claim 1, It is characterized in that The background image acquisition method is: Obtain a video stream of an empty scene and intercept a preset time period to obtain an intercepted video stream; Preprocessing is performed based on the intercepted video stream to obtain the background image.

3. The scene image anomaly detection method based on the frame difference probability grammar dictionary according to claim 2, It is characterized in that The pre-processing comprises: Calculate the pixel values ​​of each channel of all frame images in the intercepted video stream; The background image is obtained based on all the pixel values.

4. The scene image anomaly detection method based on a frame difference probability grammar dictionary according to claim 1, It is characterized in that Obtaining the background frame difference grayscale difference image and the background frame difference binary difference image specifically includes: Based on the current frame image in the scene video stream and the background image, the absolute value of the difference is calculated pixel by pixel and channel by channel to obtain a background difference map; Converting the background difference map into the background frame difference grayscale difference map based on the background difference map; Setting a first threshold to perform binarization processing on the background grayscale difference image to obtain a first binarized difference image; An expansion and corrosion operation is performed based on the first binary difference image to obtain the background frame difference binary difference image.

5. The scene image anomaly detection method based on the frame difference probability grammar dictionary according to claim 4, It is characterized in that Obtaining the sequence frame difference grayscale difference image and the sequence frame difference binary difference image specifically includes: Based on the absolute value calculation of the difference between the current frame image and the previous interval frame image in the scene video stream, pixel by pixel and channel by channel, a sequence difference map is obtained; Converting the sequence difference map into an intermediate grayscale difference map; Performing edge detection based on the intermediate grayscale difference image to obtain an edge mask image; Performing binarization processing based on the intermediate grayscale difference image to obtain a second binarized difference image; Performing an expansion and corrosion operation based on the second binary difference image to obtain the intermediate binary difference image; Based on the edge mask image, the intermediate binary difference image is subtracted, and all negative values ​​are set to 0 to obtain the sequence frame difference binary difference image; The grayscale value of the position marked by the edge mask image in the intermediate grayscale difference image is set to 0 to obtain the sequence frame difference grayscale difference image.

6. The scene image anomaly detection method based on a frame difference probability grammar dictionary according to claim 1, It is characterized in that The method for obtaining the frame difference probability grammar dictionary is: Based on the grid range, the background frame difference grayscale difference map, the background frame difference binary difference map, the sequence frame difference grayscale difference map and the sequence frame difference binary difference map, four attribute values ​​of the number of background frame difference dynamic blocks, the background frame difference activation strength, the number of sequence frame difference dynamic blocks and the sequence frame difference activation strength are obtained; Based on the four attribute values ​​in each grid in all frame images in the scene video stream, four one-dimensional vectors w are combined i , i=1,2,3,4; Based on all the one-dimensional vectors w i Perform statistical modeling, and obtain a Gaussian mixture model corresponding to the four attributes for each grid; The frame difference probability grammar dictionary is obtained based on the Gaussian mixture models corresponding to the four attributes in all grids.

7. The scene image anomaly detection method based on a frame difference probability grammar dictionary according to claim 6, It is characterized in that The method for obtaining the number of background frame difference dynamic blocks and the background frame difference activation strength is: Counting the number of all non-zero values ​​within the grid range in the background frame difference binary difference image as the number of background frame difference dynamic blocks; Counting the average grayscale intensity of all non-zero pixels within the grid range in the background frame difference binary difference image as the background frame difference activation intensity; The average grayscale intensity is obtained based on the determined non-zero pixel points in the background frame difference grayscale difference map.

8. The scene image anomaly detection method based on the frame difference probability grammar dictionary according to claim 7, It is characterized in that The method for obtaining the number of the sequence frame difference dynamic blocks and the sequence frame difference activation strength is: Counting the number of all non-zero values ​​within the grid range in the sequence frame difference binary difference image as the number of sequence frame difference dynamic blocks; Counting the average grayscale intensity of all non-zero pixels within the grid range in the sequence frame difference binary difference image as the sequence frame difference activation intensity; The average grayscale intensity is obtained based on the determined non-zero pixel points in the sequence frame difference grayscale difference map.

9. The scene image anomaly detection method based on the frame difference probability grammar dictionary according to claim 8, It is characterized in that The attribute matrix acquisition method is: Based on the grid range, the background frame difference grayscale difference map, the background frame difference binary difference map, the sequence frame difference grayscale difference map and the sequence frame difference binary difference map, four attribute values ​​of the background frame difference dynamic block number, the background frame difference activation strength, the sequence frame difference dynamic block number and the sequence frame difference activation strength are obtained accordingly; Based on the four attribute values ​​and the number of grids n, an attribute matrix of size 4*n is obtained, where 4 represents four attribute values.

10. The scene image anomaly detection method based on the frame difference probability grammar dictionary according to claim 9, It is characterized in that The method for obtaining the information value is: Converting the probability matrix into an information matrix; Set the first weight of each mesh; Calculating the weighted sum of each information vector based on the first dimension axis direction of the information matrix and the first weight to obtain the information vectors corresponding to the four attribute values; Setting the acquisition probability of the background frame difference dynamic block quantity and the background frame difference activation intensity as a second weight; The number of the sequence frame difference dynamic blocks and the acquisition probability of the sequence frame difference activation intensity are both set as a third weight; The information value is obtained by performing weighted summation based on the second weight, the third weight, and the information vectors corresponding to the four attribute values.

11. A scene image anomaly detection system based on frame difference probability grammar dictionary, It is characterized in that include: Data acquisition module, difference map acquisition module, grid setting module, image processing module, information value acquisition module and result determination module; The data acquisition module is used to acquire the scene video stream to be detected and its corresponding background image; The difference image acquisition module is used to obtain a background frame difference grayscale difference image, a background frame difference binary difference image, a sequence frame difference grayscale difference image and a sequence frame difference binary difference image based on the background image and the scene video stream; The grid setting module is used to set the grid range and the number of grids; The image processing module is used to obtain a frame difference probability grammar dictionary and an attribute matrix of each frame image based on the grid range, the number of grids, the background frame difference grayscale difference map, the background frame difference binary difference map, the sequence frame difference grayscale difference map and the sequence frame difference binary difference map; The information value acquisition module is used to bring the attribute matrix into the frame difference probability grammar dictionary to obtain a probability matrix; Obtaining an information value based on the probability matrix; The result determination module is used to compare the information amount value with a set threshold value, and when the information amount value is greater than the set threshold value, determine it as an abnormal frame image.

Citation Information

Cited By

  • Communication method and system for time-delay-free ultra-high-definition image transmission IP core software

    CN119325015A

  • Communication method and system for delay-free ultra-high-definition image transmission IP core software

    CN119325015B