Video abnormal behavior detection method and system based on image recognition

Abstract

The application discloses a video abnormal behavior detection method and system based on image recognition, and relates to the field of image recognition, and the method comprises the following steps: collecting infrared monitoring video in a target scene, performing abnormal behavior recognition, and obtaining an initial abnormal behavior recognition result; when the target behavior recognition result is yes, extracting a feature temperature point in the infrared image sequence and extracting pixel points in a nearby preset window to obtain a feature window; performing similarity analysis indexing in the infrared image sequence according to the feature window to obtain an indexed feature window, and analyzing to obtain a motion cycle; analyzing the feature temperature point and the motion cycle to obtain a temperature behavior similarity and a motion behavior similarity, calculating a behavior similarity, judging whether the behavior similarity is greater than a behavior similarity threshold value, and obtaining an abnormal behavior detection result. The video abnormal behavior detection method based on image recognition solves the problems of low detection efficiency and poor accuracy in abnormal behavior detection.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and more specifically to a method and system for detecting abnormal video behavior based on image recognition. Background Technology

[0002] Video abnormal behavior detection refers to a detection method that automatically detects and locates abnormal behavior by utilizing the differences between normal and abnormal behavior feature representations. It is of great significance in the security field.

[0003] Existing video abnormal behavior detection methods rely on a single approach, simply identifying behavioral actions. This results in high false alarm and false negative rates, large errors, low detection efficiency, poor accuracy, and ultimately, unsatisfactory detection performance. Summary of the Invention

[0004] This application provides a video abnormal behavior detection method and system based on image recognition, which is used to address the problems of low detection efficiency, poor accuracy, and poor detection and judgment methods in the prior art, resulting in poor detection effect. By analyzing temperature characteristics and motion cycle, the method improves the accuracy and reliability of abnormal behavior detection.

[0005] In view of the above problems, this application provides a video abnormal behavior detection method and system based on image recognition.

[0006] In a first aspect, this application provides a video abnormal behavior detection method based on image recognition, the method comprising:

[0007] Infrared surveillance video of the target scene is collected, abnormal behavior is identified, and an initial abnormal behavior identification result is obtained. The infrared surveillance video includes an infrared image sequence, and the target behavior identification result includes yes or no.

[0008] When the target behavior recognition result is yes, extract the feature temperature points in the infrared image sequence, and extract the pixel points in a preset window near the feature temperature points to obtain the feature window;

[0009] Based on the feature window, a similarity analysis index is performed within the infrared image sequence to obtain the index feature window, and the motion cycle is obtained through analysis.

[0010] Based on the characteristic temperature points and motion cycles, temperature behavior similarity and motion behavior similarity are analyzed and obtained. Behavior similarity is calculated and it is determined whether it is greater than the behavior similarity threshold to obtain abnormal behavior detection results.

[0011] Secondly, this application provides a video abnormal behavior detection system based on image recognition, including:

[0012] An abnormal behavior recognition module is used to collect infrared surveillance video in a target scene, perform abnormal behavior recognition, and obtain an initial abnormal behavior recognition result. The infrared surveillance video includes an infrared image sequence, and the target behavior recognition result includes yes or no.

[0013] The feature window acquisition module is used to extract feature temperature points in the infrared image sequence when the target behavior recognition result is yes, and to extract pixel points in a preset window near the feature temperature points to obtain a feature window;

[0014] The similarity analysis index module is used to perform similarity analysis indexing within the infrared image sequence based on the feature window, obtain the index feature window, and analyze and obtain the motion cycle;

[0015] The abnormal behavior detection module is used to analyze and obtain temperature behavior similarity and motion behavior similarity based on the characteristic temperature point and motion cycle, calculate the behavior similarity, determine whether it is greater than the behavior similarity threshold, and obtain the abnormal behavior detection result.

[0016] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0017] This application proposes a video abnormal behavior detection method and system based on image recognition. The abnormal behavior detection method uses an infrared camera to acquire high-quality infrared surveillance video, performs abnormal behavior identification through an initial abnormal behavior recognizer, and when the target behavior identification result is positive, acquires and extracts pixels within a preset window near the feature temperature point to obtain a feature window, performs similarity analysis indexing, and calculates behavior similarity to finally obtain the abnormal behavior detection result. Simultaneously, the video abnormal behavior detection system based on image recognition of this invention constructs an initial abnormal behavior recognizer through an abnormal behavior identification module to perform initial abnormal behavior identification, and obtains the abnormal behavior detection result through feature window acquisition, similarity analysis indexing, and behavior similarity calculation. It constructs a detection system with image acquisition, image recognition, initial abnormal behavior identification, abnormal behavior detection analysis, and abnormal behavior detection result output. Compared with previous single-method image behavior recognition, this application can accurately and efficiently detect abnormal behavior in images, solving the problems of low detection efficiency and high false alarm rate. It avoids the problem of high error rate in abnormal behavior detection due to a single detection method, improves the detection effect, and achieves accurate and efficient video abnormal behavior detection. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating a video anomaly detection method based on image recognition.

[0020] Figure 2 This is a schematic diagram of a video abnormal behavior detection system based on image recognition.

[0021] In the attached diagram, the labels represent the following: Abnormal behavior recognition module 11; Feature window acquisition module 12; Similarity analysis index module 13; Abnormal behavior detection module 14. Detailed Implementation

[0022] This application provides a video abnormal behavior detection method and system based on image recognition, which addresses the problems in existing technologies where detection is greatly affected by lighting or relies solely on motion features, resulting in low false alarm efficiency and consequently low detection accuracy, thus affecting the detection effect of video abnormal behavior detection.

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0024] It should be noted that the terms "comprising" and "having" are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0025] The present invention will now be described in detail with reference to the accompanying drawings.

[0026] Example 1, as Figure 1 As shown, this application discloses a video abnormal behavior detection method based on image recognition, the method comprising:

[0027] S10: Acquire infrared surveillance video within the target scene, perform abnormal behavior identification, and obtain initial abnormal behavior identification results, wherein the infrared surveillance video includes an infrared image sequence, and the target behavior identification results include yes or no;

[0028] In this embodiment of the application, for example, video is captured in the target scene by an infrared camera, and behavior recognition is performed by an initial recognizer to determine whether smoking behavior may exist, and the result of behavior recognition is output as either yes or no. However, for actions similar to smoking, there may be errors in behavior recognition. For example, when a user raises his hand to drink water, he may be mistakenly identified as having smoking behavior.

[0029] Step S10 in the method provided in this application embodiment includes:

[0030] Infrared surveillance video of the target scene is acquired, wherein the infrared surveillance video includes a sequence of infrared images;

[0031] Obtain the initial abnormal behavior identifier;

[0032] The infrared image sequence is input into the initial abnormal behavior recognizer, and the recognition output obtains the initial abnormal behavior recognition result, wherein the target behavior recognition result includes yes or no.

[0033] Specifically, infrared cameras are used to collect infrared surveillance videos of the target scene. The resulting infrared surveillance videos include infrared image sequences, which are a series of continuous images captured by infrared sensors. These sequences are used for target detection and other applications, such as obtaining the LLVIP Dataset dataset.

[0034] Furthermore, an initial abnormal behavior identifier is obtained. This identifier can be constructed using a convolutional neural network model to classify the acquired infrared image sequences.

[0035] Furthermore, the obtained infrared image sequences are input into an initial abnormal behavior identifier. This identifier classifies and identifies the infrared image sequences, outputting an initial abnormal behavior identification result, where the target behavior identification result includes "yes" or "no". A set of infrared image sequences is input, and the output of the initial abnormal behavior identifier is analyzed. A final result is output through similarity testing. Infrared image sequences with an output of "no" from the initial abnormal behavior identifier are directly excluded, while those with an output of "yes" are further verified.

[0036] For example, a sequence of infrared images of men smoking in a bus stop (e.g., 30 frames per second) is input into the initial abnormal behavior recognizer, and the output result is "yes". A sequence of infrared images of men standing in a bus stop is input, and the output result is "no". The infrared image sequence with the output result of "yes" will be verified and analyzed.

[0037] In this embodiment of the application, step S10, obtaining the initial abnormal behavior identifier, includes:

[0038] Based on historical surveillance video recordings of abnormal behavior, a set of sample infrared image sequences was collected.

[0039] For each sample infrared image sequence, label whether abnormal behavior occurs to obtain the initial set of abnormal behavior identification results for the sample.

[0040] An initial abnormal behavior recognizer is constructed based on a convolutional neural network. The initial abnormal behavior recognizer is trained under supervision using the set of sample infrared image sequences and the set of sample initial abnormal behavior recognition results. The training is completed after the test converges, and the initial abnormal behavior recognizer is obtained.

[0041] Specifically, a set of sample infrared image sequences is collected using video recordings of abnormal behavior over a historical period. To ensure the effectiveness of subsequent analysis and processing, the collected sample infrared image sequence set should be valid, ensuring that the monitored object exists in the infrared image sequence. Infrared image sequences without monitored objects are removed, thereby ensuring the quality and validity of the samples.

[0042] For example, if no monitored object exists in the acquired infrared image sequence, it will be removed and not used as a sample infrared image sequence.

[0043] Furthermore, using annotation tools, either manually or with annotation software, each sample infrared image sequence is labeled to indicate whether abnormal behavior has occurred. Different colored borders can be used to label "yes" and "no" respectively. The infrared image sequences labeled "yes" are then integrated to obtain the initial abnormal behavior identification result set for the samples.

[0044] Furthermore, an initial abnormal behavior recognizer is constructed using a convolutional neural network (CNN) in machine learning algorithms. A CNN is a deep learning model particularly adept at handling image-related tasks, such as image classification and object detection.

[0045] The initial abnormal behavior detector can be trained using the following technical path: Model Construction: It mainly consists of an input layer, convolutional layer, pooling layer, and output layer. The input layer divides the sample infrared image sequence set into two categories: "yes" and "no". The convolutional layer extracts sample features from the sample infrared image sequence set. The pooling layer reduces the spatial size of the data through downsampling, preserving the important features of the sample infrared image sequence set. The output layer maps the two categories of the sample infrared image sequence set one-to-one to the initial abnormal behavior recognition result set, summarizes the extracted features, and outputs the recognition result. Model Training: Using the sample infrared image sequence set and the initial abnormal behavior recognition result set as input, the Adam optimizer (initial learning rate 1e-4) is used. The MSE loss function is selected to calculate the error between the sample infrared image sequence set and the initial abnormal behavior recognition result set. Forward propagation is used to obtain the prediction result, and backpropagation is used to update the parameters. Performance is evaluated using a validation set after each training epoch to avoid overfitting. When the MSE loss decreases by <1e for 5 consecutive training epochs, the model is considered successful. -5 When the MSE loss on the validation set stabilizes below 0.01, the model is considered to have converged. Training is stopped and the final network parameters are saved to obtain the initial abnormal behavior recognizer.

[0046] For example, 1,000 infrared image sequences collected from abnormal behavior monitoring video recordings over the past year were input into an initial abnormal behavior recognizer for initial abnormal behavior identification. 300 of these sequences were marked "yes" and 700 were marked "no".

[0047] In this embodiment, a set of sample infrared image sequences extracted from historical abnormal behavior monitoring videos and a set of initial abnormal behavior identification results obtained through annotation are used as inputs. An initial abnormal behavior recognizer is constructed and trained using a convolutional neural network. This initial abnormal behavior recognizer can be input into the set of infrared image sequences and output the initial abnormal behavior identification results. The use of infrared cameras to collect infrared monitoring videos within the target scene ensures data reliability and high accuracy, avoiding interference from unreliable data sources that could affect the abnormal behavior identification and analysis. This efficient judgment of potential abnormal behaviors in the infrared monitoring videos and rapid acquisition of potential abnormal behavior identification results represents a high-speed identification but low-accuracy screening stage, aiming to reduce the amount of data required for subsequent complex calculations.

[0048] In this embodiment of the application, in order to avoid similar behaviors interfering with the identification of abnormal behaviors, the target behaviors that output "yes" need to be verified in order to improve the accuracy of abnormal behavior detection.

[0049] S20: When the target behavior recognition result is yes, extract the feature temperature points in the infrared image sequence, and extract the pixel points in the preset window near the feature temperature points to obtain the feature window;

[0050] In this embodiment, instead of selecting the highest temperature point, representative characteristic temperature points are selected by comparing their similarity with the average temperature parameters at the time of the abnormal behavior. Since infrared images can display temperature, the temperature point with the most similar temperature parameters to the temperature of a cigarette butt under the same acquisition conditions is extracted as the characteristic temperature point. After segmentation, image pixels within a window near the characteristic temperature point are extracted as a feature window; for example, 20×20 pixels around the feature temperature point are extracted as a feature window.

[0051] Step S20 in the method provided in this application embodiment includes:

[0052] Obtain the average temperature parameter when the abnormal behavior occurs;

[0053] When the target behavior recognition result is yes, temperature points are randomly selected iteratively within the infrared image sequence, the similarity with the temperature parameter is calculated, and the feature temperature point with the highest similarity is selected.

[0054] Extract the pixels within a preset window near the feature temperature point to obtain the feature window.

[0055] Specifically, based on historical infrared image sequences, an average temperature parameter is set for abnormal behavior. This average temperature parameter can be preset or calculated based on temperature points within the scene. The average temperature parameter is used to distinguish other temperatures, highlighting abnormal behavior.

[0056] For example, the average temperature parameter of the cigarette butt when smoking is set to 700°C.

[0057] Furthermore, within the infrared image sequence where the target behavior recognition result is "yes", temperature points are iteratively randomly selected, and their similarity to the average temperature parameter is calculated. During iterative random selection, first, a temperature point is randomly selected, and the similarity K between the temperature point and the average temperature parameter is calculated. Then, another temperature point is randomly selected, and the similarity W between the temperature point and the average temperature parameter is calculated. If K < W, the similarity K is removed, and the similarity W is retained. If K > W, the similarity K is retained, and one random selection is completed. In the next iteration, the similarity K or W is used to continue comparing with the similarity of the randomly selected temperature point and the average temperature parameter. This process of random selection and iteration is repeated multiple times until the feature temperature point with the highest similarity is obtained. Here, similarity = [1 - (randomly selected temperature point - average temperature parameter) / average temperature parameter]. The larger the value, the more similar the temperature point, and the greater the probability that it will be used as a feature temperature point.

[0058] For example, by randomly selecting and iterating 1000 times, the feature temperature point with the highest similarity is 705℃.

[0059] Furthermore, taking the feature temperature point with the highest similarity as the center, pixels within a preset window near the feature temperature point are extracted from the infrared image corresponding to the feature window to obtain the feature window.

[0060] For example, a rectangular area of ​​20×20 pixels around 705°C is selected as the feature window.

[0061] In this embodiment, when the target behavior recognition result is positive, feature temperature points are extracted from the infrared image sequence, and then pixels within a preset window near the feature temperature points are selected as the feature window. The average temperature parameter of the abnormal behavior is used for calculation to improve performance stability, and an iterative random selection strategy avoids obtaining only the feature temperature points with the highest local similarity, ensuring the accuracy of the selection.

[0062] S30: Based on the feature window, perform similarity analysis indexing within the infrared image sequence to obtain the index feature window and analyze to obtain the motion cycle;

[0063] In this embodiment, a feature window is obtained from the infrared image sequence where the target behavior recognition result is positive. However, since the initial abnormal behavior recognizer may misidentify actions such as raising a hand to drink water as abnormal behavior when recognizing smoking, it is necessary to analyze the user's motion cycle. Users exhibit repetitive actions while smoking, such as the repetitive actions of letting their hand fall naturally and picking up a cigarette. Randomly divided windows are extracted from other infrared images, and the similarity to these feature windows is calculated to obtain the most similar window and its corresponding infrared image timestamp. The time interval between the infrared image timestamp corresponding to the feature window and the timetamp is calculated as the motion cycle.

[0064] In this embodiment of the application, step S30 includes:

[0065] The infrared image containing the feature window is marked as the reference infrared image;

[0066] Within other infrared images, the window to be matched is traversed and divided, and the similarity with the feature window is calculated to obtain the window similarity.

[0067] The window with the highest similarity to be matched is selected as the index feature window.

[0068] The infrared image containing the index feature window is marked as the index infrared image, and the time interval between the reference infrared image and the index infrared image is calculated as the motion period.

[0069] Specifically, the infrared image containing the feature window is marked as the reference infrared image, which is used for comparison with randomly divided windows in other infrared images.

[0070] Furthermore, all pixels in the other infrared images are divided into several matching windows through a moving window traversal. The similarity between each matching window and the feature window is calculated, for example, the similarity between the pixels within the matching window and the feature window. A higher similarity value indicates a greater likelihood that the user's smoking behavior is verified as abnormal.

[0071] At the same time, the numerical values ​​of similarity are compared, and several windows to be matched are filtered to select the window with the highest similarity as an index feature window.

[0072] For example, a set of similarities is obtained as: [20%, 50%, 80%, 30%, 60%]. The window to be matched corresponding to the largest similarity of 80% is selected as an index feature window, resulting in a set of similarities as: [80%, 85%, 90%, 90%, 95%]. The window to be matched with a similarity of 95% is selected as the index feature window.

[0073] In addition, the infrared images corresponding to the selected index feature windows are marked as index infrared images. The time interval between the baseline infrared image and the index infrared image is calculated, and the obtained time interval is used as the motion cycle. The motion cycle can reflect the periodicity of behavior and can be used as a basis for judging repetitive abnormal behaviors such as smoking, thus identifying user behavior. The time interval is the difference between the acquisition time of the baseline infrared image and the acquisition time of the index infrared image.

[0074] For example, in the acquired infrared image sequence, the reference infrared image is acquired at 1 second, and the index infrared image corresponding to the index feature window with a similarity of 95% is acquired at 5 seconds. The time interval is (5-1)=4 seconds, that is, the motion cycle is 4 seconds.

[0075] In this embodiment of the application, step S30, which involves traversing and dividing the window to be matched within other infrared images, calculating the similarity with the feature window, and obtaining the window similarity, includes:

[0076] The matching window is divided by traversing other infrared images.

[0077] Calculate the mean similarity of pixel values ​​at the same pixel locations within each window to be matched and the feature window, and use this as the window similarity.

[0078] Specifically, all pixels within the other infrared images are divided to obtain several matching windows. Simultaneously, the similarity of pixel values ​​at the same pixel locations between the matching windows and the feature windows is calculated. The mean squared error (MSE) function can be used to calculate the closeness of the corresponding pixel locations between the matching windows and the feature windows. A higher MSE value indicates a higher similarity between the pixel values ​​at the same pixel locations between the matching windows and the feature windows. The mean squared error between the matching windows and the feature windows is defined as:

[0079]

[0080] Where MSE is the mean squared error, mn is the product of the number of pixels, which represents the area of ​​the image, m is the number of pixels in the image width, n is the number of pixels in the image height, I is the window to be matched, K is the feature window, I(i,j) is the pixel coordinate of the window to be matched, K(i,j) is the coordinate of the same pixel in the feature window, i is any point in m, and j is any point in n.

[0081] For example, the surrounding 20×20 pixels are extracted as a feature window, mn is 400, and a window similarity of 0.8 is calculated.

[0082] In this embodiment, the similarity between the window to be matched and the feature window is calculated using MSE (Mean Search Estimate), and a similarity analysis index is performed within the infrared image sequence to obtain the indexed feature window. The time interval between the baseline infrared image and the indexed infrared image is calculated to obtain the motion cycle, and abnormal user behavior is judged based on the motion cycle. Precise image recognition at the pixel level improves the accuracy of abnormal behavior detection.

[0083] S40: Based on the characteristic temperature point and motion cycle, analyze and obtain temperature behavior similarity and motion behavior similarity, calculate behavior similarity, determine whether it is greater than the behavior similarity threshold, and obtain abnormal behavior detection results.

[0084] In this embodiment, based on a characteristic temperature point and a motion cycle, the similarity between the temperature of the characteristic temperature point and the average temperature parameter formed by the smoking behavior within the image is calculated and used as the temperature behavior similarity. Similarly, the similarity between the motion cycle and the average time cycle of repetitive smoking actions by multiple users is calculated and used as the motion behavior similarity. The average of the temperature behavior similarity and the motion behavior similarity is used as the behavior similarity. Whether the similarity is greater than a behavior similarity threshold is determined to obtain the abnormal behavior detection result.

[0085] In this embodiment of the application, step S40 includes:

[0086] The pixel points of the feature temperature point and the average temperature parameter when the abnormal behavior occurs are obtained as the temperature behavior similarity.

[0087] Obtain the average motion cycle of abnormal behavior;

[0088] Calculate the similarity between the stated motion cycle and the average motion cycle, and use this as the motion behavior similarity.

[0089] Based on the temperature behavior similarity and motion behavior similarity, the behavior similarity is calculated.

[0090] Determine whether the behavior similarity is greater than the behavior similarity threshold. If yes, obtain a positive abnormal behavior detection result; otherwise, obtain a negative abnormal behavior detection result.

[0091] Specifically, the similarity between the characteristic temperature point and the average temperature parameter at the time of the abnormal behavior is obtained, and the similarity between the characteristic temperature point and the average temperature parameter at the time of the abnormal behavior is calculated as the temperature behavior similarity. Similarity = [1 - (characteristic temperature point - average temperature parameter at the time of the abnormal behavior) / average temperature parameter at the time of the abnormal behavior], where the larger the value, the closer the characteristic temperature point and the average temperature parameter at the time of the abnormal behavior are, and the greater the probability that the abnormal behavior detection result is "yes".

[0092] For example, the feature temperature point is 705℃, and the average temperature parameter when the abnormal behavior occurs is 700℃. The similarity between the feature temperature point and the pixel point of the average temperature parameter when the abnormal behavior occurs is approximately [1-(705-700) / 700]≈0.992. The value is relatively large, which means that the probability of the abnormal behavior detection result being "yes" is relatively high.

[0093] Furthermore, by using methods such as database queries, the average movement cycle of abnormal behavior is obtained, and the similarity between the movement cycle and the average movement cycle is calculated as the movement behavior similarity. The movement behavior similarity is calculated as [1 - (movement cycle - average movement cycle) / average movement cycle]. The larger the value, the closer the user's movement behavior is to the movement behavior of abnormal behavior in the database.

[0094] For example, the movement cycle is 4s, the average movement cycle is 3.5s, and the similarity is [1-(4-3.5) / 3.5]≈0.857, which is a relatively large value, meaning that the probability of the abnormal behavior detection result being "yes" is relatively high.

[0095] Furthermore, the average of temperature behavior similarity and motion behavior similarity is used as the behavior similarity. Behavior similarity = [(temperature behavior similarity + motion behavior similarity) / 2]. Temperature behavior similarity and motion behavior similarity directly affect the magnitude of behavior similarity; the higher the behavior similarity, the greater the probability that the abnormal behavior detection output is "yes". An abnormal behavior detection result is obtained based on the behavior similarity. A corresponding behavior similarity threshold is set. If the behavior similarity is greater than the threshold, the abnormal behavior detection result is output as "yes"; if the behavior similarity is less than the threshold, a pending abnormal behavior detection result is obtained, and then staff are notified for manual review.

[0096] For example, temperature behavior similarity ≈ 0.992, motion behavior similarity ≈ 0.857, and behavior similarity = [(0.992 + 0.857) / 2] ≈ 0.924. If the behavior similarity threshold is 0.75, and 0.924 > 0.75, it means that the user's behavior in the infrared image sequence within the target scene has passed abnormal behavior detection, and the abnormal behavior detection result is output as "yes".

[0097] In this embodiment, temperature behavior similarity and motion behavior similarity are obtained by using characteristic temperature points and motion cycles. Behavioral similarity is then calculated, and finally, the abnormal behavior detection result is judged based on the behavioral similarity and a behavioral similarity threshold. Compared to existing technologies, this embodiment verifies the initial recognition result of the initial abnormal behavior recognizer through behavioral similarity, avoiding a single verification method that leads to a high false alarm rate in abnormal behavior recognition. Through behavioral similarity analysis, more accurate abnormal behavior detection results are obtained.

[0098] The embodiments of this application, through the specific implementation methods described above, achieve the following technical effects:

[0099] In this embodiment, firstly, the initial abnormal behavior recognition result set is obtained by extracting sample infrared image sequences from historical abnormal behavior monitoring videos and using them as input. An initial abnormal behavior recognizer is then constructed and trained using a convolutional neural network. This initial abnormal behavior recognizer can be input into the infrared image sequence set and output the initial abnormal behavior recognition result. The use of an infrared camera to collect infrared monitoring videos within the target scene ensures data reliability and high accuracy, avoiding interference from unreliable data sources. Efficiently judging potential abnormal behaviors in the infrared monitoring videos and quickly obtaining possible abnormal behavior recognition results represents a high-speed but less accurate screening stage, aiming to reduce the amount of data required for subsequent complex calculations. Secondly, when the initial abnormal behavior recognition result is "yes," this embodiment extracts feature temperature points within the infrared image sequence and then extracts pixels within a preset window near the feature temperature points as a feature window. The average temperature parameter of abnormal behavior is used to improve performance stability. An iterative random selection strategy avoids obtaining only the feature temperature points with the highest local similarity, ensuring selection accuracy. Next, the similarity between the matching window and the feature window is calculated using MSE (Mean Separation of Equations), and similarity analysis indexing is performed within the infrared image sequence to obtain the indexed feature window. The time interval between the baseline infrared image and the indexed infrared image is calculated to obtain the motion cycle, which is used to identify abnormal user behavior. Precise image recognition at the pixel level improves the accuracy of abnormal behavior detection. Finally, temperature behavior similarity and motion behavior similarity are calculated using feature temperature points and motion cycles to obtain behavior similarity. The final judgment of abnormal behavior detection results is based on behavior similarity and a behavior similarity threshold.

[0100] Compared to existing technologies, this application's embodiments verify the initial recognition results of the initial abnormal behavior recognizer through behavioral similarity, reducing the amount of data required for subsequent complex calculations and avoiding the problem of high false alarm rates in abnormal behavior recognition caused by a single verification method. By performing precise image recognition at the pixel level, the accuracy of abnormal behavior detection is improved. Through behavioral similarity analysis, more accurate abnormal behavior detection results are obtained. Simultaneously, by acquiring infrared image sequences using an infrared detection device, it becomes applicable to complex environments, improving the adaptability of image recognition for video abnormal behavior detection.

[0101] Example 2, as Figure 2 As shown, based on the same inventive concept as the method provided in Embodiment 1, this embodiment of the invention also provides a video abnormal behavior detection system based on image recognition, including:

[0102] The abnormal behavior recognition module 11 is used to collect infrared monitoring video in the target scene, perform abnormal behavior recognition, and obtain an initial abnormal behavior recognition result. The infrared monitoring video includes an infrared image sequence, and the target behavior recognition result includes yes or no.

[0103] The feature window acquisition module 12 is used to extract feature temperature points in the infrared image sequence and extract pixel points in a preset window near the feature temperature points when the target behavior recognition result is yes, so as to obtain a feature window.

[0104] The similarity analysis index module 13 is used to perform similarity analysis indexing within the infrared image sequence based on the feature window, obtain the index feature window, and analyze and obtain the motion cycle.

[0105] The abnormal behavior detection module 14 is used to analyze and obtain temperature behavior similarity and motion behavior similarity based on the characteristic temperature point and motion cycle, calculate behavior similarity, determine whether it is greater than the behavior similarity threshold, and obtain abnormal behavior detection results.

[0106] In one embodiment, the abnormal behavior identification module 11 is used for:

[0107] Infrared surveillance video of the target scene is acquired, wherein the infrared surveillance video includes a sequence of infrared images;

[0108] Obtain the initial abnormal behavior identifier;

[0109] The infrared image sequence is input into the initial abnormal behavior recognizer, and the recognition output obtains the initial abnormal behavior recognition result, wherein the target behavior recognition result includes yes or no.

[0110] The acquisition of the initial abnormal behavior identifier includes:

[0111] Based on historical surveillance video recordings of abnormal behavior, a set of sample infrared image sequences was collected.

[0112] For each sample infrared image sequence, label whether abnormal behavior occurs to obtain the initial set of abnormal behavior identification results for the sample.

[0113] An initial abnormal behavior recognizer is constructed based on a convolutional neural network. The initial abnormal behavior recognizer is trained under supervision using the set of sample infrared image sequences and the set of sample initial abnormal behavior recognition results. The training is completed after the test converges, and the initial abnormal behavior recognizer is obtained.

[0114] In one embodiment, the feature window acquisition module 12 is used for:

[0115] Obtain the average temperature parameter when the abnormal behavior occurs;

[0116] When the target behavior recognition result is yes, temperature points are randomly selected iteratively within the infrared image sequence, the similarity with the temperature parameter is calculated, and the feature temperature point with the highest similarity is selected.

[0117] Extract the pixels within a preset window near the feature temperature point to obtain the feature window.

[0118] In one embodiment, the similarity analysis index module 13 is used for:

[0119] The infrared image containing the feature window is marked as the reference infrared image;

[0120] Within other infrared images, the window to be matched is traversed and divided, and the similarity with the feature window is calculated to obtain the window similarity.

[0121] The window with the highest similarity to be matched is selected as the index feature window.

[0122] The infrared image containing the index feature window is marked as the index infrared image, and the time interval between the reference infrared image and the index infrared image is calculated as the motion period.

[0123] The step of traversing and dividing the window to be matched within other infrared images, calculating the similarity with the feature window, and obtaining the window similarity includes:

[0124] The matching window is divided by traversing other infrared images.

[0125] Calculate the mean similarity of pixel values ​​at the same pixel locations within each window to be matched and the feature window, and use this as the window similarity.

[0126] In one embodiment, the abnormal behavior detection module 14 is used to: obtain the pixel points of the feature temperature point and the average temperature parameter when the abnormal behavior occurs, as the temperature behavior similarity;

[0127] Obtain the average motion cycle of abnormal behavior;

[0128] Calculate the similarity between the stated motion cycle and the average motion cycle, and use this as the motion behavior similarity.

[0129] Based on the temperature behavior similarity and motion behavior similarity, the behavior similarity is calculated.

[0130] Determine whether the behavior similarity is greater than the behavior similarity threshold. If yes, obtain a positive abnormal behavior detection result; otherwise, obtain a negative abnormal behavior detection result.

[0131] This application first utilizes an abnormal behavior recognition module to construct an initial abnormal behavior recognizer for initial abnormal behavior recognition. The initial abnormal behavior recognition result is used as the basis for analyzing the abnormal behavior detection result. Second, through a feature window acquisition module, when the target behavior recognition result is positive, feature temperature points and pixels within a preset window near the feature temperature points are extracted from the infrared image sequence to obtain a feature window. Third, through a similarity analysis indexing module, the feature window is used to perform similarity analysis indexing within the infrared image sequence to obtain an indexed feature window, thus obtaining the motion cycle of the abnormal behavior. The motion cycle provides data support for behavior similarity calculation. Finally, through an abnormal behavior detection module, the feature temperature points and motion cycle are used to analyze temperature behavior similarity and motion behavior similarity, perform similarity calculation, obtain behavior similarity, and determine whether it exceeds a behavior similarity threshold to obtain the abnormal behavior detection result. This constructs a detection system with image acquisition, image recognition, initial abnormal behavior recognition, abnormal behavior detection analysis, and abnormal behavior detection result output. Compared to previous single-method image behavior and action recognition, this application can accurately and efficiently detect abnormal behavior in images, avoiding the problem of high error rate in abnormal behavior detection caused by a single detection method, thus improving the detection effect and achieving efficient and accurate detection.

[0132] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0133] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A video abnormal behavior detection method based on image recognition, characterized in that, The method includes: Infrared surveillance video of the target scene is collected, abnormal behavior is identified, and an initial abnormal behavior identification result is obtained. The infrared surveillance video includes an infrared image sequence, and the target behavior identification result includes yes or no. When the target behavior recognition result is positive, feature temperature points are extracted from the infrared image sequence, and pixels within a preset window near the feature temperature points are extracted to obtain a feature window. This includes: obtaining the average temperature parameter when the abnormal behavior occurs; when the target behavior recognition result is positive, iteratively and randomly selecting temperature points within the infrared image sequence, calculating the similarity with the temperature parameter, and filtering to obtain the feature temperature point with the highest similarity; and extracting pixels within a preset window near the feature temperature points to obtain a feature window. Based on the feature window, similarity analysis is performed within the infrared image sequence to obtain an index feature window, and the motion cycle is obtained through analysis. This includes: marking the infrared image containing the feature window as a reference infrared image; traversing and dividing other infrared images into matching windows; calculating the average similarity of pixel values ​​at the same pixel position within each matching window and the feature window as the window similarity; filtering to obtain the matching window with the highest similarity as the index feature window; marking the infrared image containing the index feature window as an index infrared image; and calculating the time interval between the reference infrared image and the index infrared image as the motion cycle. Based on the characteristic temperature point and motion cycle, analyze and obtain temperature behavior similarity and motion behavior similarity, calculate behavior similarity, determine whether it is greater than the behavior similarity threshold, and obtain abnormal behavior detection results. This includes: obtaining the pixel points of the characteristic temperature point and the average temperature parameter when the abnormal behavior occurs as temperature behavior similarity; obtaining the average motion cycle of the abnormal behavior; calculating the similarity between the motion cycle and the average motion cycle as motion behavior similarity; calculating behavior similarity based on the temperature behavior similarity and motion behavior similarity; determining whether the behavior similarity is greater than the behavior similarity threshold. If yes, obtain a positive abnormal behavior detection result; otherwise, obtain a negative abnormal behavior detection result.

2. The video abnormal behavior detection method based on image recognition according to claim 1, characterized in that, Infrared surveillance video of the target scene is acquired, abnormal behavior is identified, and initial abnormal behavior identification results are obtained, including: Infrared surveillance video of the target scene is acquired, wherein the infrared surveillance video includes a sequence of infrared images; Obtain the initial abnormal behavior identifier; The infrared image sequence is input into the initial abnormal behavior recognizer, and the recognition output obtains the initial abnormal behavior recognition result, wherein the target behavior recognition result includes yes or no.

3. The video abnormal behavior detection method based on image recognition according to claim 2, characterized in that, Obtain the initial abnormal behavior identifier, including: Based on historical surveillance video recordings of abnormal behavior, a set of sample infrared image sequences was collected. For each sample infrared image sequence, label whether abnormal behavior occurs to obtain the initial set of abnormal behavior identification results for the sample. An initial abnormal behavior recognizer is constructed based on a convolutional neural network. The initial abnormal behavior recognizer is trained under supervision using the set of sample infrared image sequences and the set of sample initial abnormal behavior recognition results. The training is completed after the test converges, and the initial abnormal behavior recognizer is obtained.

4. A video abnormal behavior detection system based on image recognition, characterized in that, The system for implementing the method according to any one of claims 1-3, the system comprising: An abnormal behavior recognition module is used to collect infrared surveillance video in a target scene, perform abnormal behavior recognition, and obtain an initial abnormal behavior recognition result. The infrared surveillance video includes an infrared image sequence, and the target behavior recognition result includes yes or no. The feature window acquisition module is used to extract feature temperature points within the infrared image sequence and extract pixels within a preset window near the feature temperature points when the target behavior recognition result is positive, thereby obtaining a feature window. This includes: acquiring the average temperature parameter when the abnormal behavior occurs; iteratively and randomly selecting temperature points within the infrared image sequence when the target behavior recognition result is positive, calculating the similarity to the temperature parameter, and filtering to obtain the feature temperature point with the highest similarity; and extracting pixels within a preset window near the feature temperature points to obtain the feature window. The similarity analysis indexing module is used to perform similarity analysis indexing within the infrared image sequence based on the feature window, obtain an index feature window, and analyze and obtain a motion cycle. The module includes: marking the infrared image containing the feature window as a reference infrared image; traversing and dividing other infrared images into matching windows; calculating the average similarity of pixel values ​​at the same pixel location within each matching window and the feature window as the window similarity; filtering to obtain the matching window with the highest similarity as the index feature window; marking the infrared image containing the index feature window as an index infrared image; and calculating the time interval between the reference infrared image and the index infrared image as the motion cycle. An abnormal behavior detection module is used to analyze and obtain temperature behavior similarity and motion behavior similarity based on the feature temperature point and motion cycle, calculate the behavior similarity, determine whether it is greater than the behavior similarity threshold, and obtain an abnormal behavior detection result. This includes: acquiring the pixel points of the feature temperature point and the average temperature parameter when the abnormal behavior occurs, as the temperature behavior similarity; acquiring the average motion cycle of the abnormal behavior; calculating the similarity between the motion cycle and the average motion cycle, as the motion behavior similarity; calculating the behavior similarity based on the temperature behavior similarity and motion behavior similarity; and determining whether the behavior similarity is greater than the behavior similarity threshold. If yes, a positive abnormal behavior detection result is obtained; otherwise, a negative abnormal behavior detection result is obtained.

Images