Video abnormal behavior detection method and system based on image recognition

Abstract

The invention discloses a video abnormal behavior detection method and system based on image recognition, and relates to the field of image recognition, and the method comprises the steps: collecting an infrared monitoring video in a target scene, carrying out the abnormal behavior recognition, and obtaining an initial abnormal behavior recognition result; when the target behavior recognition result is yes, extracting feature temperature points in the infrared image sequence, and extracting pixel points in a nearby preset window to obtain a feature window; according to the feature window, similarity analysis indexing is carried out in the infrared image sequence, an index feature window is obtained, and a motion period is obtained through analysis; according to the characteristic temperature point and the motion period, analyzing to obtain a temperature behavior similarity and a motion behavior similarity, calculating to obtain a behavior similarity, judging whether the behavior similarity is greater than a behavior similarity threshold or not, and obtaining an abnormal behavior detection result. According to the video abnormal behavior detection method based on image recognition, the problems of low detection efficiency and poor accuracy in abnormal behavior detection are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image recognition, in particular to a video abnormal behavior detection method and system based on image recognition. BACKGROUND

[0002] Video abnormal behavior detection refers to a detection method for automatically detecting and positioning abnormal behavior by using the difference between normal and abnormal behavior characteristic representations, which is of great significance in the field of security and protection.

[0003] The existing video abnormal behavior detection method has a single detection and judgment mode, relies only on simple behavior action recognition, has high false positive rate and false negative rate for abnormal behavior, large error, low detection efficiency, poor accuracy, and poor detection effect. SUMMARY

[0004] The present application provides a video abnormal behavior detection method and system based on image recognition, which is used to solve the problems of low detection efficiency, poor accuracy, and single detection and judgment mode in the prior art, and improve the accuracy and reliability of abnormal behavior detection through temperature feature and motion cycle analysis.

[0005] In view of the above problems, the present application provides a video abnormal behavior detection method and system based on image recognition.

[0006] In a first aspect, the present application provides a video abnormal behavior detection method based on image recognition, which comprises: Collecting an infrared monitoring video in a target scene, performing abnormal behavior recognition, and obtaining an initial abnormal behavior recognition result, wherein the infrared monitoring video comprises an infrared image sequence, and the target behavior recognition result comprises yes or no; When the target behavior recognition result is yes, extracting a feature temperature point in the infrared image sequence, and extracting pixel points in a preset window near the feature temperature point to obtain a feature window; According to the feature window, performing similarity analysis indexing in the infrared image sequence to obtain an indexed feature window, and analyzing to obtain a motion cycle; According to the feature temperature point and the motion cycle, analyzing to obtain a temperature behavior similarity and a motion behavior similarity, calculating to obtain a behavior similarity, determining whether it is greater than a behavior similarity threshold, and obtaining an abnormal behavior detection result.

[0007] In a second aspect, the present application provides a video abnormal behavior detection system based on image recognition, which comprises: An abnormal behavior recognition module is configured to collect an infrared monitoring video in a target scene, perform abnormal behavior recognition, and obtain an initial abnormal behavior recognition result, wherein the infrared monitoring video comprises an infrared image sequence, and the target behavior recognition result comprises yes or no. a feature window acquisition module configured to extract a feature temperature point in the infrared image sequence and extract pixel points in a preset window near the feature temperature point to obtain a feature window when the target behavior recognition result is yes; a similarity analysis index module configured to perform similarity analysis index in the infrared image sequence according to the feature window to obtain an index feature window and analyze to obtain a motion cycle; an abnormal behavior detection module configured to analyze to obtain a temperature behavior similarity and a motion behavior similarity according to the feature temperature point and the motion cycle, calculate to obtain a behavior similarity, determine whether the behavior similarity is greater than a behavior similarity threshold, and obtain an abnormal behavior detection result.

[0008] One or more technical solutions provided in the present application have at least the following technical effects or advantages: The present application provides a video abnormal behavior detection method and system based on image recognition. The abnormal behavior detection method uses an infrared camera to obtain high-quality infrared monitoring video, performs abnormal behavior recognition through an initial abnormal behavior recognizer, and when the target behavior recognition result is yes, obtains and extracts pixel points in a preset window near a feature temperature point to obtain a feature window, performs similarity analysis index, calculates a behavior similarity, and finally obtains an abnormal behavior detection result. Meanwhile, the video abnormal behavior detection system based on image recognition constructs an initial abnormal behavior recognizer through an abnormal behavior recognition module to perform initial abnormal behavior recognition, and finally obtains an abnormal behavior detection result through feature window acquisition, similarity analysis index, and behavior similarity calculation. The detection system is constructed to have image acquisition, image recognition, initial abnormal behavior recognition, abnormal behavior detection analysis, and abnormal behavior detection result output. Compared with the previous single image behavior action recognition method, the present application can accurately and efficiently detect abnormal behavior in images, solves the problems of low abnormal behavior detection efficiency and high false positive rate, avoids the problem of high abnormal behavior detection error rate caused by single detection method, improves the detection effect, and realizes accurate and efficient video abnormal behavior detection. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0010] Figure 1 is a flowchart of a video abnormal behavior detection method based on image recognition; Figure 2This is a schematic diagram of a video abnormal behavior detection system based on image recognition.

[0011] In the attached diagram, the labels represent the following: Abnormal behavior recognition module 11; Feature window acquisition module 12; Similarity analysis index module 13; Abnormal behavior detection module 14. Detailed Implementation

[0012] This application provides a video abnormal behavior detection method and system based on image recognition, which addresses the problems in existing technologies where detection is greatly affected by lighting or relies solely on motion features, resulting in low false alarm efficiency and consequently low detection accuracy, thus affecting the detection effect of video abnormal behavior detection.

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0014] It should be noted that the terms "comprising" and "having" are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0015] The present invention will now be described in detail with reference to the accompanying drawings.

[0016] Example 1, as Figure 1 As shown, this application discloses a video abnormal behavior detection method based on image recognition, the method comprising: S10: Acquire infrared surveillance video within the target scene, perform abnormal behavior identification, and obtain initial abnormal behavior identification results, wherein the infrared surveillance video includes an infrared image sequence, and the target behavior identification results include yes or no; In this embodiment of the application, for example, video is captured in the target scene by an infrared camera, and behavior recognition is performed by an initial recognizer to determine whether smoking behavior may exist, and the result of behavior recognition is output as either yes or no. However, for actions similar to smoking, there may be errors in behavior recognition. For example, when a user raises his hand to drink water, he may be mistakenly identified as having smoking behavior.

[0017] Step S10 in the method provided in this application embodiment includes: Infrared surveillance video of the target scene is acquired, wherein the infrared surveillance video includes a sequence of infrared images; Obtain the initial abnormal behavior identifier; The infrared image sequence is input into the initial abnormal behavior recognizer, and the recognition output obtains the initial abnormal behavior recognition result, wherein the target behavior recognition result includes yes or no.

[0018] Specifically, infrared cameras are used to collect infrared surveillance videos of the target scene. The resulting infrared surveillance videos include infrared image sequences, which are a series of continuous images captured by infrared sensors. These sequences are used for target detection and other applications, such as obtaining the LLVIP Dataset dataset.

[0019] Furthermore, an initial abnormal behavior identifier is obtained. This identifier can be constructed using a convolutional neural network model to classify the acquired infrared image sequences.

[0020] Furthermore, the obtained infrared image sequences are input into an initial abnormal behavior identifier. This identifier classifies and identifies the infrared image sequences, outputting an initial abnormal behavior identification result, where the target behavior identification result includes "yes" or "no". A set of infrared image sequences is input, and the output of the initial abnormal behavior identifier is analyzed. A final result is output through similarity testing. Infrared image sequences with an output of "no" from the initial abnormal behavior identifier are directly excluded, while those with an output of "yes" are further verified.

[0021] For example, a sequence of infrared images of men smoking in a bus stop (e.g., 30 frames per second) is input into the initial abnormal behavior recognizer, and the output result is "yes". A sequence of infrared images of men standing in a bus stop is input, and the output result is "no". The infrared image sequence with the output result of "yes" will be verified and analyzed.

[0022] In this embodiment of the application, step S10, obtaining the initial abnormal behavior identifier, includes: Based on historical surveillance video recordings of abnormal behavior, a set of sample infrared image sequences was collected. For each sample infrared image sequence, label whether abnormal behavior occurs to obtain the initial set of abnormal behavior identification results for the sample. An initial abnormal behavior recognizer is constructed based on a convolutional neural network. The initial abnormal behavior recognizer is trained under supervision using the set of sample infrared image sequences and the set of sample initial abnormal behavior recognition results. The training is completed after the test converges, and the initial abnormal behavior recognizer is obtained.

[0023] Specifically, a set of sample infrared image sequences is collected using video recordings of abnormal behavior over a historical period. To ensure the effectiveness of subsequent analysis and processing, the collected sample infrared image sequence set should be valid, ensuring that the monitored object exists in the infrared image sequence. Infrared image sequences without monitored objects are removed, thereby ensuring the quality and validity of the samples.

[0024] For example, if no monitored object exists in the acquired infrared image sequence, it will be removed and not used as a sample infrared image sequence.

[0025] Furthermore, using annotation tools, either manually or with annotation software, each sample infrared image sequence is labeled to indicate whether abnormal behavior has occurred. Different colored borders can be used to label "yes" and "no" respectively. The infrared image sequences labeled "yes" are then integrated to obtain the initial abnormal behavior identification result set for the samples.

[0026] Furthermore, an initial abnormal behavior recognizer is constructed using a convolutional neural network (CNN) in machine learning algorithms. A CNN is a deep learning model particularly adept at handling image-related tasks, such as image classification and object detection.

[0027] The initial abnormal behavior detector can be trained using the following technical path: Model Construction: It mainly consists of an input layer, convolutional layer, pooling layer, and output layer. The input layer divides the sample infrared image sequence set into two categories: "yes" and "no". The convolutional layer extracts sample features from the sample infrared image sequence set. The pooling layer reduces the spatial size of the data through downsampling, preserving the important features of the sample infrared image sequence set. The output layer maps the two categories of the sample infrared image sequence set one-to-one to the initial abnormal behavior recognition result set, summarizes the extracted features, and outputs the recognition result. Model Training: Using the sample infrared image sequence set and the initial abnormal behavior recognition result set as input, the Adam optimizer (initial learning rate 1e-4) is used. The MSE loss function is selected to calculate the error between the sample infrared image sequence set and the initial abnormal behavior recognition result set. Forward propagation is used to obtain the prediction result, and backpropagation is used to update the parameters. Performance is evaluated using a validation set after each training epoch to avoid overfitting. When the MSE loss decreases by <1e for 5 consecutive training epochs, the model is considered successful. -5 When the MSE loss on the validation set stabilizes below 0.01, the model is considered to have converged. Training is stopped and the final network parameters are saved to obtain the initial abnormal behavior recognizer.

[0028] For example, 1,000 infrared image sequences collected from abnormal behavior monitoring video recordings over the past year were input into an initial abnormal behavior recognizer for initial abnormal behavior identification. 300 of these sequences were marked "yes" and 700 were marked "no".

[0029] In this embodiment, a set of sample infrared image sequences extracted from historical abnormal behavior monitoring videos and a set of initial abnormal behavior identification results obtained through annotation are used as inputs. An initial abnormal behavior recognizer is constructed and trained using a convolutional neural network. This initial abnormal behavior recognizer can be input into the set of infrared image sequences and output the initial abnormal behavior identification results. The use of infrared cameras to collect infrared monitoring videos within the target scene ensures data reliability and high accuracy, avoiding interference from unreliable data sources that could affect the abnormal behavior identification and analysis. This efficient judgment of potential abnormal behaviors in the infrared monitoring videos and rapid acquisition of potential abnormal behavior identification results represents a high-speed identification but low-accuracy screening stage, aiming to reduce the amount of data required for subsequent complex calculations.

[0030] In this embodiment of the application, in order to avoid similar behaviors interfering with the identification of abnormal behaviors, the target behaviors that output "yes" need to be verified in order to improve the accuracy of abnormal behavior detection.

[0031] S20: When the target behavior recognition result is yes, extract the feature temperature points in the infrared image sequence, and extract the pixel points in the preset window near the feature temperature points to obtain the feature window; In this embodiment, instead of selecting the highest temperature point, representative characteristic temperature points are selected by comparing their similarity with the average temperature parameters at the time of the abnormal behavior. Since infrared images can display temperature, the temperature point with the most similar temperature parameters to the temperature of a cigarette butt under the same acquisition conditions is extracted as the characteristic temperature point. After segmentation, image pixels within a window near the characteristic temperature point are extracted as a feature window; for example, 20×20 pixels around the feature temperature point are extracted as a feature window.

[0032] Step S20 in the method provided in this application embodiment includes: Obtain the average temperature parameter when the abnormal behavior occurs; When the target behavior recognition result is yes, temperature points are randomly selected iteratively within the infrared image sequence, the similarity with the temperature parameter is calculated, and the feature temperature point with the highest similarity is selected. Extract the pixels within a preset window near the feature temperature point to obtain the feature window.

[0033] Specifically, based on historical infrared image sequences, an average temperature parameter is set for abnormal behavior. This average temperature parameter can be preset or calculated based on temperature points within the scene. The average temperature parameter is used to distinguish other temperatures, highlighting abnormal behavior.

[0034] For example, the average temperature parameter of the cigarette butt when smoking is set to 700°C.

[0035] Furthermore, within the infrared image sequence where the target behavior recognition result is "yes", temperature points are iteratively randomly selected, and their similarity to the average temperature parameter is calculated. During iterative random selection, first, a temperature point is randomly selected, and the similarity K between the temperature point and the average temperature parameter is calculated. Then, another temperature point is randomly selected, and the similarity W between the temperature point and the average temperature parameter is calculated. If K < W, the similarity K is removed, and the similarity W is retained. If K > W, the similarity K is retained, and one random selection is completed. In the next iteration, the similarity K or W is used to continue comparing with the similarity of the randomly selected temperature point and the average temperature parameter. This process of random selection and iteration is repeated multiple times until the feature temperature point with the highest similarity is obtained. Here, similarity = [1 - (randomly selected temperature point - average temperature parameter) / average temperature parameter]. The larger the value, the more similar the temperature point, and the greater the probability that it will be used as a feature temperature point.

[0036] For example, by randomly selecting and iterating 1000 times, the feature temperature point with the highest similarity is 705℃.

[0037] Furthermore, taking the feature temperature point with the highest similarity as the center, pixels within a preset window near the feature temperature point are extracted from the infrared image corresponding to the feature window to obtain the feature window.

[0038] For example, a rectangular area of ​​20×20 pixels around 705°C is selected as the feature window.

[0039] In this embodiment, when the target behavior recognition result is positive, feature temperature points are extracted from the infrared image sequence, and then pixels within a preset window near the feature temperature points are selected as the feature window. The average temperature parameter of the abnormal behavior is used for calculation to improve performance stability, and an iterative random selection strategy avoids obtaining only the feature temperature points with the highest local similarity, ensuring the accuracy of the selection.

[0040] S30: Based on the feature window, perform similarity analysis indexing within the infrared image sequence to obtain the index feature window and analyze to obtain the motion cycle; In this embodiment, a feature window is obtained from the infrared image sequence where the target behavior recognition result is positive. However, since the initial abnormal behavior recognizer may misidentify actions such as raising a hand to drink water as abnormal behavior when recognizing smoking, it is necessary to analyze the user's motion cycle. Users exhibit repetitive actions while smoking, such as the repetitive actions of letting their hand fall naturally and picking up a cigarette. Randomly divided windows are extracted from other infrared images, and the similarity to these feature windows is calculated to obtain the most similar window and its corresponding infrared image timestamp. The time interval between the infrared image timestamp corresponding to the feature window and the timetamp is calculated as the motion cycle.

[0041] In this embodiment of the application, step S30 includes: The infrared image containing the feature window is marked as the reference infrared image; Within other infrared images, the window to be matched is traversed and divided, and the similarity with the feature window is calculated to obtain the window similarity. The window with the highest similarity to be matched is selected as the index feature window. The infrared image containing the index feature window is marked as the index infrared image, and the time interval between the reference infrared image and the index infrared image is calculated as the motion period.

[0042] Specifically, the infrared image containing the feature window is marked as the reference infrared image, which is used for comparison with randomly divided windows in other infrared images.

[0043] Furthermore, all pixels in the other infrared images are divided into several matching windows through a moving window traversal. The similarity between each matching window and the feature window is calculated, for example, the similarity between the pixels within the matching window and the feature window. A higher similarity value indicates a greater likelihood that the user's smoking behavior is verified as abnormal.

[0044] At the same time, the numerical values ​​of similarity are compared, and several windows to be matched are filtered to select the window with the highest similarity as an index feature window.

[0045] For example, a set of similarities is obtained as: [20%, 50%, 80%, 30%, 60%]. The window to be matched corresponding to the largest similarity of 80% is selected as an index feature window, resulting in a set of similarities as: [80%, 85%, 90%, 90%, 95%]. The window to be matched with a similarity of 95% is selected as the index feature window.

[0046] In addition, the infrared images corresponding to the selected index feature windows are marked as index infrared images. The time interval between the baseline infrared image and the index infrared image is calculated, and the obtained time interval is used as the motion cycle. The motion cycle can reflect the periodicity of behavior and can be used as a basis for judging repetitive abnormal behaviors such as smoking, thus identifying user behavior. The time interval is the difference between the acquisition time of the baseline infrared image and the acquisition time of the index infrared image.

[0047] For example, in the acquired infrared image sequence, the reference infrared image is acquired at 1 second, and the index infrared image corresponding to the index feature window with a similarity of 95% is acquired at 5 seconds. The time interval is (5-1)=4 seconds, that is, the motion cycle is 4 seconds.

[0048] In this embodiment of the application, step S30, which involves traversing and dividing the window to be matched within other infrared images, calculating the similarity with the feature window, and obtaining the window similarity, includes: The matching window is divided by traversing other infrared images. Calculate the mean similarity of pixel values ​​at the same pixel locations within each window to be matched and the feature window, and use this as the window similarity.

[0049] Specifically, all pixels within the other infrared images are divided to obtain several matching windows. Simultaneously, the similarity of pixel values ​​at the same pixel locations between the matching windows and the feature windows is calculated. The mean squared error (MSE) function can be used to calculate the closeness of the corresponding pixel locations between the matching windows and the feature windows. A higher MSE value indicates a higher similarity between the pixel values ​​at the same pixel locations between the matching windows and the feature windows. The mean squared error between the matching windows and the feature windows is defined as:

[0050] Where MSE is the mean squared error, mn is the product of the number of pixels, which represents the area of ​​the image, m is the number of pixels in the image width, n is the number of pixels in the image height, I is the window to be matched, K is the feature window, I(i,j) is the pixel coordinate of the window to be matched, K(i,j) is the coordinate of the same pixel in the feature window, i is any point in m, and j is any point in n.

[0051] For example, the surrounding 20×20 pixels are extracted as a feature window, mn is 400, and a window similarity of 0.8 is calculated.

[0052] In this embodiment, the similarity between the window to be matched and the feature window is calculated using MSE (Mean Search Estimate), and a similarity analysis index is performed within the infrared image sequence to obtain the indexed feature window. The time interval between the baseline infrared image and the indexed infrared image is calculated to obtain the motion cycle, and abnormal user behavior is judged based on the motion cycle. Precise image recognition at the pixel level improves the accuracy of abnormal behavior detection.

[0053] S40: Based on the characteristic temperature point and motion cycle, analyze and obtain temperature behavior similarity and motion behavior similarity, calculate behavior similarity, determine whether it is greater than the behavior similarity threshold, and obtain abnormal behavior detection results.

[0054] In this embodiment, based on a characteristic temperature point and a motion cycle, the similarity between the temperature of the characteristic temperature point and the average temperature parameter formed by the smoking behavior within the image is calculated and used as the temperature behavior similarity. Similarly, the similarity between the motion cycle and the average time cycle of repetitive smoking actions by multiple users is calculated and used as the motion behavior similarity. The average of the temperature behavior similarity and the motion behavior similarity is used as the behavior similarity. Whether the similarity is greater than a behavior similarity threshold is determined to obtain the abnormal behavior detection result.

[0055] In this embodiment of the application, step S40 includes: The pixel points of the feature temperature point and the average temperature parameter when the abnormal behavior occurs are obtained as the temperature behavior similarity. Obtain the average motion cycle of abnormal behavior; Calculate the similarity between the stated motion cycle and the average motion cycle, and use this as the motion behavior similarity. Based on the temperature behavior similarity and motion behavior similarity, the behavior similarity is calculated. Determine whether the behavior similarity is greater than the behavior similarity threshold. If yes, obtain a positive abnormal behavior detection result; otherwise, obtain a negative abnormal behavior detection result.

[0056] Specifically, the similarity between the characteristic temperature point and the average temperature parameter at the time of the abnormal behavior is obtained, and the similarity between the characteristic temperature point and the average temperature parameter at the time of the abnormal behavior is calculated as the temperature behavior similarity. Similarity = [1 - (characteristic temperature point - average temperature parameter at the time of the abnormal behavior) / average temperature parameter at the time of the abnormal behavior], where the larger the value, the closer the characteristic temperature point and the average temperature parameter at the time of the abnormal behavior are, and the greater the probability that the abnormal behavior detection result is "yes".

[0057] For example, the feature temperature point is 705℃, and the average temperature parameter when the abnormal behavior occurs is 700℃. The similarity between the feature temperature point and the pixel point of the average temperature parameter when the abnormal behavior occurs is approximately [1-(705-700) / 700]≈0.992. The value is relatively large, which means that the probability of the abnormal behavior detection result being "yes" is relatively high.

[0058] Furthermore, by using methods such as database queries, the average movement cycle of abnormal behavior is obtained, and the similarity between the movement cycle and the average movement cycle is calculated as the movement behavior similarity. The movement behavior similarity is calculated as [1 - (movement cycle - average movement cycle) / average movement cycle]. The larger the value, the closer the user's movement behavior is to the movement behavior of abnormal behavior in the database.

[0059] For example, the movement cycle is 4s, the average movement cycle is 3.5s, and the similarity is [1-(4-3.5) / 3.5]≈0.857, which is a relatively large value, meaning that the probability of the abnormal behavior detection result being "yes" is relatively high.

[0060] Furthermore, the average of temperature behavior similarity and motion behavior similarity is used as the behavior similarity. Behavior similarity = [(temperature behavior similarity + motion behavior similarity) / 2]. Temperature behavior similarity and motion behavior similarity directly affect the magnitude of behavior similarity; the higher the behavior similarity, the greater the probability that the abnormal behavior detection output is "yes". An abnormal behavior detection result is obtained based on the behavior similarity. A corresponding behavior similarity threshold is set. If the behavior similarity is greater than the threshold, the abnormal behavior detection result is output as "yes"; if the behavior similarity is less than the threshold, a pending abnormal behavior detection result is obtained, and then staff are notified for manual review.

[0061] For example, temperature behavior similarity ≈ 0.992, motion behavior similarity ≈ 0.857, and behavior similarity = [(0.992 + 0.857) / 2] ≈ 0.924. If the behavior similarity threshold is 0.75, and 0.924 > 0.75, it means that the user's behavior in the infrared image sequence within the target scene has passed abnormal behavior detection, and the abnormal behavior detection result is output as "yes".

[0062] In this embodiment, temperature behavior similarity and motion behavior similarity are obtained by using characteristic temperature points and motion cycles. Behavioral similarity is then calculated, and finally, the abnormal behavior detection result is judged based on the behavioral similarity and a behavioral similarity threshold. Compared to existing technologies, this embodiment verifies the initial recognition result of the initial abnormal behavior recognizer through behavioral similarity, avoiding a single verification method that leads to a high false alarm rate in abnormal behavior recognition. Through behavioral similarity analysis, more accurate abnormal behavior detection results are obtained.

[0063] The embodiments of this application, through the specific implementation methods described above, achieve the following technical effects: In this embodiment, firstly, the initial abnormal behavior recognition result set is obtained by extracting sample infrared image sequences from historical abnormal behavior monitoring videos and using them as input. An initial abnormal behavior recognizer is then constructed and trained using a convolutional neural network. This initial abnormal behavior recognizer can be input into the infrared image sequence set and output the initial abnormal behavior recognition result. The use of an infrared camera to collect infrared monitoring videos within the target scene ensures data reliability and high accuracy, avoiding interference from unreliable data sources. Efficiently judging potential abnormal behaviors in the infrared monitoring videos and quickly obtaining possible abnormal behavior recognition results represents a high-speed but less accurate screening stage, aiming to reduce the amount of data required for subsequent complex calculations. Secondly, when the initial abnormal behavior recognition result is "yes," this embodiment extracts feature temperature points within the infrared image sequence and then extracts pixels within a preset window near the feature temperature points as a feature window. The average temperature parameter of abnormal behavior is used to improve performance stability. An iterative random selection strategy avoids obtaining only the feature temperature points with the highest local similarity, ensuring selection accuracy. Next, the similarity between the matching window and the feature window is calculated using MSE (Mean Separation of Equations), and similarity analysis indexing is performed within the infrared image sequence to obtain the indexed feature window. The time interval between the baseline infrared image and the indexed infrared image is calculated to obtain the motion cycle, which is used to identify abnormal user behavior. Precise image recognition at the pixel level improves the accuracy of abnormal behavior detection. Finally, temperature behavior similarity and motion behavior similarity are calculated using feature temperature points and motion cycles to obtain behavior similarity. The final judgment of abnormal behavior detection results is based on behavior similarity and a behavior similarity threshold.

[0064] Compared to existing technologies, this application's embodiments verify the initial recognition results of the initial abnormal behavior recognizer through behavioral similarity, reducing the amount of data required for subsequent complex calculations and avoiding the problem of high false alarm rates in abnormal behavior recognition caused by a single verification method. By performing precise image recognition at the pixel level, the accuracy of abnormal behavior detection is improved. Through behavioral similarity analysis, more accurate abnormal behavior detection results are obtained. Simultaneously, by acquiring infrared image sequences using an infrared detection device, it becomes applicable to complex environments, improving the adaptability of image recognition for video abnormal behavior detection.

[0065] Example 2, as Figure 2 As shown, based on the same inventive concept as the method provided in Embodiment 1, this embodiment of the invention also provides a video abnormal behavior detection system based on image recognition, including: The abnormal behavior recognition module 11 is used to collect infrared monitoring video in the target scene, perform abnormal behavior recognition, and obtain an initial abnormal behavior recognition result. The infrared monitoring video includes an infrared image sequence, and the target behavior recognition result includes yes or no. The feature window acquisition module 12 is used to extract feature temperature points in the infrared image sequence and extract pixel points in a preset window near the feature temperature points when the target behavior recognition result is yes, so as to obtain a feature window. The similarity analysis index module 13 is used to perform similarity analysis indexing within the infrared image sequence based on the feature window, obtain the index feature window, and analyze and obtain the motion cycle. The abnormal behavior detection module 14 is used to analyze and obtain temperature behavior similarity and motion behavior similarity based on the characteristic temperature point and motion cycle, calculate behavior similarity, determine whether it is greater than the behavior similarity threshold, and obtain abnormal behavior detection results.

[0066] In one embodiment, the abnormal behavior identification module 11 is used for: Infrared surveillance video of the target scene is acquired, wherein the infrared surveillance video includes a sequence of infrared images; Obtain the initial abnormal behavior identifier; The infrared image sequence is input into the initial abnormal behavior recognizer, and the recognition output obtains the initial abnormal behavior recognition result, wherein the target behavior recognition result includes yes or no.

[0067] The acquisition of the initial abnormal behavior identifier includes: Based on historical surveillance video recordings of abnormal behavior, a set of sample infrared image sequences was collected. For each sample infrared image sequence, label whether abnormal behavior occurs to obtain the initial set of abnormal behavior identification results for the sample. An initial abnormal behavior recognizer is constructed based on a convolutional neural network. The initial abnormal behavior recognizer is trained under supervision using the set of sample infrared image sequences and the set of sample initial abnormal behavior recognition results. The training is completed after the test converges, and the initial abnormal behavior recognizer is obtained.

[0068] In one embodiment, the feature window acquisition module 12 is used for: Obtain the average temperature parameter when the abnormal behavior occurs; When the target behavior recognition result is yes, temperature points are randomly selected iteratively within the infrared image sequence, the similarity with the temperature parameter is calculated, and the feature temperature point with the highest similarity is selected. Extract the pixels within a preset window near the feature temperature point to obtain the feature window.

[0069] In one embodiment, the similarity analysis index module 13 is used for: The infrared image containing the feature window is marked as the reference infrared image; Within other infrared images, the window to be matched is traversed and divided, and the similarity with the feature window is calculated to obtain the window similarity. The window with the highest similarity to be matched is selected as the index feature window. The infrared image containing the index feature window is marked as the index infrared image, and the time interval between the reference infrared image and the index infrared image is calculated as the motion period.

[0070] The step of traversing and dividing the window to be matched within other infrared images, calculating the similarity with the feature window, and obtaining the window similarity includes: The matching window is divided by traversing other infrared images. Calculate the mean similarity of pixel values ​​at the same pixel locations within each window to be matched and the feature window, and use this as the window similarity.

[0071] In one embodiment, the abnormal behavior detection module 14 is used to: obtain the pixel points of the feature temperature point and the average temperature parameter when the abnormal behavior occurs, as the temperature behavior similarity; Obtain the average motion cycle of abnormal behavior; Calculate the similarity between the stated motion cycle and the average motion cycle, and use this as the motion behavior similarity. Based on the temperature behavior similarity and motion behavior similarity, the behavior similarity is calculated. Determine whether the behavior similarity is greater than the behavior similarity threshold. If yes, obtain a positive abnormal behavior detection result; otherwise, obtain a negative abnormal behavior detection result.

[0072] This application first utilizes an abnormal behavior recognition module to construct an initial abnormal behavior recognizer for initial abnormal behavior recognition. The initial abnormal behavior recognition result is used as the basis for analyzing the abnormal behavior detection result. Second, through a feature window acquisition module, when the target behavior recognition result is positive, feature temperature points and pixels within a preset window near the feature temperature points are extracted from the infrared image sequence to obtain a feature window. Third, through a similarity analysis indexing module, the feature window is used to perform similarity analysis indexing within the infrared image sequence to obtain an indexed feature window, thus obtaining the motion cycle of the abnormal behavior. The motion cycle provides data support for behavior similarity calculation. Finally, through an abnormal behavior detection module, the feature temperature points and motion cycle are used to analyze temperature behavior similarity and motion behavior similarity, perform similarity calculation, obtain behavior similarity, and determine whether it exceeds a behavior similarity threshold to obtain the abnormal behavior detection result. This constructs a detection system with image acquisition, image recognition, initial abnormal behavior recognition, abnormal behavior detection analysis, and abnormal behavior detection result output. Compared to previous single-method image behavior and action recognition, this application can accurately and efficiently detect abnormal behavior in images, avoiding the problem of high error rate in abnormal behavior detection caused by a single detection method, thus improving the detection effect and achieving efficient and accurate detection.

[0073] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0074] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A video abnormal behavior detection method based on image recognition, characterized in that, The method includes: Infrared surveillance video of the target scene is collected, abnormal behavior is identified, and an initial abnormal behavior identification result is obtained. The infrared surveillance video includes an infrared image sequence, and the target behavior identification result includes yes or no. When the target behavior recognition result is yes, extract the feature temperature points in the infrared image sequence, and extract the pixel points in a preset window near the feature temperature points to obtain the feature window; Based on the feature window, a similarity analysis index is performed within the infrared image sequence to obtain the index feature window, and the motion cycle is obtained through analysis. Based on the characteristic temperature points and motion cycles, temperature behavior similarity and motion behavior similarity are analyzed and obtained. Behavior similarity is calculated and it is determined whether it is greater than the behavior similarity threshold to obtain abnormal behavior detection results.

2. The video abnormal behavior detection method based on image recognition according to claim 1, characterized in that, Infrared surveillance video of the target scene is acquired, abnormal behavior is identified, and initial abnormal behavior identification results are obtained, including: Infrared surveillance video of the target scene is acquired, wherein the infrared surveillance video includes a sequence of infrared images; Obtain the initial abnormal behavior identifier; The infrared image sequence is input into the initial abnormal behavior recognizer, and the recognition output obtains the initial abnormal behavior recognition result, wherein the target behavior recognition result includes yes or no.

3. The video abnormal behavior detection method based on image recognition according to claim 2, characterized in that, Obtain the initial abnormal behavior identifier, including: Based on historical surveillance video recordings of abnormal behavior, a set of sample infrared image sequences was collected. For each sample infrared image sequence, label whether abnormal behavior occurs to obtain the initial set of abnormal behavior identification results for the sample. An initial abnormal behavior recognizer is constructed based on a convolutional neural network. The initial abnormal behavior recognizer is trained under supervision using the set of sample infrared image sequences and the set of sample initial abnormal behavior recognition results. The training is completed after the test converges, and the initial abnormal behavior recognizer is obtained.

4. The video abnormal behavior detection method based on image recognition according to claim 1, characterized in that, When the target behavior recognition result is yes, feature temperature points are extracted from the infrared image sequence, and pixels within a preset window near the feature temperature points are extracted to obtain a feature window, including: Obtain the average temperature parameter when the abnormal behavior occurs; When the target behavior recognition result is yes, temperature points are randomly selected iteratively within the infrared image sequence, the similarity with the temperature parameter is calculated, and the feature temperature point with the highest similarity is selected. Extract the pixels within a preset window near the feature temperature point to obtain the feature window.

5. The video abnormal behavior detection method based on image recognition according to claim 1, characterized in that, Based on the feature window, similarity analysis indexing is performed within the infrared image sequence to obtain the index feature window, and the motion period is obtained through analysis, including: The infrared image containing the feature window is marked as the reference infrared image; Within other infrared images, the window to be matched is traversed and divided, and the similarity with the feature window is calculated to obtain the window similarity. The window with the highest similarity to be matched is selected as the index feature window. The infrared image containing the index feature window is marked as the index infrared image, and the time interval between the reference infrared image and the index infrared image is calculated as the motion period.

6. The video abnormal behavior detection method based on image recognition according to claim 5, characterized in that, Within other infrared images, a window to be matched is traversed and divided, and the similarity with the feature window is calculated to obtain the window similarity, including: The matching window is divided by traversing other infrared images. Calculate the mean similarity of pixel values ​​at the same pixel locations within each window to be matched and the feature window, and use this as the window similarity.

7. The video abnormal behavior detection method based on image recognition according to claim 1, characterized in that, Based on the characteristic temperature points and motion cycles, temperature behavior similarity and motion behavior similarity are analyzed and obtained. Behavioral similarity is calculated, and it is determined whether it exceeds a behavioral similarity threshold to obtain abnormal behavior detection results, including: The pixel points of the feature temperature point and the average temperature parameter when the abnormal behavior occurs are obtained as the temperature behavior similarity. Obtain the average motion cycle of abnormal behavior; Calculate the similarity between the stated motion cycle and the average motion cycle, and use this as the motion behavior similarity. Based on the temperature behavior similarity and motion behavior similarity, the behavior similarity is calculated. Determine whether the behavior similarity is greater than the behavior similarity threshold. If yes, obtain a positive abnormal behavior detection result; otherwise, obtain a negative abnormal behavior detection result.

8. A video abnormal behavior detection system based on image recognition, characterized in that, The system for implementing the method according to any one of claims 1-7 comprises: An abnormal behavior recognition module is used to collect infrared surveillance video in a target scene, perform abnormal behavior recognition, and obtain an initial abnormal behavior recognition result. The infrared surveillance video includes an infrared image sequence, and the target behavior recognition result includes yes or no. The feature window acquisition module is used to extract feature temperature points in the infrared image sequence when the target behavior recognition result is yes, and to extract pixel points in a preset window near the feature temperature points to obtain a feature window; The similarity analysis index module is used to perform similarity analysis indexing within the infrared image sequence based on the feature window, obtain the index feature window, and analyze and obtain the motion cycle; The abnormal behavior detection module is used to analyze and obtain temperature behavior similarity and motion behavior similarity based on the characteristic temperature point and motion cycle, calculate the behavior similarity, determine whether it is greater than the behavior similarity threshold, and obtain the abnormal behavior detection result.

Images