A video monitoring early warning method based on AI analysis

Abstract

The application relates to the field of safety early warning processing, in particular to a video monitoring early warning method based on AI analysis, which comprises the following steps: constructing a human body key point detection model by using historical image data, extracting key point positions and change sequences, labeling the sequences according to behaviors, calculating the importance of the key point sequences to each behavior, screening important key points, constructing a time sequence prediction model, determining an input length, at the same time, constructing a region division model, dynamically adjusting a detection threshold according to the safety early warning grades of different regions to behaviors, inputting a real-time image into the model, extracting key point sequences, predicting behavior probabilities and comparing the behavior probabilities with a threshold, judging abnormal behaviors and early warning. The application can automatically extract key point position information and change sequences to accurately identify personnel behaviors by constructing a human body key point detection model and a time sequence prediction model, and can dynamically adjust a detection threshold by setting safety early warning grades of different regions, thereby improving the efficiency, accuracy and flexibility of a monitoring system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of safety early warning processing. In particular, it relates to a video monitoring early warning method based on AI analysis. BACKGROUND

[0002] With the development of technology, video monitoring early warning systems are introduced. With the help of advanced AI technology systems, the monitoring screen can be automatically analyzed, thus effectively making up for the deficiency of traditional monitoring which only relies on manual observation, and injecting new vitality into safety prevention and control work.

[0003] The data generated by the video monitoring system is extremely large, and the data types are complex and diverse, covering images, video streams and other forms. In the face of such a large amount of data, the traditional manual processing method is not up to the task and cannot complete timely and accurate analysis of all data in a short time, which greatly limits the effectiveness of the monitoring system. SUMMARY

[0004] To solve the problem of large amount of data generated by the video monitoring system and the problem of traditional manual processing being difficult to analyze timely and accurately, which limits the effectiveness of the monitoring system, the present application provides a solution in the following aspects.

[0005] A video monitoring early warning method based on AI analysis, comprising: obtaining a historical image data set, constructing a human key point detection model to extract the position information and change sequence of the key points of the personnel, and identifying the specific behavior of the personnel, setting labels for the change sequence based on the specific behavior, wherein the labels include: normal, robbery behavior, theft behavior and abnormal gathering; calculating the importance of each human key point sequence for different behaviors, and screening the key points according to the importance, constructing a time series prediction model to accurately identify different behaviors using the change sequence of the screened key points, and determining the input length of the time series prediction model; constructing a region division model to extract the image division region in the historical image set, setting a safety early warning level according to the different regions in the image corresponding to different behaviors, and dynamically adjusting the detection threshold of each region; inputting real-time images into the key point detection model, extracting the change sequence of the screened key points, inputting the key point change sequence into the time series prediction model, obtaining the probability of different behaviors, comparing the behavior probability with the detection threshold of different regions corresponding to different behaviors, judging whether there is an abnormal behavior, and completing the video monitoring early warning of AI analysis.

[0006] Preferably, the human key point detection model is constructed, comprising:

[0007] Adjust the uniform size and label each image in the historical image dataset to obtain the key point positions of the human body, take 80% of the labeled images as a training dataset and 20% as a verification set for training a preset detection model, wherein the key point positions include: wrist, elbow, shoulder, hip joint, knee, ankle, eye, ear;

[0008] Use a deep convolutional neural network as a feature extraction part of the preset detection model, input an image containing a human body, and output the positions of each key point of the human body in the image, a heat map and a change sequence, use the error between the predicted position of the key point and the real labeled position as a loss function, the loss function is optimized by using mean square error, and the performance is evaluated by using the verification set, the model training is completed, and a human key point detection model is obtained.

[0009] Preferably, the importance of each human key point sequence to different behaviors is calculated, including:

[0010] Taking any behavior as a target behavior, the key points corresponding to the target behavior are to-be-analyzed points, calculating the mean value of the change sequence of the to-be-analyzed points corresponding to the target behavior in the historical image set, and the mean value of the change sequence of the to-be-analyzed points corresponding to other behaviors except the target behavior, and calculating the similarity between the two mean values by a similarity function, and taking the difference between 1 and the similarity as the importance of the to-be-analyzed point under the target behavior.

[0011] Preferably, the importance of each human key point sequence to different behaviors is calculated, including:

[0012] According to the ranking result of the similarity, the key point sequence corresponding to the similarity of the preset position is selected as the representative sequence, wherein the preset position includes: the change sequence with the smallest similarity, the change sequence with the similarity located at the quantile position, the change sequence with the similarity located at the middle position, the change sequence with the similarity located at the quantile position and the change sequence with the largest similarity;

[0013] Obtain the representative sequence corresponding to each key point under the target behavior, calculate the similarity between any two combinations of the representative sequence, divide the mean value of the similarity of all combinations by the total number of any two combinations of the representative sequence, to obtain the average similarity, and take the difference between 1 and the average similarity as the importance of the to-be-analyzed point under the target behavior.

[0014] Preferably, the key points are screened according to the importance, including:

[0015] The importance of the key points is normalized, and in response to the normalized importance of the key points being greater than a preset threshold, the key points are retained as significant features for behavior recognition for subsequent model training and behavior prediction; otherwise, the key points are discarded.

[0016] Preferably, the input length of the time series prediction model is determined, including:

[0017] The period of the significant key point sequence is calculated using Fourier transform, and the maximum value of the periods of the change sequences corresponding to all significant key points is selected as the input length of the time series prediction model to cover the change characteristics of all key point sequences.

[0018] Preferably, the time series prediction model includes:

[0019] The change sequence corresponding to the screened key points is used as the model input, and the output is the behavior label corresponding to the change sequence. During training of the time series prediction model, a cross-entropy loss function is used to evaluate the difference between the predicted behavior label and the true label, and a gradient descent algorithm is used to optimize the model parameters to minimize the loss value. When the model reaches a preset maximum training number or the loss value is lower than a set threshold, the training is stopped, and the optimal model is selected according to the evaluation index of the model.

[0020] Preferably, the region division model is constructed, including:

[0021] The regions of the images in the historical image dataset are divided, wherein the regions include dynamic regions and static regions. The real-time acquired images are input into the key point detection model and the region division model to obtain the key point sequence of each region, and then input into the time series prediction model to obtain the behavior output probability.

[0022] Preferably, the training process of the region division model includes:

[0023] The region division model is a convolutional neural network (CNN). The input layer is an image captured by a video monitoring device. The convolutional layer is used to extract image features. The pooling layer is used to reduce the spatial dimension of the feature image. The up-sampling layer is used to restore the feature image to the same spatial dimension as the input image for pixel-level classification. The output layer is the number of classes of the divided regions.

[0024] The cross-entropy loss function and the gradient descent algorithm are used to optimize the model parameters to accurately divide the dynamic regions and the static regions in the monitoring images.

[0025] Preferably, the detection threshold satisfies the following relationship:

[0026] ;

[0027] In the formula, represents the detection threshold of the first behavior in the first region, represents the detection threshold of the first behavior in the first region, represents the detection threshold of the first behavior in the first region, represents an exponential function with a natural number as the base, represents the initial threshold of behavior detection, represents a constant, represents the safety warning level of the first behavior in the first region.

[0028] The present application has the following effects:

[0029] 1. The present application can automatically extract key point position information and change sequence, and accurately identify the specific behavior of personnel by constructing a human key point detection model and a time series prediction model. This method not only improves the efficiency of the monitoring system, but also reduces the error and fatigue of manual monitoring, ensuring the accuracy and reliability of the monitoring system.

[0030] 2. The present application can effectively reduce the input dimension of the model, improve the training efficiency and prediction accuracy of the model by calculating the importance of each key point sequence for different behaviors and filtering key points according to the importance. At the same time, the input length of the time series prediction model is determined to ensure that the model can fully capture the time dynamic characteristics of the key point sequence, further improving the performance of the model.

[0031] 3. The present application can dynamically adjust the detection threshold according to the specific needs of different regions by constructing a region division model and setting different safety warning levels for different regions. This method not only improves the flexibility of the monitoring system, but also can be adjusted in real time according to the changes of the actual scene, ensuring the effectiveness and adaptability of the monitoring system in different environments. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 is a method flowchart of steps S1-S4 in a video monitoring and warning method based on AI analysis according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all embodiments.

[0034] Referring to Figure 1 , a video monitoring and warning method based on AI analysis includes steps S1-S4, as follows:

[0035] ​​​S1: Obtain a historical image dataset, construct a human key point detection model to extract the position information and change sequence of the key points of the personnel, and identify the specific behavior of the personnel, and set a label for the change sequence based on the specific behavior, wherein the label includes: normal, robbery behavior, theft behavior and abnormal gathering.

[0036] Constructing a human key point detection model comprises:

[0037] Adjusting the uniform size of each image in the historical image dataset and labeling, obtaining the key point position of the human body, taking 80% of the labeled image as the training data set and 20% as the verification set for training the preset detection model, wherein the key point position includes: wrist, elbow, shoulder, hip joint, knee, ankle, eye, ear;

[0038] Using a deep convolutional neural network as the feature extraction part of the preset detection model, the input is an image containing a human body, and the output is the position of each key point of the human body in the image, a heat map and a change sequence, using the error between the predicted position of the key point and the real labeled position as the loss function, the loss function is optimized by mean square error, and the performance is evaluated by using the verification set, the model training is completed, and the human key point detection model is obtained.

[0039] For example, the model architecture of the convolutional neural network can adopt ResNet, MobileNet or HourglassNetwork, which are well-known technologies to those skilled in the art and will not be described in detail.

[0040] Further analysis, in abnormal behavior recognition, different key points of the human body have different importance to different behaviors, for example: the key points of the hands have a greater correlation with theft, and the key points of the head have a greater correlation with abnormal gathering, therefore, according to the changes of each key point under different behaviors, the importance of each key point to each behavior is determined, and the specific steps are as follows:

[0041] S2: Calculate the importance of each human key point sequence to different behaviors, and select key points according to the importance, construct a time series prediction model to accurately identify different behaviors from the change sequence of the selected key points, and determine the input length of the time series prediction model.

[0042] Calculating the importance of each human key point sequence to different behaviors comprises:

[0043] Taking any behavior as a target behavior, the key point corresponding to the target behavior as an analysis point, calculating the mean of the change sequence of the analysis point corresponding to the target behavior in the historical image set, and the mean of the change sequence of the analysis point corresponding to other behaviors except the target behavior, and calculating the similarity between the two means through a similarity function, and taking the difference between 1 and the similarity as the importance of the analysis point under the target behavior.

[0044] Specifically, importance satisfies the following relationship:

[0045] ;

[0046] in, Indicates the first The first action The importance of these key points Indicates similarity. Indicates the historical number The first action The mean of the change sequence corresponding to each key point. Indicates the historical number Under all other behaviors besides the first behavior, the first The mean of the change sequence corresponding to each key point.

[0047] It's important to note that each behavior corresponds to a sequence of changes, and a single behavior may have multiple key points. Therefore, it's necessary to analyze the importance of each key point to each behavior. The greater the similarity between the two, the smaller the difference in the key point sequence under different behaviors, and the greater the likelihood of error when using the key point sequence to identify the behavior. The Pearson correlation coefficient can be used to calculate the similarity.

[0048] In addition, another embodiment includes:

[0049] Based on the similarity ranking results, keypoint sequences corresponding to the similarity at preset positions are selected as representative sequences. These preset positions include: the variation sequence with the lowest similarity, and sequences with similarity at... Sequences showing changes in quantile positions, sequences showing changes in similarity at the middle position, sequences showing similarity at... The sequence of changes in quantile positions and the sequence of changes with the greatest similarity;

[0050] Obtain representative sequences corresponding to each key point under the target behavior, calculate the similarity between representative sequences of any pairwise combination, divide the mean similarity of all combinations by the total number of representative sequences of any pairwise combination to obtain the average similarity, and take the difference between 1 and the average similarity as the importance of the point to be analyzed under the target behavior.

[0051] Specifically, importance satisfies the following relationship:

[0052] ;

[0053] in, Indicates the first The first action The importance of these key points Indicates similarity. represents the first representative sequence in the change sequence corresponding to the first key point under the first behavior, represents the first representative sequence in the change sequence corresponding to the first key point under the first behavior, represents the first representative sequence in the change sequence corresponding to the first key point under the first behavior, represents the first representative sequence in the change sequence corresponding to the first key point under the first behavior, represents the first representative sequence in the change sequence corresponding to the first key point under the first behavior, represents the first representative sequence in the change sequence corresponding to the first key point under the first behavior, represents the first representative sequence in the change sequence corresponding to the first key point under the first behavior, represents the first representative sequence in the change sequence corresponding to the first key point under the first behavior.

[0054] That is, the importance is reflected by calculating the similarity of the overall distribution of the two sequences, increasing the amount of importance calculation and improving the accuracy of the importance. By selecting the change sequence corresponding to the preset position similarity as the representative, the distribution characteristics of the key point change sequence can be fully reflected. The minimum value and the maximum value respectively represent the minimum and maximum changes, and the quantile provides different levels of intermediate changes. These representative sequences can be used for subsequent analysis and model training to better describe and distinguish the key point change patterns under different behaviors.

[0055] Screening key points according to importance, including:

[0056] Normalizing the importance of the key points, and in response to the normalized importance of the key points being greater than a preset threshold, retaining the key points as significant features for behavior recognition for subsequent model training and behavior prediction; otherwise, discarding the key points.

[0057] For example, the preset threshold is 0.7, which can be adjusted according to specific circumstances.

[0058] Determining the input length of the time series prediction model, including:

[0059] Using Fourier transform to calculate the period of the significant key point sequence, and selecting the maximum value of the periods of the change sequences corresponding to all significant key points as the input length of the time series prediction model to cover the change characteristics of all key point sequences.

[0060] Time series prediction model, including:

[0061] Using the change sequence corresponding to the screened key points as the model input, and the output is the behavior label corresponding to the change sequence. During training of the time series prediction model, a cross-entropy loss function is used to evaluate the difference between the predicted behavior label and the true label, and a gradient descent algorithm is used to optimize the model parameters to minimize the loss value. When the model reaches a preset maximum training number or the loss value is lower than a set threshold, the training is stopped, and the optimal model is selected according to the evaluation index of the model.

[0062] S3: Construct a region segmentation model to extract image regions from the historical image set, set security warning levels according to different behaviors corresponding to different regions in the image, and dynamically adjust the detection threshold of each region.

[0063] The images in the historical image dataset are divided into regions, including dynamic regions and static regions. The real-time acquired images are input into the keypoint detection model and the region division model to obtain the keypoint sequence of each region. Then, the sequence is input into the time series prediction model to obtain the behavior output probability.

[0064] In other words, the higher the warning level, the more closely the behavior needs to be monitored in that area, and therefore, the lower the detection threshold for that behavior in that area. For example, the threshold for theft is lower in static areas than in dynamic areas; the threshold for abnormal clustering is lower in dynamic areas than in static areas.

[0065] The training process of the region partitioning model includes the following steps:

[0066] The region segmentation model is a convolutional neural network (CNN). The input layer is an image captured by video surveillance. The convolutional layers are used to extract image features. The pooling layers are used to reduce the spatial dimension of the feature images. The upsampling layers restore the feature images to the same spatial dimension as the input images for pixel-level classification. The output layer is the number of categories for the segmented regions.

[0067] The model parameters are optimized using the cross-entropy loss function and gradient descent algorithm to achieve accurate segmentation of dynamic and static regions in monitoring images.

[0068] The detection threshold satisfies the following relationship:

[0069] ;

[0070] In the formula, Indicates the first In the region, the first The detection threshold for each behavior. Represented by natural numbers An exponential function with base 0. This represents the initial threshold for behavior detection. Represents a constant. Indicates the first In the region, the first The safety warning level for each behavior.

[0071] In other words, settings The goal is to balance security levels and thresholds.

[0072] The security early warning system of the present application is mainly deployed in the area of lottery shops and the like, and in the area of the staff in the image, which is the storage place of money and other valuable objects. Therefore, the security requirement for theft / robbery behavior in this area is higher, and the security requirement for abnormal gathering behavior due to physical conflict or too dense personnel in the personnel flow area is higher. Therefore, different security early warning thresholds are set for behaviors in different areas. The higher the early warning level, the more the behavior needs to be monitored in the area, and therefore the detection threshold of the behavior in the area is low. For example, the threshold of theft behavior in a static area is lower than that in a dynamic area, and the threshold of abnormal gathering in a dynamic area is lower than that in a static area.

[0073] In the present application, the static area is the area where the lottery shop staff (such as the area where the lottery shop owner works) work. There are fewer personnel in this area, and only a small number of staff are generally in this area. There are more personnel in the dynamic area, and there are a large number of customers and a large flow of people.

[0074] S4: input the real-time image into the key point detection model, extract the transformation sequence of the screened key points, input the key point transformation sequence into the time series prediction model, obtain the probability of different behaviors, compare the behavior probability with the detection threshold of different behaviors corresponding to different areas, judge whether there is an abnormal behavior, and complete the AI analysis of video monitoring and early warning.

[0075] If the behavior probability is higher than the detection threshold of the corresponding area, it is considered that there is an abnormal behavior, the system issues an alarm, and the AI analysis of video monitoring and early warning is completed; otherwise, it is considered that there is no abnormal behavior.

[0076] It should be noted that, for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the protection scope of the present application patent should be subject to the appended claims.

Claims

1.A method for video monitoring early warning based on AI analysis, characterized in that, The application relates to a video monitoring early warning method based on AI analysis, and belongs to the field of video monitoring early warning. The method comprises the following steps: acquiring a historical image data set, constructing a human body key point detection model to extract position information and change sequences of key points of a person, and identifying specific behaviors of the person, wherein the change sequences are labeled based on the specific behaviors, and the labels comprise normal, robbery behavior, theft behavior and abnormal gathering; calculating the importance of each human body key point sequence for different behaviors, screening key points according to the importance, and constructing a time sequence prediction model to accurately identify different behaviors by using the change sequences of the screened key points, and determining the input length of the time sequence prediction model; constructing a region division model to extract image division regions in the historical image data set, setting a safety warning level according to different regions corresponding to different behaviors in the image, and dynamically adjusting the detection threshold of each region; inputting a real-time image into the key point detection model, extracting the change sequences of the screened key points, inputting the key point change sequences into the time sequence prediction model to obtain the probabilities of different behaviors, comparing the behavior probabilities with the detection threshold of different regions corresponding to different behaviors, judging whether there is an abnormal behavior, and completing AI analysis of the video monitoring early warning; wherein the calculation of the importance of each human body key point sequence for different behaviors comprises: taking any behavior as a target behavior, and taking the key points corresponding to the target behavior as analysis points, calculating the mean value of the change sequences of the analysis points corresponding to the target behavior in the historical image data set, and the mean value of the change sequences of the analysis points corresponding to other behaviors except the target behavior, and calculating the similarity between the two mean values through a similarity function, and taking the difference between 1 and the similarity as the importance of the analysis points under the target behavior; According to the ranking result of the similarity, a key point sequence corresponding to the similarity of a preset position is selected as a representative sequence, wherein the preset position includes: a change sequence with the minimum similarity, a change sequence with the similarity located in a quantile position, a change sequence with the similarity located in a middle position, a change sequence with the similarity located in a quantile position, and a change sequence with the maximum similarity. the calculation of the importance of each human body key point sequence for different behaviors comprises: 2.The AI analysis-based video monitoring early warning method of claim 1, wherein, acquiring the representative sequences corresponding to the key points under the target behavior, calculating the similarity between any two combinations of the representative sequences, dividing the average similarity of all combinations by the total number of any two combinations of the representative sequences, obtaining the average similarity, and taking the difference between 1 and the average similarity as the importance of the analysis points under the target behavior. The construction of the human body key point detection model comprises: adjusting the uniform size of each image in the historical image data set and labeling the image, acquiring the position of the key points of the human body, taking 80% of the labeled images as a training data set and 20% as a verification set, and using the training data set to train a preset detection model, wherein the position of the key points comprises a wrist, an elbow, a shoulder, a hip joint, a knee, an ankle, an eye and an ear; 3.The AI analysis-based video monitoring and early warning method of claim 1, wherein, using a deep convolutional neural network as a feature extraction part of the preset detection model, inputting an image containing a human body, outputting the position of each key point of the human body in the image, a heat map and a change sequence, using the error between the predicted position of the key points and the real labeled position as a loss function, optimizing the loss function by using a mean square error, simultaneously performing performance evaluation by using the verification set, completing model training, and obtaining the human body key point detection model. The screening of the key points according to the importance comprises: The importance of the key points is normalized, and in response to the normalized importance of the key points being greater than a preset threshold, the key points are retained as significant features for behavior recognition for subsequent model training and behavior prediction; otherwise, the key points are discarded. 4.The AI analysis-based video monitoring early warning method of claim 1, wherein, The input length of the time sequence prediction model is determined, including: The period of the significant key point sequence is calculated using Fourier transform, and the maximum value of the periods of the change sequences corresponding to all significant key points is selected as the input length of the time sequence prediction model to cover the change characteristics of all key point sequences. 5.The AI analysis-based video monitoring and early warning method of claim 1, wherein, The time sequence prediction model includes: The change sequence corresponding to the screened key points is used as the model input, and the output is the behavior label corresponding to the change sequence. During training of the time sequence prediction model, a cross-entropy loss function is used to evaluate the difference between the predicted behavior label and the true label, and a gradient descent algorithm is used to optimize the model parameters to minimize the loss value. When the model reaches a preset maximum training number or the loss value is lower than a set threshold, the training is stopped, and the optimal model is selected according to the evaluation index of the model. 6.The AI analysis-based video monitoring and early warning method of claim 1, wherein, The region division model is constructed, including: The images in the historical image dataset are divided into regions, where the regions include dynamic regions and static regions. The real-time acquired images are input into the key point detection model and the region division model to obtain the key point sequence of each region, and then input into the time sequence prediction model to obtain the behavior output probability. 7.The AI analysis-based video monitoring and early warning method of claim 1, wherein, The training process of the region division model includes: The region division model is a convolutional neural network (CNN), the input layer is an image captured by a video monitoring device, the convolutional layer is used to extract image features, the pooling layer is used to reduce the spatial dimension of the feature image, the up-sampling layer is used to restore the feature image to the same spatial dimension as the input image for pixel-level classification, and the output layer is the number of classes of the divided regions. The cross-entropy loss function and the gradient descent algorithm are used to optimize the model parameters to accurately divide the dynamic regions and the static regions in the monitoring images. 8.The AI analysis-based video monitoring early warning method of claim 1, wherein, The detection threshold satisfies the following relationship: ； wherein represents a detection threshold for the th behavior in the th zone, represents an exponential function with a natural number as the base, represents an initial threshold for behavior detection, represents a constant, represents a safety warning level for the th behavior in the th zone.

Images