Video monitoring alarm method based on deep learning

Abstract

The invention relates to the technical field of video monitoring, in particular to a video monitoring alarm method based on deep learning. According to the technical scheme, the method comprises the steps of video collection, image preprocessing, target detection, alarm judgment, alarm output and result recording, video collection comprises the steps that a camera is used for collecting video streams in real time, the video streams are stored in a video buffer area, meanwhile, real-time frame rate stability analysis is carried out on the collected video streams, and if frame rate fluctuation exceeds a preset threshold value, the video streams are detected; and camera parameters are automatically adjusted to stabilize the frame rate. According to the invention, through improving links of image preprocessing, target detection, alarm judgment, alarm output, result recording, model updating and the like, target detection accuracy, alarm precision, resource allocation rationality, data security and model updating efficiency and adaptability are improved, and the performance of a video monitoring alarm system is comprehensively improved.

Description

Technical Field

[0001] The present invention relates to the field of video surveillance technology, and in particular to a video surveillance alarm method based on deep learning. Background Art

[0002] With the rapid development of science and technology, the importance of video surveillance technology in the field of security has become increasingly prominent. In modern society, video surveillance systems are widely deployed in public places, commercial facilities, industrial sites, etc. to ensure the safety of people and property. These systems have gradually evolved from simple analog monitoring in the early days to today's complex systems based on digital and intelligent technologies.

[0003] Early video surveillance mainly relied on manual observation of surveillance images to detect abnormal situations, which is inefficient and prone to omissions. With the rise of computer vision and deep learning technology, video surveillance alarm methods based on deep learning have become a research hotspot. Deep learning models can automatically learn features from large amounts of data and have demonstrated excellent performance in target detection and recognition.

[0004] However, the current video surveillance alarm technology based on deep learning faces a prominent problem: in actual application scenarios, the monitoring environment is complex and changeable, and the types of targets are numerous and constantly updated, which leads to the need for frequent updates of deep learning models to adapt to new targets, but the model update process faces huge challenges. On the one hand, if the entire model is retrained to adapt to new targets, it will take a lot of computing resources and time, especially when processing large-scale video data. This problem is more serious. For example, in a large urban traffic monitoring system, a huge amount of video data is generated every day, and new vehicle types (such as new concept cars, special-purpose vehicles, etc.) continue to appear. If the model is retrained every time, the system will be overloaded and unable to meet the needs of real-time monitoring. On the other hand, simply fine-tuning the model locally may not be able to effectively identify new targets, because new targets may have large differences in features from existing targets. This difference may lead to false detection or missed detection, thereby reducing the accuracy and reliability of the monitoring alarm system and affecting the security effect. Therefore, how to efficiently update model parameters and reduce resource consumption while ensuring the adaptability of the model to new targets has become a key issue in the development of video surveillance alarm technology based on deep learning.

[0005] Therefore, this application proposes a video surveillance alarm method based on deep learning. Summary of the invention

[0006] The purpose of the present invention is to propose a video surveillance alarm method based on deep learning in view of the fact that video surveillance alarm technology in the background technology faces complex and changeable monitoring environments in actual application scenarios, and there are many types of targets that are constantly updated, which leads to the need for deep learning models to be frequently updated to adapt to new targets.

[0007] The technical solution of the present invention is a video monitoring alarm method based on deep learning, comprising the following steps:

[0008] S1, video acquisition: use the camera to collect video streams in real time and save the video streams to the video buffer. At the same time, perform real-time frame rate stability analysis on the collected video streams. If the frame rate fluctuation exceeds the preset threshold, the camera parameters are automatically adjusted to stabilize the frame rate.

[0009] S2, image preprocessing: extract frames from the collected video stream, resize and convert the colors of each frame, calculate the width and height scaling factors of the image, encode the alarm area, and perform wavelet transform on the image to enhance image details. The formula is:

[0010] I enhanced =W -1 (T threshold (W(I)))

[0011] Where I is the original image, W is the wavelet transform, T threshold is the threshold processing function, I enhanced is the enhanced image;

[0012] S3, Target detection: Target detection is performed on the preprocessed image by calling the API interface of the deep learning model. The model outputs the detected target box, category and confidence, and post-processes the detection results, including removing targets of specific categories. At the same time, a multi-scale feature fusion algorithm is used to improve the accuracy of small target detection. The formula is:

[0013]

[0014] Among them, F i are feature maps of different scales, w i is the corresponding weight, F fused is the fused feature map;

[0015] S4, alarm judgment: According to the location, category and confidence of the target, combined with the preset alarm conditions, the number of frames in which the target is not detected is counted, and the vehicle information is reset when the set threshold is exceeded. At the same time, the target behavior trajectory analysis is introduced to judge abnormal behavior by calculating the movement direction and speed changes of the target in consecutive frames. The speed calculation formula is:

[0016]

[0017] Among them, (x n ,y n ) and (x n-1 ,y n-1 ) is the coordinate of the target in the two frames, t is the time interval between frames;

[0018] S5. Alarm output: If abnormal behavior is detected, an alarm message is generated and sent to the monitoring center through the MQTT protocol. At the same time, the relevant equipment can be controlled to respond. The alarm level is calculated according to the vehicle type, and whether to trigger the alarm is determined according to the alarm level. At the same time, the push frequency of the alarm information is dynamically adjusted according to the alarm level. The higher the alarm level, the faster the push frequency. The formula is:

[0019] f=f0(1+kL)

[0020] Where f is the push frequency, f0 is the initial frequency, k is the proportional coefficient, and L is the alarm level;

[0021] S6. Result recording: The video frames, alarm information and related data at the time of alarm are saved to the local storage, the processed video frames are written to the video file, and a JSON description file of the alarm event is generated. At the same time, the saved data is encrypted using the AES encryption algorithm, and the key is a dynamic key generated based on the current time and the system unique identifier.

[0022] Optionally, in step S1, the adjustment of camera parameters includes adjustment of focal length, aperture and sensitivity. For focal length adjustment, when the frame rate fluctuates and the image is blurred, firstly, the clarity of the target in the image is analyzed to preliminarily determine whether the focal length is appropriate. The gradient operator is used to calculate the image gradient. The range of the image gradient amplitude is assumed to be [0, Gmax], where Gmax is the maximum value of the gradient amplitude. A rectangular area centered on the target is 1.5-2 times the size of the target. The difference between the maximum and minimum values ​​of the gradient amplitude is less than 0.1*Gmax, and the standard deviation of the gradient amplitude in the area is less than 0.05*Gmax. It is considered that the target blur is a focal length problem. The target distance information is calculated by the binocular vision algorithm. The specific calculation process is as follows: In the binocular vision system, the left and right cameras capture the target image at the same time. The internal parameters and external parameters of the camera are known. The internal parameters include the focal length f and the principal point coordinates. The external parameters include the rotation and translation relationship. The baseline length B is the distance between the optical centers of the two cameras. The horizontal coordinates of the target in the left and right camera images are x and y, respectively. l and x r , according to the formula:

[0023]

[0024] The target distance d is calculated. At the same time, the ambient light intensity is measured by light sensors. The light sensors are distributed in multiple positions around the camera. After collecting light intensity data in different directions, the average value is taken as the current ambient light intensity.

[0025] Optionally, in step S2, the threshold processing function T threshold Adaptively determine according to the grayscale histogram of the image. First, convert the image into a grayscale image, calculate the grayscale histogram, count the number of pixels at each grayscale level, and then calculate the grayscale mean μ and standard deviation σ, where the grayscale mean is:

[0026]

[0027] L is the total number of gray levels, which is 256, n i is the number of pixels with gray level i, N is the total number of pixels in the image; standard deviation:

[0028]

[0029] The threshold is set to μ+kσ, where k is an empirical value and k=2.

[0030] Optionally, in step S3, the weights w of feature maps of different scales i The information volume of the feature graph and its contribution to target detection are determined. The information volume is calculated by information entropy, and the information entropy formula is:

[0031]

[0032] Among them, p j is the probability of the jth eigenvalue appearing in the feature map. When calculating the information entropy of the feature map, the feature map is divided into multiple sub-regions, and the information entropy of each sub-region is calculated and the average value is taken as the information content of the feature map. For the contribution evaluation of target detection, the target detection experiment is carried out on the training set, and the detection accuracy of the target when using feature maps of different scales is recorded. After the information content and contribution are normalized, the weight w is determined by weighted summation. i .

[0033] Optionally, in step S4, in the target behavior trajectory analysis, for the occlusion of the target in a complex scene, the position and speed of the target are predicted and corrected by a Kalman filter algorithm. The Kalman filter algorithm is based on a linear system state equation, and its prediction equation is:

[0034]

[0035] in, is the predicted state vector at time k, is the estimated state vector at time k-1, A is the state transfer matrix, B is the control input matrix, u k-1 is the control input vector at time k-1. In target tracking, the state vector includes the position and velocity information of the target. When the target is blocked, it is predicted based on the previous state information and motion model. After the target reappears, the position of the new target in the image is used to pass the Kalman filter update equation:

[0036]

[0037] The predicted value is corrected, where is the estimated state vector at time k, K k is the Kalman gain, z k is the observation vector at time k, and H is the observation matrix.

[0038] Optionally, in step S5, the proportionality coefficient k is in the range of 0.2-0.6.

[0039] Optionally, in step S6, the encryption mode of the AES encryption algorithm adopts the CBC mode. In the CBC mode, the initial vector is a fixed-length vector randomly generated when the system starts, which is 128 bits and stored in the secure memory. During the encryption process, each plaintext block is XORed with the previous ciphertext block before being encrypted. Specifically, the first plaintext block is first XORed with the initial vector, and then encrypted using the AES encryption algorithm to obtain the first ciphertext block; subsequent plaintext blocks are XORed with the previous ciphertext block in turn and then encrypted.

[0040] Optionally, in step S2, the wavelet transform uses Haar wavelet, whose wavelet function ψ(t) and scale function The definition is as follows:

[0041]

[0042]

[0043] In image enhancement, the image is decomposed and reconstructed through two-dimensional Haar wavelet transform. One-dimensional Haar wavelet transform is performed on each row of the image, and then one-dimensional Haar wavelet transform is performed on each column of the transformed result to obtain four sub-bands of the image: low frequency, horizontal high frequency, vertical high frequency and diagonal high frequency.

[0044] Optionally, in step S4, for abnormal determination of target speed change, a speed change rate threshold is set, and the speed change rate formula is:

[0045]

[0046] Among them, v n is the velocity of the target at frame n, vn-1 is the speed of the target in the n-1th frame. The pedestrian threshold is 0.2 and the vehicle threshold is 0.5. When the threshold is exceeded, it means that the target has abnormal behavior and an alarm is triggered.

[0047] Optionally, in step S3, for updating the deep learning model, an incremental learning algorithm is used to update only the model parameters related to the newly appeared target features. Specifically, when a new target appears, the feature vector of the new target is first extracted, and the cosine similarity is used to calculate the similarity between the new target features and the existing features. The formula is:

[0048]

[0049] in, and They are the new target feature vector and the existing target feature vector respectively. The update parameters are determined according to the similarity value. If the similarity is lower than or equal to the threshold value of 0.6, it means that the new target feature is quite different from the existing feature, and more model parameters related to the feature need to be updated; if the similarity is higher than the threshold value of 0.6, only the relevant parameters need to be fine-tuned.

[0050] Compared with the prior art, the present invention has at least one of the following beneficial technical effects:

[0051] The image preprocessing step can improve image quality, and the multi-scale feature fusion algorithm and post-processing of detection results can further optimize target detection, improve the detection accuracy of small targets and targets in complex scenes, and reduce false detections and missed detections.

[0052] Alarm judgment is made by integrating multiple factors, including target location, category, confidence, behavior trajectory, etc. It can also deal with target occlusion, effectively distinguish between normal and abnormal behavior, and reduce false alarms and missed alarms.

[0053] Dynamically adjust the frequency of alarm information push according to the importance of the monitoring scene and the alarm level to ensure that abnormalities in important scenes are paid attention to in a timely manner, while avoiding interference from irrelevant alarm information and reasonably allocating monitoring resources.

[0054] Comprehensively record alarm-related information to facilitate subsequent analysis and auditing, and use encryption to protect data to ensure data security and privacy.

[0055] The incremental learning algorithm updates the model parameters according to the similarity between the new target features and the existing features, reducing the consumption of computing resources, enabling the model to quickly adapt to new targets and maintaining the accuracy of the system's detection of new target types.

[0056] The present invention improves the target detection accuracy, alarm precision, resource allocation rationality, data security, model update efficiency and adaptability by improving image preprocessing, target detection, alarm judgment, alarm output, result recording and model update, thereby comprehensively improving the performance of the video surveillance alarm system. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 The figure is a flow chart of a video surveillance alarm method based on deep learning. DETAILED DESCRIPTION

[0058] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0059] Example 1

[0060] like Figure 1 As shown, a video surveillance alarm method based on deep learning proposed in the present invention includes video acquisition, image preprocessing, target detection, alarm judgment, alarm output and result recording. Each step is described in detail below.

[0061] 1. Video acquisition: Use the camera to collect video streams in real time and save the video streams to the video buffer. At the same time, perform real-time frame rate stability analysis on the collected video streams. If the frame rate fluctuation exceeds the preset threshold, automatically adjust the camera parameters to stabilize the frame rate; the adjustment of camera parameters includes the adjustment of focal length, aperture and sensitivity. For focal length adjustment, when the frame rate fluctuates and the image is blurred, first analyze the clarity of the target in the image to preliminarily determine whether the focal length is appropriate. Use the gradient operator to calculate the image gradient. Assume that the range of the image gradient amplitude is [0, Gmax], where Gmax is the maximum value of the gradient amplitude. A rectangular area centered on the target The domain is 1.5-2 times the size of the target. The difference between the maximum and minimum values ​​of the gradient amplitude is less than 0.1*Gmax, and the standard deviation of the gradient amplitude in this area is less than 0.05*Gmax. The target blur is considered to be a focal length problem. The target distance information is calculated by the binocular vision algorithm. The specific calculation process is as follows: In the binocular vision system, the left and right cameras take the target image at the same time. The internal parameters and external parameters of the camera are known. The internal parameters include the focal length f and the coordinates of the principal point. The external parameters include the rotation and translation relationship. The baseline length B is the distance between the optical centers of the two cameras. The horizontal coordinates of the target in the left and right camera images are x l and x r , according to the formula:

[0062]

[0063] The target distance d is calculated. At the same time, the ambient light intensity is measured by light sensors. The light sensors are distributed in multiple positions around the camera. After collecting light intensity data in different directions, the average value is taken as the current ambient light intensity.

[0064] 2. Image preprocessing: Extract frames from the collected video stream, resize and convert the colors of each frame, calculate the width and height scaling factors of the image, encode the alarm area, and perform wavelet transform on the image to enhance image details. The formula is:

[0065] I enhanced =W -1 (T threshold (W(I)))

[0066] Where I is the original image, W is the wavelet transform, T threshold is the threshold processing function, I enhanced is the enhanced image; the threshold processing function T threshold According to the image's grayscale histogram, the image is first converted to a grayscale image, the grayscale histogram is calculated, and the number of pixels at each grayscale level is counted. Then, the grayscale mean μ and standard deviation σ are calculated, where the grayscale mean

[0067]

[0068] L is the total number of gray levels, which is 256, n i is the number of pixels with gray level i, N is the total number of pixels in the image; standard deviation

[0069]

[0070] The threshold is set to μ+kσ, where k is an empirical value and k=2. The wavelet transform uses Haar wavelet, whose wavelet function ψ(t) and scale function The definition is as follows:

[0071]

[0072] In image enhancement, the image is decomposed and reconstructed through two-dimensional Haar wavelet transform. One-dimensional Haar wavelet transform is performed on each row of the image, and then one-dimensional Haar wavelet transform is performed on each column of the transformed result to obtain four sub-bands of the image: low frequency, horizontal high frequency, vertical high frequency and diagonal high frequency.

[0073] In step S4, for abnormal determination of target speed change, a speed change rate threshold is set, and the speed change rate formula is:

[0074]

[0075] Among them, v nis the velocity of the target at frame n, v n-1 is the speed of the target in the n-1th frame. The pedestrian threshold is 0.2 and the vehicle threshold is 0.5. When the threshold is exceeded, it means that the target has abnormal behavior and an alarm is triggered.

[0076] Image quality can be effectively improved by calculating image width and height scaling factors, encoding alarm areas, and enhancing image details through wavelet transform. For example, under complex lighting conditions, wavelet transform can highlight the contours and textures of target objects, reduce image blur caused by uneven lighting, and make the target more clearly identifiable in the image, thereby providing better input for subsequent target detection and improving the accuracy of target detection.

[0077] 3. Target detection: Target detection is performed on the preprocessed image by calling the API interface of the deep learning model. The model outputs the detected target box, category and confidence, and post-processes the detection results, including removing targets of specific categories. At the same time, a multi-scale feature fusion algorithm is used to improve the accuracy of small target detection. The formula is:

[0078]

[0079] Among them, F i are feature maps of different scales, w i is the corresponding weight, F fused is the fused feature map; the weights w of feature maps of different scales i The information volume of the feature graph and its contribution to target detection are determined. The information volume is calculated by measuring the information entropy. The information entropy formula is:

[0080]

[0081] Among them, p j is the probability of the jth eigenvalue appearing in the feature map. When calculating the information entropy of the feature map, the feature map is divided into multiple sub-regions, and the information entropy of each sub-region is calculated and the average value is taken as the information content of the feature map. For the contribution evaluation of target detection, the target detection experiment is carried out on the training set, and the detection accuracy of the target when using feature maps of different scales is recorded. After the information content and contribution are normalized, the weight w is determined by weighted summation. i .

[0082] For the update of deep learning models, an incremental learning algorithm is used to update only the model parameters related to the newly appeared target features. Specifically, when a new target appears, the feature vector of the new target is first extracted. For the similarity calculation between the new target features and the existing features, the cosine similarity is used. The formula is:

[0083]

[0084] in, and They are the new target feature vector and the existing target feature vector respectively. The update parameters are determined according to the similarity value. If the similarity is lower than or equal to the threshold value of 0.6, it means that the new target feature is quite different from the existing feature, and more model parameters related to the feature need to be updated; if the similarity is higher than the threshold value of 0.6, only the relevant parameters need to be fine-tuned.

[0085] The target detection step uses a multi-scale feature fusion algorithm and post-processing of the detection results. Multi-scale feature fusion can effectively integrate feature information at different levels by reasonably determining the weights of feature maps of different scales, which can better capture the details and overall information of the target, especially for small target detection. Operations such as removing specific categories of targets in post-processing can reduce interference factors, further improve the accuracy of detection results, and reduce the false detection rate.

[0086] 4. Alarm judgment: According to the location, category and confidence of the target, combined with the preset alarm conditions, the number of frames in which the target is not detected is counted, and the vehicle information is reset when the set threshold is exceeded. At the same time, the target behavior trajectory analysis is introduced to judge abnormal behavior by calculating the movement direction and speed changes of the target in consecutive frames. The speed calculation formula is:

[0087]

[0088] Among them, (x n ,y n ) and (x n-1 ,y n-1 ) is the coordinate of the target in the two frames, and t is the time interval between frames; in the target behavior trajectory analysis, for the occlusion of the target in complex scenes, the Kalman filter algorithm is used to predict and correct the position and speed of the target. The Kalman filter algorithm is based on the linear system state equation, and its prediction equation is:

[0089]

[0090] in, is the predicted state vector at time k, is the estimated state vector at time k-1, A is the state transfer matrix, B is the control input matrix, u k-1 is the control input vector at time k-1. In target tracking, the state vector includes the position and velocity information of the target. When the target is blocked, it is predicted based on the previous state information and motion model. After the target reappears, the position of the new target in the image is used to pass the Kalman filter update equation:

[0091]

[0092] The predicted value is corrected, where is the estimated state vector at time k, K k is the Kalman gain, z k is the observation vector at time k, and H is the observation matrix.

[0093] The alarm judgment link not only considers the location, category and confidence of the target, but also introduces the analysis of the target behavior trajectory and the counting of the number of consecutive frames in which the target is not detected. By calculating the target speed and analyzing the speed change, combined with the change in the target's movement direction in consecutive frames, it is possible to more accurately judge whether the target behavior is abnormal. For example, in a personnel monitoring scenario, it is possible to distinguish between normal walking and abnormal behaviors such as sudden running, avoid false alarms or missed alarms caused by a single factor, and improve the accuracy of alarm judgment.

[0094] Improved ability to cope with complex scenarios

[0095] For complex scenarios such as target occlusion, the Kalman filter algorithm is used to predict and correct the target position and speed, making the alarm judgment reliable in complex situations. In vehicle tracking scenarios, even if the vehicle is partially blocked by other objects, the system can still accurately determine whether its behavior is abnormal by predicting its trajectory, enhancing the system's adaptability in complex environments.

[0096] 5. Alarm output: If abnormal behavior is detected, an alarm message will be generated and sent to the monitoring center through the MQTT protocol. At the same time, the relevant equipment can be controlled to respond. The alarm level is calculated according to the vehicle type, and whether to trigger the alarm is determined according to the alarm level. At the same time, the push frequency of the alarm information is dynamically adjusted according to the alarm level. The higher the alarm level, the faster the push frequency. The formula is:

[0097] f=f0(1+kL)

[0098] Among them, f is the push frequency, f0 is the initial frequency, k is the proportional coefficient, the range of the proportional coefficient k is 0.2-0.6, and L is the alarm level;

[0099] The alarm output step determines the frequency of alarm information push according to the importance level of the monitoring scene. For scenes with high importance levels, such as those storing important confidential documents or areas with dense personnel and high security requirements, the push frequency is increased by a suitable proportional coefficient when the alarm level is increased to ensure that abnormal situations in important scenes can be paid attention to in a timely manner. For relatively unimportant scenes, the frequency of unnecessary alarm information push is appropriately reduced to avoid monitoring personnel being disturbed by a large amount of irrelevant alarm information, thus achieving a reasonable allocation of monitoring resources.

[0100] The push frequency of alarm information is dynamically adjusted according to the alarm level, making the alarm output more flexible. High-level alarms can be conveyed to monitoring personnel more quickly, allowing them to prioritize serious abnormal situations, further improving the timeliness and effectiveness of security protection.

[0101] 6. Result recording: Save the video frames, alarm information and related data at the time of alarm to the local storage, write the processed video frames to the video file, and generate a JSON description file of the alarm event. At the same time, encrypt the saved data using the AES encryption algorithm, and the key is a dynamic key generated based on the current time and the system's unique identifier. The encryption mode of the AES encryption algorithm adopts the CBC mode. In the CBC mode, the initial vector is a fixed-length vector randomly generated when the system starts, which is 128 bits and stored in the secure memory. During the encryption process, each plaintext block is encrypted after being XORed with the previous ciphertext block. Specifically, the first plaintext block is first XORed with the initial vector, and then encrypted using the AES encryption algorithm to obtain the first ciphertext block; subsequent plaintext blocks are XORed with the previous ciphertext block in turn and then encrypted.

[0102] The result recording step not only saves the video frames, alarm information and related data at the time of the alarm to the local storage, but also writes the processed video frames into the video file and generates a JSON description file of the alarm event. This comprehensive recording method provides rich information for post-analysis and auditing, making it convenient to conduct a detailed review of abnormal events, find the cause and summarize experience.

[0103] The saved data is encrypted using the AES encryption algorithm with a dynamic key generated based on the current time and the system's unique identifier. This effectively prevents the data from being illegally obtained and tampered with during storage, ensuring the security and privacy of the monitoring data, which is especially important for monitoring scenarios involving sensitive information.

[0104] In terms of model updating, an incremental learning algorithm is used to determine the update parameters based on the similarity between the new target features and the existing features. This method only updates the model parameters related to the newly appeared target features, greatly reducing the consumption of computing resources. In a monitoring environment where new targets are constantly appearing, it can quickly adapt to new targets while avoiding frequent retraining of the entire model, improving the model update efficiency and the adaptability of the entire monitoring system, ensuring that the system can continuously and accurately detect new target types.

[0105] The above specific embodiments are only several optional embodiments of the present invention. Based on the technical solutions of the present invention and the relevant inspirations of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.

Claims

1. A video surveillance alarm method based on deep learning, characterized in that: The following steps are involved: S1, video acquisition: use the camera to collect video streams in real time and save the video streams to the video buffer. At the same time, perform real-time frame rate stability analysis on the collected video streams. If the frame rate fluctuation exceeds the preset threshold, the camera parameters are automatically adjusted to stabilize the frame rate. S2, image preprocessing: extract frames from the collected video stream, resize and convert the colors of each frame, calculate the width and height scaling factors of the image, encode the alarm area, and perform wavelet transform on the image to enhance image details. The formula is: I enhanced =W -1 (T threshold (W(I))) Where I is the original image, W is the wavelet transform, T threshold is the threshold processing function, I enhanced is the enhanced image; S3, Target detection: Target detection is performed on the preprocessed image by calling the API interface of the deep learning model. The model outputs the detected target box, category and confidence, and post-processes the detection results, including removing targets of specific categories. At the same time, a multi-scale feature fusion algorithm is used to improve the accuracy of small target detection. The formula is: Among them, F i are feature maps of different scales, w i is the corresponding weight, F fused is the fused feature map; S4, alarm judgment: According to the location, category and confidence of the target, combined with the preset alarm conditions, the number of frames in which the target is not detected is counted, and the vehicle information is reset when the set threshold is exceeded. At the same time, the target behavior trajectory analysis is introduced to judge abnormal behavior by calculating the movement direction and speed changes of the target in consecutive frames. The speed calculation formula is: Among them, (x n ,y n ) and (x n-1 ,y n-1 ) is the coordinate of the target in the two frames, t is the time interval between frames; S5. Alarm output: If abnormal behavior is detected, an alarm message is generated and sent to the monitoring center through the MQTT protocol. At the same time, the relevant equipment can be controlled to respond. The alarm level is calculated according to the vehicle type, and whether to trigger the alarm is determined according to the alarm level. At the same time, the push frequency of the alarm information is dynamically adjusted according to the alarm level. The higher the alarm level, the faster the push frequency. The formula is: f=f0(1+kL) Where f is the push frequency, f0 is the initial frequency, k is the proportional coefficient, and L is the alarm level; S6. Result recording: The video frames, alarm information and related data at the time of alarm are saved to the local storage, the processed video frames are written to the video file, and a JSON description file of the alarm event is generated. At the same time, the saved data is encrypted using the AES encryption algorithm, and the key is a dynamic key generated based on the current time and the system unique identifier.

2. According to the deep learning-based video surveillance alarm method of claim 1, it is characterized in that: In step S1, the adjustment of camera parameters includes the adjustment of focal length, aperture and sensitivity. For focal length adjustment, when the frame rate fluctuates and the image is blurred, firstly analyze the clarity of the target in the image to preliminarily judge whether the focal length is appropriate, and use the gradient operator to calculate the image gradient. Assume that the range of the image gradient amplitude is [0, Gmax], where Gmax is the maximum value of the gradient amplitude. A rectangular area centered on the target is 1.5-2 times the size of the target. The difference between the maximum and minimum values ​​of the gradient amplitude is less than 0.1*Gmax, and the standard deviation of the gradient amplitude in the area is less than 0.05*Gmax. It is considered that the target blur is a focal length problem. The target distance information is calculated by the binocular vision algorithm. The specific calculation process is as follows: In the binocular vision system, the left and right cameras capture the target image at the same time. The internal parameters and external parameters of the camera are known. The internal parameters include the focal length f and the coordinates of the principal point. The external parameters include the rotation and translation relationship. The baseline length B is the distance between the optical centers of the two cameras. The horizontal coordinates of the target in the left and right camera images are x and y, respectively. l and x r , according to the formula: The target distance d is calculated. At the same time, the ambient light intensity is measured by light sensors. The light sensors are distributed in multiple positions around the camera. After collecting light intensity data in different directions, the average value is taken as the current ambient light intensity.

3. The video surveillance alarm method based on deep learning according to claim 1 is characterized in that: In step S2, the threshold processing function T threshold Adaptively determine according to the grayscale histogram of the image. First, convert the image into a grayscale image, calculate the grayscale histogram, count the number of pixels at each grayscale level, and then calculate the grayscale mean μ and standard deviation σ, where the grayscale mean is: L is the total number of gray levels, which is 256, n i is the number of pixels with gray level i, and N is the total number of pixels in the image; Standard Deviation: The threshold is set to μ+kσ, where k is an empirical value and k=2.

4. The video surveillance alarm method based on deep learning according to claim 1 is characterized in that: In step S3, the weights w of feature maps of different scales are i The information volume of the feature graph and its contribution to target detection are determined. The information volume is calculated by information entropy, and the information entropy formula is: Among them, p j is the probability of the first j eigenvalues ​​appearing in the feature map. When calculating the information entropy of the feature map, the feature map is divided into multiple sub-regions, and the information entropy of each sub-region is calculated and the average value is taken as the information content of the feature map. For the contribution evaluation of target detection, the target detection experiment is carried out on the training set, and the detection accuracy of the target when using feature maps of different scales is recorded. After the information content and contribution are normalized, the weight w is determined by weighted summation. i .

5. The video surveillance alarm method based on deep learning according to claim 1 is characterized in that: In step S4, in the target behavior trajectory analysis, for the occlusion of the target in complex scenes, the position and speed of the target are predicted and corrected by the Kalman filter algorithm. The Kalman filter algorithm is based on the linear system state equation, and its prediction equation is: in, is the predicted state vector at time k, is the estimated state vector at time k-1, A is the state transfer matrix, B is the control input matrix, u k-1 is the control input vector at time k-1. In target tracking, the state vector includes the position and velocity information of the target. When the target is blocked, it is predicted based on the previous state information and motion model. After the target reappears, the position of the new target in the image is used to pass the Kalman filter update equation: The predicted value is corrected, where is the estimated state vector at time k, K k is the Kalman gain, z k is the observation vector at time k, and H is the observation matrix.

6. The video surveillance alarm method based on deep learning according to claim 1 is characterized in that: In step S5, the range of the proportionality coefficient k is 0.2-0.

6.

7. The video surveillance alarm method based on deep learning according to claim 1 is characterized in that: In step S6, the encryption mode of the AES encryption algorithm adopts the CBC mode. In the CBC mode, the initial vector is a fixed-length vector randomly generated when the system starts, which is 128 bits and stored in the secure memory. During the encryption process, each plaintext block is XORed with the previous ciphertext block before being encrypted. Specifically, the first plaintext block is first XORed with the initial vector, and then encrypted using the AES encryption algorithm to obtain the first ciphertext block; subsequent plaintext blocks are XORed with the previous ciphertext block in turn and then encrypted.

8. The video surveillance alarm method based on deep learning according to claim 1 is characterized in that: In step S2, the wavelet transform uses Haar wavelet, whose wavelet function ψ(t) and scale function The definition is as follows: In image enhancement, the image is decomposed and reconstructed through two-dimensional Haar wavelet transform. One-dimensional Haar wavelet transform is performed on each row of the image, and then one-dimensional Haar wavelet transform is performed on each column of the transformed result to obtain four sub-bands of the image: low frequency, horizontal high frequency, vertical high frequency and diagonal high frequency.

9. The video surveillance alarm method based on deep learning according to claim 1 is characterized in that: In step S4, for abnormal determination of target speed change, a speed change rate threshold is set, and the speed change rate formula is: Among them, v n is the velocity of the target at frame n, v n-1 is the speed of the target in the n-1th frame. The pedestrian threshold is 0.2 and the vehicle threshold is 0.

5. When the threshold is exceeded, it means that the target has abnormal behavior and an alarm is triggered.

10. The video surveillance alarm method based on deep learning according to claim 1, characterized in that: In step S3, for updating the deep learning model, an incremental learning algorithm is used to update only the model parameters related to the newly appeared target features. Specifically, when a new target appears, the feature vector of the new target is first extracted. For the similarity calculation between the new target features and the existing features, the cosine similarity is used, and the formula is: in, and They are the new target feature vector and the existing target feature vector respectively. The update parameters are determined according to the similarity value. If the similarity is lower than or equal to the threshold value of 0.6, it means that the new target feature is quite different from the existing feature, and more model parameters related to the feature need to be updated; if the similarity is higher than the threshold value of 0.6, only the relevant parameters need to be fine-tuned.

Images