Method for detecting anomaly in target-oriented monitoring video

Abstract

The invention relates to the technical field of video anomaly detection, and discloses a target-oriented anomaly detection method in a monitoring video, and the method comprises the steps: S1, deploying a plurality of cameras in a monitoring region, achieving the synchronization and fusion of multi-channel video data, and carrying out the real-time target detection and tracking of multi-view video data through an AI algorithm, estimating or filling missing information caused by shielding; s2, receiving the output of the target detection and tracking module, judging whether shielding occurs or not, designing a specific shielding detection algorithm, identifying when a shielding phenomenon occurs in the video, and estimating or filling missing information caused by shielding; and S3, through multi-sensor fusion, providing additional clues when single vision is blocked, dynamically adjusting a camera angle or a focal length to adjust a visual angle, and identifying an abnormal action or event of a monitored target based on an abnormal behavior detection model of deep learning.

Description

Technical Field

[0001] The present invention relates to the technical field of video anomaly detection, and in particular to a method for detecting anomalies in a surveillance video facing a target. Background Art

[0002] As the social security situation becomes increasingly severe, the demand for video surveillance is increasing, whether it is public security, corporate security or family security. In particular, in the fields of urban security, traffic management, commercial facilities, and protection of important infrastructure, early detection of abnormal behavior is particularly important.

[0003] In order to deal with the problem of limited vision of a single camera due to occlusion, modern surveillance systems have extremely high requirements for real-time performance and accuracy. Occlusion anomaly detection not only needs to quickly identify occlusion events, but also needs to respond appropriately when occlusion occurs, such as dispatching other cameras or taking remedial measures to ensure that the target is always under effective monitoring.

[0004] After searching, the invention patent with Chinese publication number CN105184818A discloses a video surveillance abnormal behavior detection method and its detection system. Compared with the existing technology, the invention patent with Chinese publication number CN105184818A uses video image sequence as original data and establishes a background model using a mixed Gaussian model, which can reduce the impact of lighting changes and slight background changes in the scene on detection; at the same time, it also refines the classification of abnormal behavior, greatly increasing the accuracy of detection.

[0005] However, while reducing the impact of the environment and improving detection accuracy, occlusion often occurs in the surveillance video part of the anomaly detection system, which will cause the surveillance video to directly lose target information and also affect the system's anomaly detection results. Therefore, a method for anomaly detection in surveillance video facing the target is proposed. Summary of the invention

[0006] The purpose of the present invention is to solve the shortcoming in the prior art that the occlusion phenomenon will cause the monitoring video to directly lose the target information, and to propose an anomaly detection method in the monitoring video facing the target.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for detecting anomalies in a surveillance video facing a target, wherein the anomaly detection steps are as follows: S1: deploying multiple cameras in a surveillance area to achieve synchronization and fusion of multi-channel video data, and using an AI algorithm to perform real-time target detection and tracking on multi-view video data, and to estimate or fill in missing information caused by occlusion;

[0009] S2: Receive the output of the target detection and tracking module, determine whether occlusion occurs, and design a specific occlusion detection algorithm to identify when occlusion occurs in the video and estimate or fill in the missing information caused by occlusion;

[0010] S3: Through multi-sensor fusion, it provides additional clues when a single vision is blocked, dynamically adjusts the camera angle or focal length to adjust the viewing angle, and identifies abnormal actions or events of the monitored target based on the abnormal behavior detection model based on deep learning;

[0011] S4: Based on the abnormal behavior recognition module, integrate the previous decision information, comprehensively judge whether abnormal behavior occurs, and rely on the judgment result of the abnormal behavior recognition module to execute system-level decision.

[0012] In S1, there are multi-view monitoring and synchronization parts. Multiple cameras are deployed in the same monitoring area. When one view is blocked, other views can still provide target information. Video stream synchronization technology is introduced to fuse the images or video clips captured by each camera in real time. NTP is used to provide a unified time reference source for all cameras. The offset and drift between the local clock and the remote reference clock are calculated based on an order linear model, where:

[0013] Clock offset calculation: offset = (T2-T1)-(T3-T4);

[0014] Clock frequency deviation calculation: drift = (T3-T2)-(T4-T1);

[0015] Among them, (T1, T2) and (T3, T4) represent the local time and remote time in the two timestamp request-response processes respectively. Then, when each camera captures a video frame, an accurate timestamp is attached to it. During the video encoding and transmission process, the timestamp information is retained for synchronization processing.

[0016] In S1, there are target detection and tracking parts. Based on the AI ​​target detection algorithm, the position and size of all objects in the monitoring area are located in real time. The target is tracked with the long-term dependent sequence model. Even if the target is temporarily occluded, it can be quickly re-associated after recovery. Based on the association metric in target tracking, it is used for frame matching in target tracking to determine whether the targets in the previous frame and the current frame correspond. Among them:

[0017] State prediction: x(k|k-1)=A*x(k-1|k-1)+Bu(k)

[0018] x(k|k-1) represents the predicted value of the state at the kth time based on the information at the k-1th time, that is, the prior estimate;

[0019] x(k-1|k-1) is the state estimate obtained at the k-1th time, which is the best known estimate;

[0020] A is the system dynamic matrix, which describes the change of state from one time step to the next time step;

[0021] u(k) is the control input or external influencing factor at the kth time;

[0022] B is the input matrix, which shows how the control input affects the system state;

[0023] When the state at the previous moment and the current control input are known, the kinematic model of the system is used to predict the state at the current moment, and then:

[0024] State covariance prediction: P(k|k-1)=A*P(k-1|k-1)*A^T+Q

[0025] P(k|k-1) is the covariance matrix of the state prediction at the kth moment, reflecting the uncertainty of the predicted state;

[0026] P(k-1|k-1) is the covariance matrix of the estimated state at the k-1th time, representing the uncertainty of the state at that time;

[0027] A^T is the transpose of matrix A;

[0028] Q is the process noise covariance matrix, which quantifies the inaccuracy of state prediction due to the random process noise inside the system;

[0029] By predicting the state covariance at the next moment, the state uncertainty at the previous moment and the dynamic characteristics of the system, plus the process noise of the system itself, the uncertainty of the current predicted state is obtained, and then the position of the target is continuously predicted based on past observations and model knowledge, and the reliability of the prediction is evaluated. The target position is accurately estimated and tracked by updating and correcting it through actual observation data.

[0030] In S2, there is an occlusion detection and estimation part, which receives the output of the target detection and tracking module, determines the occlusion state, collects the data output by the target detection and tracking module, including the position, size, and motion trajectory information of the target, forms continuous time series data with the tracking information of the same target in different time periods, calculates the target motion speed, acceleration, and position change index by using the motion information of two or more consecutive frames of the target, and determines whether the change of the target position and size between adjacent frames exceeds the threshold. The positions of the target in two consecutive frames are (x1, y1) and (x2, y2), respectively, and the time interval is Δt. The speed can be approximated as: v = ((x2-x1), (y2-y1)) / Δt;

[0031] The occlusion detection threshold θ is defined. When the change of the target area in consecutive frames is greater than the threshold, occlusion is determined to have occurred. The sudden disappearance of the target or the unnatural break of its motion trajectory indicates that occlusion has occurred.

[0032] In S2, a deep neural network model is trained based on historical occlusion cases to learn the characteristic patterns when occlusion occurs. The feature vector output by the target detection and tracking module is input, and the model outputs the probability of occlusion or the occlusion status label. A classifier is trained to output the probability of occlusion or not:

[0033] P(occurrence_of_occlusion|input_features)

[0034] When this probability is higher than the preset threshold, occlusion is determined to have occurred. When there are multiple targets, the relative position relationship between the targets is analyzed through the association algorithm, and the IoU value of the target between the two frames is calculated. When it is lower than the threshold, occlusion occurs: IoU(A,B)=|A∩B| / |A∪B|

[0035] A and B represent the rectangular bounding boxes of the target in the two frames, respectively;

[0036] |A∩B| represents the area of ​​the intersection of two bounding boxes, the part where the two bounding boxes overlap;

[0037] |A∪B| represents the area of ​​the union of two bounding boxes, which is the total area of ​​the two bounding boxes plus their overlapping area;

[0038] Based on the association relationship, if a target loses tracking for a short period of time but reappears later, and its movement trajectory and speed are consistent with those before occlusion, it is judged that occlusion occurs.

[0039] In S3, based on the occlusion detection in S2, the multi-sensor fusion mechanism is immediately started to synchronously collect and fuse data from different sensors to compensate for the loss of visual information caused by occlusion. According to the occlusion situation and multi-sensor data, the relevant cameras are instructed to adjust the shooting angle or zoom to capture the occluded target object from a new perspective. The video stream processing module provides real-time feedback on the camera's field of view change effect to find the best observation position or range.

[0040] In S3, based on surveillance video data, including normal behavior and various abnormal behavior samples, such as intrusion, fighting, falling, and fast running, video data is obtained and finely labeled to identify the normal or abnormal behavior category of the target in each video, as well as the specific type of abnormal behavior, extract key frame feature data, design a deep learning model for normal behavior detection based on the feature data, input the extracted features into the model for training, optimize the model parameters through back propagation, and distinguish normal behavior from abnormal behavior;

[0041] Multi-sensor fusion technology integrates sensor information other than vision, converts these multimodal data into a unified feature representation, and fuses them with visual features to form a comprehensive feature vector. When occlusion occurs, the comprehensive features generated by multi-sensor fusion are input into a pre-trained deep learning model for real-time analysis. The model outputs a probability score for abnormal behavior. When the score exceeds the preset threshold, the system will trigger an alarm.

[0042] In S4, the results output by the deep learning abnormal behavior detection model are combined with the conclusions of the previous occlusion monitoring module. The information provided by the multi-sensor fusion during the occlusion period also indicates the existence of abnormal behavior. The occurrence of abnormal behavior is comprehensively judged, and the abnormal probability score and behavior category output by the abnormal behavior recognition module are integrated to form an abnormal behavior report. The severity of the current abnormal behavior is quantitatively evaluated. If the abnormal behavior occurs in a specific sensitive period or area, the system will assign a higher alert level.

[0043] The present invention has the following beneficial effects:

[0044] 1. In the present invention, through the AI-driven occlusion detection and estimation technology, the occlusion area can be discovered and located in real time, and the target state after occlusion can be predicted through the algorithm to reduce the interference of occlusion on abnormal detection. Even in the case of occlusion, the tracking of the target can be quickly restored with the help of historical trajectory prediction and algorithm optimization, which greatly reduces the risk of tracking interruption caused by occlusion.

[0045] 2. In the present invention, by combining stereoscopic vision and depth perception technology, the system can use 3D information to bypass obstructions on the plane, provide richer scene information, and automatically adjust the camera angle or switch to an unobstructed backup camera when occlusion occurs, so as to maintain continuous and effective monitoring of the target, thereby enhancing the ability to recognize obstructed targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A method step diagram of a method for detecting anomalies in a surveillance video facing a target proposed by the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Embodiment 1

[0049] like Figure 1As shown, the present invention proposes a method for detecting anomalies in a surveillance video facing a target, and the anomaly detection steps are as follows: S1: deploy multiple cameras in the surveillance area to achieve synchronization and fusion of multi-channel video data, and use AI algorithms to perform real-time target detection and tracking on multi-view video data, and estimate or fill in missing information caused by occlusion;

[0050] S2: Receive the output of the target detection and tracking module, determine whether occlusion occurs, and design a specific occlusion detection algorithm to identify when occlusion occurs in the video and estimate or fill in the missing information caused by occlusion;

[0051] S3: Through multi-sensor fusion, it provides additional clues when a single vision is blocked, dynamically adjusts the camera angle or focal length to adjust the viewing angle, and identifies abnormal actions or events of the monitored target based on the abnormal behavior detection model based on deep learning;

[0052] S4: Based on the abnormal behavior recognition module, integrate the previous decision information, comprehensively judge whether abnormal behavior occurs, and rely on the judgment result of the abnormal behavior recognition module to execute system-level decision.

[0053] In S1, there are multi-view monitoring and synchronization parts. Multiple cameras are deployed in the same monitoring area. When one view is blocked, other views can still provide target information. Video stream synchronization technology is introduced to fuse the images or video clips captured by each camera in real time. NTP is used to provide a unified time reference source for all cameras. The offset and drift between the local clock and the remote reference clock are calculated based on an order linear model, where:

[0054] Clock offset calculation: offset = (T2-T1)-(T3-T4);

[0055] Clock frequency deviation calculation: drift = (T3-T2)-(T4-T1);

[0056] Among them, (T1, T2) and (T3, T4) represent the local time and remote time in the two timestamp request-response processes respectively. Then, when each camera captures a video frame, an accurate timestamp is attached to it. During the video encoding and transmission process, the timestamp information is retained for synchronization processing.

[0057] In S1, there are target detection and tracking parts. Based on the AI ​​target detection algorithm, the position and size of all objects in the monitoring area are located in real time. The target is tracked with the long-term dependent sequence model. Even if the target is temporarily occluded, it can be quickly re-associated after recovery. Based on the association metric in target tracking, it is used for frame matching in target tracking to determine whether the targets in the previous frame and the current frame correspond. Among them:

[0058] State prediction: x(k|k-1)=A*x(k-1|k-1)+Bu(k)

[0059] x(k|k-1) represents the predicted value of the state at the kth time based on the information at the k-1th time, that is, the prior estimate;

[0060] x(k-1|k-1) is the state estimate obtained at the k-1th time, which is the best known estimate;

[0061] A is the system dynamic matrix, which describes the change of state from one time step to the next time step;

[0062] u(k) is the control input or external influencing factor at the kth time;

[0063] B is the input matrix, which shows how the control input affects the system state;

[0064] When the state at the previous moment and the current control input are known, the kinematic model of the system is used to predict the state at the current moment, and then:

[0065] State covariance prediction: P(k|k-1)=A*P(k-1|k-1)*A^T+Q

[0066] P(k|k-1) is the covariance matrix of the state prediction at the kth moment, reflecting the uncertainty of the predicted state;

[0067] P(k-1|k-1) is the covariance matrix of the estimated state at the k-1th time, representing the uncertainty of the state at that time;

[0068] A^T is the transpose of matrix A;

[0069] Q is the process noise covariance matrix, which quantifies the inaccuracy of state prediction due to the random process noise inside the system;

[0070] By predicting the state covariance at the next moment, the state uncertainty at the previous moment and the dynamic characteristics of the system, plus the process noise of the system itself, the uncertainty of the current predicted state is obtained, and then the position of the target is continuously predicted based on past observations and model knowledge, and the reliability of the prediction is evaluated. The target position is accurately estimated and tracked by updating and correcting it through actual observation data.

[0071] In S2, there is an occlusion detection and estimation part, which receives the output of the target detection and tracking module, determines the occlusion state, collects the data output by the target detection and tracking module, including the position, size, and motion trajectory information of the target, forms continuous time series data with the tracking information of the same target in different time periods, calculates the target motion speed, acceleration, and position change index by using the motion information of two or more consecutive frames of the target, and determines whether the change of the target position and size between adjacent frames exceeds the threshold. The positions of the target in two consecutive frames are (x1, y1) and (x2, y2), respectively, and the time interval is Δt. The speed can be approximated as: v = ((x2-x1), (y2-y1)) / Δt;

[0072] The occlusion detection threshold θ is defined. When the change of the target area in consecutive frames is greater than the threshold, occlusion is determined to have occurred. The sudden disappearance of the target or the unnatural break of its motion trajectory indicates that occlusion has occurred.

[0073] In this embodiment, in the surveillance video, when tracking a person's walking path in a corridor, the state is the two-dimensional coordinate position (x, y) and velocity (vx, vy) of the target between consecutive frames;

[0074] In x(k|k-1)=A*x(k-1|k-1)+Bu(k), the current state is predicted by the kinematic model of the system when the state at the previous moment and the current control input are known. In a video surveillance device, if we are tracking the position and velocity of an object, it contains position and velocity components, and will describe how the object moves according to its current velocity and the velocity change caused by external forces;

[0075] P(k|k-1)=A*P(k-1|k-1)*A^T+Q is used to predict the state covariance at the next moment, that is, based on the state uncertainty at the previous moment and the dynamic characteristics of the system, plus the process noise of the system itself, to estimate the uncertainty of the current predicted state;

[0076] When predicting the next position of the target, if the actual observed target position is significantly different from the predicted position, that is, the residual between the observed position and the predicted position is large, it means that an abnormal situation has occurred and abnormal occlusion has occurred in the video.

[0077] Embodiment 2

[0078] like Figure 1 As shown, based on the first embodiment, in S2, a deep neural network model is trained based on historical occlusion cases to learn the characteristic pattern when occlusion occurs, and the characteristic vector output by the target detection and tracking module is input. The model outputs the probability of occlusion or the occlusion state label, and a classifier is trained to output the probability of occlusion or not:

[0079] P(occurrence_of_occlusion|input_features)

[0080] When this probability is higher than the preset threshold, occlusion is determined to have occurred. When there are multiple targets, the relative position relationship between the targets is analyzed through the association algorithm, and the IoU value of the target between the two frames is calculated. When it is lower than the threshold, occlusion occurs: IoU(A,B)=|A∩B| / |A∪B|

[0081] A and B represent the rectangular bounding boxes of the target in the two frames, respectively;

[0082] |A∩B| represents the area of ​​the intersection of two bounding boxes, the part where the two bounding boxes overlap;

[0083] |A∪B| represents the area of ​​the union of two bounding boxes, which is the total area of ​​the two bounding boxes plus their overlapping area;

[0084] Based on the association relationship, if a target loses tracking for a short period of time but reappears later, and its movement trajectory and speed are consistent with those before occlusion, it is judged that occlusion occurs.

[0085] In S3, based on the occlusion detection in S2, the multi-sensor fusion mechanism is immediately started to synchronously collect and fuse data from different sensors to compensate for the loss of visual information caused by occlusion. According to the occlusion situation and multi-sensor data, the relevant cameras are instructed to adjust the shooting angle or zoom to capture the occluded target object from a new perspective. The video stream processing module provides real-time feedback on the camera's field of view change effect to find the best observation position or range.

[0086] In S3, based on surveillance video data, including normal behavior and various abnormal behavior samples, such as intrusion, fighting, falling, and fast running, video data is obtained and finely labeled to identify the normal or abnormal behavior category of the target in each video, as well as the specific type of abnormal behavior, extract key frame feature data, design a deep learning model for normal behavior detection based on the feature data, input the extracted features into the model for training, optimize the model parameters through back propagation, and distinguish normal behavior from abnormal behavior;

[0087] Multi-sensor fusion technology integrates sensor information other than vision, converts these multimodal data into a unified feature representation, and fuses them with visual features to form a comprehensive feature vector. When occlusion occurs, the comprehensive features generated by multi-sensor fusion are input into a pre-trained deep learning model for real-time analysis. The model outputs a probability score for abnormal behavior. When the score exceeds the preset threshold, the system will trigger an alarm.

[0088] In S4, the results output by the deep learning abnormal behavior detection model are combined with the conclusions of the previous occlusion monitoring module. The information provided by the multi-sensor fusion during the occlusion period also indicates the existence of abnormal behavior. The occurrence of abnormal behavior is comprehensively judged, and the abnormal probability score and behavior category output by the abnormal behavior recognition module are integrated to form an abnormal behavior report. The severity of the current abnormal behavior is quantitatively evaluated. If the abnormal behavior occurs in a specific sensitive period or area, the system will assign a higher alert level.

[0089] In this embodiment, P (occurrence_of_occlusion|input_features) is a probability representation method, which represents the probability of target occlusion (occurrence_of_occlusion) under the condition of a series of feature information. The "|" symbol represents the conditional probability, that is, the probability of an event occurring under the condition of knowing certain features.

[0090] In the target detection and tracking system, input_features includes the following features:

[0091] The change in position, velocity, and acceleration of the target between consecutive frames;

[0092] Changes in the outline or area of ​​the target;

[0093] Changes in the grayscale value, texture or color characteristics of pixels around the target;

[0094] If there is depth information, it also includes the depth value of the target and its changes;

[0095] High-level abstract features extracted by machine learning or deep learning models;

[0096] Then, the deep learning model is used to learn and analyze these features, and the model finally outputs the probability of target occlusion. When this probability exceeds the preset threshold, the system determines that an occlusion event may occur, and then takes corresponding response strategies, such as switching perspectives and adjusting tracking algorithms.

[0097] At the same time, in IoU(A,B)=|A∩B / |A∪B|, when the target IoU value between two frames is low, it means that the bounding boxes of the targets in the two frames overlap less. This may be because the target is occluded or the target moves violently and leaves the original tracking range. Therefore, if the IoU value is lower than the preset threshold, it can be preliminarily judged that the target is occluded.

[0098] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting anomalies in a surveillance video facing a target, characterized in that: The steps of anomaly detection are as follows: S1: deploy multiple cameras in the monitoring area to achieve synchronization and fusion of multi-channel video data, and use AI algorithms to perform real-time target detection and tracking on multi-view video data, and estimate or fill in the missing information caused by occlusion; S2: Receive the output of the target detection and tracking module, determine whether occlusion occurs, and design a specific occlusion detection algorithm to identify when occlusion occurs in the video; S3: Through multi-sensor fusion, it provides additional clues when a single vision is blocked, dynamically adjusts the camera angle or focal length to adjust the viewing angle, and identifies abnormal actions or events of the monitored target based on the abnormal behavior detection model based on deep learning; S4: Based on the abnormal behavior recognition module, integrate the previous decision information, comprehensively judge whether abnormal behavior occurs, and rely on the judgment result of the abnormal behavior recognition module to execute system-level decision.

2. The method for detecting anomalies in a surveillance video facing a target according to claim 1, characterized in that: In S1, there are multi-view monitoring and synchronization parts. Multiple cameras are deployed in the same monitoring area. When one view is blocked, other views can still provide target information. Video stream synchronization technology is introduced to fuse the images or video clips captured by each camera in real time. NTP is used to provide a unified time reference source for all cameras. The offset and drift between the local clock and the remote reference clock are calculated based on an order linear model, where: Clock offset calculation: offset = (T2-T1)-(T3-T4); Clock frequency deviation calculation: drift = (T3-T2)-(T4-T1); Among them, (T1, T2) and (T3, T4) represent the local time and remote time in the two timestamp request-response processes respectively. Then, when each camera captures a video frame, an accurate timestamp is attached to it. During the video encoding and transmission process, the timestamp information is retained for synchronization processing.

3. The method for detecting anomalies in a surveillance video facing a target according to claim 1, characterized in that: In S1, there are target detection and tracking parts. Based on the AI ​​target detection algorithm, the position and size of all objects in the monitoring area are located in real time. The target is tracked with the long-term dependent sequence model. Even if the target is temporarily occluded, it can be quickly re-associated after recovery. Based on the association metric in target tracking, it is used for frame matching in target tracking to determine whether the targets in the previous frame and the current frame correspond. Among them: State prediction: x(k|k-1)=A*x(k-1|k-1)+Bu(k) x(k|k-1) represents the predicted value of the state at the kth time based on the information at the k-1th time, that is, the prior estimate; x(k-1|k-1) is the state estimate obtained at the k-1th time, which is the best known estimate; A is the system dynamic matrix, which describes the change of state from one time step to the next time step; u(k) is the control input or external influencing factor at the kth time; B is the input matrix, which shows how the control input affects the system state; When the state at the previous moment and the current control input are known, the kinematic model of the system is used to predict the state at the current moment, and then: State covariance prediction: P(k|k-1)=A*P(k-1|k-1)*A^T+Q P(k|k-1) is the covariance matrix of the state prediction at the kth moment, reflecting the uncertainty of the predicted state; P(k-1|k-1) is the covariance matrix of the estimated state at the k-1th time, representing the uncertainty of the state at that time; A^T is the transpose of matrix A; Q is the process noise covariance matrix, which quantifies the inaccuracy of state prediction due to the random process noise inside the system; By predicting the state covariance at the next moment, the state uncertainty at the previous moment and the dynamic characteristics of the system, plus the process noise of the system itself, the uncertainty of the current predicted state is obtained, and then the position of the target is continuously predicted based on past observations and model knowledge, and the reliability of the prediction is evaluated. The target position is accurately estimated and tracked by updating and correcting it through actual observation data.

4. The method for detecting anomalies in a surveillance video facing a target according to claim 1, characterized in that: In S2, there is an occlusion detection and estimation part, which receives the output of the target detection and tracking module, determines the occlusion state, collects the data output by the target detection and tracking module, including the position, size, and motion trajectory information of the target, forms continuous time series data with the tracking information of the same target in different time periods, calculates the target motion speed, acceleration, and position change index by using the motion information of two or more consecutive frames of the target, and determines whether the change of the target position and size between adjacent frames exceeds the threshold. The positions of the target in two consecutive frames are (x1, y1) and (x2, y2), respectively, and the time interval is Δt. The speed can be approximated as: v = ((x2-x1), (y2-y1)) / Δt; The occlusion detection threshold θ is defined. When the change of the target area in consecutive frames is greater than the threshold, occlusion is determined to have occurred. The sudden disappearance of the target or the unnatural break of its motion trajectory indicates that occlusion has occurred.

5. The method for detecting anomalies in a surveillance video facing a target according to claim 1, characterized in that: In S2, a deep neural network model is trained based on historical occlusion cases to learn the characteristic patterns when occlusion occurs. The feature vector output by the target detection and tracking module is input, and the model outputs the probability of occlusion or the occlusion status label. A classifier is trained to output the probability of occlusion or not: P(occurrence_of_occlusion|input_features) When this probability is higher than the preset threshold, occlusion is determined to have occurred. When there are multiple targets, the relative position relationship between the targets is analyzed through the association algorithm, and the IoU value of the target between the two frames is calculated. When it is lower than the threshold, occlusion occurs: IoU(A,B)=|A∩B| / |A∪B| A and B represent the rectangular bounding boxes of the target in the two frames, respectively; |A∩B| represents the area of ​​the intersection of two bounding boxes, the part where the two bounding boxes overlap; |A∪B| represents the area of ​​the union of two bounding boxes, which is the total area of ​​the two bounding boxes plus their overlapping area; Based on the association relationship, if a target loses tracking for a short period of time but reappears later, and its movement trajectory and speed are consistent with those before occlusion, it is judged that occlusion occurs.

6. The method for detecting anomalies in a surveillance video facing a target according to claim 1, characterized in that: In S3, based on the occlusion detection in S2, the multi-sensor fusion mechanism is immediately started to synchronously collect and fuse data from different sensors to compensate for the loss of visual information caused by occlusion. According to the occlusion situation and multi-sensor data, the relevant cameras are instructed to adjust the shooting angle or zoom to capture the occluded target object from a new perspective. The video stream processing module provides real-time feedback on the camera's field of view change effect to find the best observation position or range.

7. The method for detecting anomalies in a surveillance video facing a target according to claim 1, characterized in that: In S3, based on surveillance video data, including normal behavior and various abnormal behavior samples, such as intrusion, fighting, falling, and fast running, video data is obtained and finely labeled to identify the normal or abnormal behavior category of the target in each video, as well as the specific type of abnormal behavior, extract key frame feature data, design a deep learning model for normal behavior detection based on the feature data, input the extracted features into the model for training, optimize the model parameters through back propagation, and distinguish normal behavior from abnormal behavior; Multi-sensor fusion technology integrates sensor information other than vision, converts these multimodal data into a unified feature representation, and fuses them with visual features to form a comprehensive feature vector. When occlusion occurs, the comprehensive features generated by multi-sensor fusion are input into a pre-trained deep learning model for real-time analysis. The model outputs a probability score for abnormal behavior. When the score exceeds the preset threshold, the system will trigger an alarm.

8. The method for detecting anomalies in a surveillance video facing a target according to claim 1, characterized in that: In S4, the results output by the deep learning abnormal behavior detection model are combined with the conclusions of the previous occlusion monitoring module. The information provided by the multi-sensor fusion during the occlusion period also indicates the existence of abnormal behavior. The occurrence of abnormal behavior is comprehensively judged, and the abnormal probability score and behavior category output by the abnormal behavior recognition module are integrated to form an abnormal behavior report. The severity of the current abnormal behavior is quantitatively evaluated. If the abnormal behavior occurs in a specific sensitive period or area, the system will assign a higher alert level.

Images