Real-time video anti-theft monitoring method and system based on Internet of Things

Abstract

The invention discloses a real-time video anti-theft monitoring method and system based on the Internet of Things, and relates to the technical field of data processing, and the method comprises the steps: obtaining monitoring data in a historical preset time period, and calculating a behavior purposiveness index of each dynamic target in the monitoring data; based on the behavior purposiveness index, determining a suspicious target in each dynamic target, and calculating a risk mapping index of the motion behavior of the suspicious target to a risk area; based on the risk mapping index and the behavior purposiveness index, determining a theft risk assessment value of the suspicious target for the risk area; and performing anti-theft monitoring on the suspicious target based on the theft risk assessment value. According to the invention, the false alarm rate and the missing report rate of the monitoring system for risks are reduced.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a real-time video anti-theft monitoring method and system based on the Internet of Things. Background Art

[0002] Nowadays, smart devices and sensors combined with Internet of Things technology can collect video and environmental data in real time, promptly issue alarms and handle potential security threats, thereby improving monitoring efficiency and safety, and are widely used to solve public security problems.

[0003] Currently, the IoT-based video anti-theft surveillance systems on the market mainly achieve accurate alarms by fusing multi-source data and using deep learning models (such as target detection and behavior recognition) to dynamically assess security risk levels. However, actual security risk assessment is limited by the highly nonlinear and coupled relationship between the behavioral characteristics of each target (moving, staying, carrying items, etc.) and security risks, which will lead to high false alarm and missed alarm rates of the monitoring system. Summary of the Invention

[0004] The main purpose of this application is to provide a real-time video anti-theft monitoring method and system based on the Internet of Things, aiming to solve the technical problem in related technologies that the security risk level is dynamically evaluated through a deep learning model, which is limited by the highly nonlinear and coupled relationship between the behavioral characteristics of each target and the security risk, resulting in a high false alarm rate and missed alarm rate of the monitoring system.

[0005] To achieve the above objectives, the present invention provides a real-time video anti-theft monitoring method based on the Internet of Things, comprising:

[0006] Obtain monitoring data within a preset historical time period and calculate the behavioral purpose indicators of each dynamic target in the monitoring data;

[0007] Based on the behavioral purpose indicators, the suspicious targets among the dynamic targets are identified, and the risk mapping indicators of the movement behaviors of the suspicious targets for the risk areas are calculated;

[0008] Determine the theft risk assessment value of suspicious targets in risk areas based on risk mapping indicators and behavioral purpose indicators;

[0009] Based on the theft risk assessment value, anti-theft monitoring is carried out on suspicious targets.

[0010] In one possible implementation of the present application, calculating the behavioral purposefulness index of each dynamic target in the monitoring data includes:

[0011] Based on the preset target detection algorithm, each static frame in the monitoring data is marked and selected to obtain multiple dynamic targets;

[0012] Calculate the behavioral activity of each dynamic target and filter the scene data frames when the dynamic target is blocked in the monitoring data;

[0013] When the area of ​​the dynamic target blocked in the scene data frame is greater than a preset area threshold, it is determined that the dynamic target is in an occlusion obstacle scene, and the motion trajectory continuity index of the dynamic target in the occlusion obstacle scene is calculated;

[0014] Based on the motion trajectory continuity index and behavioral activity, the behavioral purpose index of each dynamic target in the monitoring data is determined.

[0015] In a possible implementation of the present application, calculating the activity level of each dynamic target includes:

[0016] Determine the joint point coordinate data of each dynamic target;

[0017] Calculate the mean square error of the joint point coordinate data of the same dynamic target in adjacent static frames to obtain the morphological change activity of the dynamic target;

[0018] Extracting the center-of-mass coordinates of the dynamic target in each static frame, and calculating the first displacement of the center-of-mass coordinates of the same dynamic target between the current static frame and the adjacent static frame;

[0019] Based on the morphological change activity and the first displacement, the instantaneous behavioral activity of the dynamic target in each static frame is calculated;

[0020] The instantaneous behavioral activity corresponding to each static frame is integrated to obtain the behavioral activity of each dynamic target.

[0021] In a possible implementation of the present application, calculating the motion trajectory continuity index of a dynamic target in an occlusion obstacle scenario includes:

[0022] Extract the last first data frame of the dynamic target before being blocked and the first second data frame of the dynamic target after the blockage disappears;

[0023] respectively calculating a first motion direction of the dynamic target in the first data frame and a second motion direction of the dynamic target in the second data frame;

[0024] Determine a direction deviation coefficient between the first motion direction and the second motion direction, and calculate a time deviation coefficient between the first data frame and the second data frame;

[0025] Based on the direction deviation coefficient and the time deviation coefficient, the motion trajectory continuity index is calculated.

[0026] In a possible implementation of the present application, calculating a first motion direction of a dynamic target in a first data frame includes:

[0027] Extracting multiple optical flow vectors from the first data frame;

[0028] Calculate the average angle between each optical flow vector and the horizontal direction, and use the average angle as the first motion direction before occlusion.

[0029] In a possible implementation of the present application, calculating the time deviation coefficient between the first data frame and the second data frame includes:

[0030] Obtaining the instantaneous speed of the dynamic target and the area width of the occluded area in the first data frame;

[0031] Determining a first time period required for the dynamic target to pass through the obstructed area based on the area width and the instantaneous speed;

[0032] Determining an expected appearance time point of the dynamic target based on a first time point and a first time period corresponding to the first data frame;

[0033] The time difference between the expected occurrence time point and the second time point corresponding to the second data frame is calculated to obtain a time deviation coefficient.

[0034] In a possible implementation of the present application, calculating a risk mapping index of the movement behavior of a suspicious target for a risk area includes:

[0035] When a suspicious target appears within the corresponding monitoring range of the risk area, multiple real-time motion data frames of the suspicious target are obtained;

[0036] Calculating a third motion direction of the suspicious target in the real-time motion data frame and a fourth motion direction relative to the risk area;

[0037] Determining a motion direction sensitivity factor of the suspicious target based on the third motion direction and the fourth motion direction;

[0038] Obtaining the first instantaneous behavioral activity corresponding to the suspicious target in each static frame in the monitoring data, and calculating the motion activity feature based on each first instantaneous behavioral activity;

[0039] Based on the movement direction sensitivity factor and movement activity characteristics, the risk mapping index is calculated.

[0040] In a possible implementation of the present application, a risk mapping index is calculated based on a motion direction sensitivity factor and motion activity characteristics, including:

[0041] Calculating the instantaneous activity difference of the first instantaneous behavior activity in adjacent static frames;

[0042] Classify static frames whose instantaneous activity difference is less than a preset activity threshold into the same similar motion activity stage;

[0043] Based on the motion activity characteristics, the activity difference value of the suspicious target between similar motion activity stages is calculated;

[0044] Constructing a first data sequence corresponding to the activity difference value and a second data sequence corresponding to the motion direction sensitivity factor;

[0045] The Spearman correlation coefficient between the first data series and the second data series is calculated and used as a risk mapping indicator.

[0046] In a possible implementation of the present application, anti-theft monitoring of a suspicious target based on the theft risk assessment value includes:

[0047] If the theft risk assessment value is greater than the preset risk threshold, the suspicious target is determined to be at risk of theft and an early warning is issued;

[0048] Track suspicious targets based on early warning prompts and generate suspicious target behavior trajectories;

[0049] The suspicious target's behavior trajectory is sent to the relevant security personnel so that the relevant security personnel can track the suspicious target.

[0050] The present application also provides a real-time video anti-theft monitoring system based on the Internet of Things, which includes:

[0051] The acquisition module is used to obtain monitoring data within a preset historical time period and calculate the behavioral purpose index of each dynamic target in the monitoring data;

[0052] A calculation module is used to determine suspicious targets among dynamic targets based on the behavioral purpose index, and calculate the risk mapping index of the movement behavior of the suspicious targets for the risk area;

[0053] A determination module is used to determine the theft risk assessment value of a suspicious target for a risk area based on risk mapping indicators and behavior purpose indicators;

[0054] The monitoring module and the acquisition module are used to perform anti-theft monitoring on suspicious targets based on the theft risk assessment value.

[0055] The present application provides a real-time video anti-theft monitoring method and system based on the Internet of Things. Compared with the related art, in which the security risk level is dynamically evaluated by a deep learning model, which is limited by the highly nonlinear and coupled relationship between the behavioral characteristics of each target and the security risk, resulting in a high false alarm rate and missed alarm rate of the monitoring system, in the present application, by obtaining monitoring data within a historical preset time period and calculating the behavioral purpose index of each dynamic target in the monitoring data, suspicious targets with suspicious behavior are determined based on the behavioral purpose index, and then the risk mapping index of the movement behavior of the suspicious target and the risk area is calculated, thereby establishing a linear correlation between the movement behavior of the suspicious target and the risk area, and then determining the theft risk assessment value of the suspicious target for the risk area based on the risk mapping index and the behavioral purpose index. Based on the theft risk assessment value, the corresponding suspicious target is selected for anti-theft monitoring, thereby improving the alarm accuracy of the monitoring system and reducing the false alarm rate and missed alarm rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flowchart of the first embodiment of the real-time video anti-theft monitoring method based on the Internet of Things of this application;

[0057] Figure 2 This is a schematic diagram of dynamic target marking involved in the real-time video anti-theft monitoring method based on the Internet of Things in this application;

[0058] Figure 3 This is a flow chart of a second embodiment of the real-time video anti-theft monitoring method based on the Internet of Things of this application;

[0059] Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application. DETAILED DESCRIPTION

[0060] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0061] The embodiment of the present application provides a real-time video anti-theft monitoring method based on the Internet of Things. In the first embodiment of the real-time video anti-theft monitoring method based on the Internet of Things, referring to Figure 1 , methods include:

[0062] Step S10: acquiring monitoring data within a preset historical time period and calculating the behavioral purposefulness index of each dynamic target in the monitoring data;

[0063] Step S20, based on the behavior purpose index, determining suspicious targets among the dynamic targets, and calculating the risk mapping index of the movement behavior of the suspicious targets to the risk area;

[0064] Step S30, determining the theft risk assessment value of the suspicious target for the risk area based on the risk mapping index and the behavior purpose index;

[0065] Step S40: performing anti-theft monitoring on the suspicious target based on the theft risk assessment value.

[0066] This embodiment aims to improve the alarm accuracy of the monitoring system and reduce the false alarm rate and missed alarm rate.

[0067] The specific steps are as follows:

[0068] Step S10: Acquire monitoring data within a preset historical time period, and calculate the behavioral purpose index of each dynamic target in the monitoring data.

[0069] As an example, the real-time video anti-theft monitoring method based on the Internet of Things can be applied to a real-time video anti-theft monitoring device based on the Internet of Things. The real-time video anti-theft monitoring device based on the Internet of Things belongs to a real-time video anti-theft monitoring system based on the Internet of Things, and the real-time video anti-theft monitoring system based on the Internet of Things belongs to a real-time video anti-theft monitoring device based on the Internet of Things.

[0070] As an example, the historical preset time period can be a collection of real-time images within the previous 1 minute, 5 minutes, or 10 minutes. This application is mainly used in a real-time video anti-theft monitoring system based on the Internet of Things. The system adopts a three-level deployment architecture of "stereo coverage + dynamic focus + multi-camera collaboration" and combines the Internet of Things edge computing nodes to build an intelligent data collection network. The specific deployment strategy is as follows:

[0071] First, 1080P high-definition cameras are deployed at all entrances and exits of open spaces and at channel hubs. These cameras cover the front view (120° ultra-wide angle), side view (90° rotatable pan / tilt, ±30° vertical adjustment range), and rear view (80° fixed angle), providing 360° surround surveillance.

[0072] Dynamically adjust camera deployment density based on the risk level of the monitored area. Deployment spacing in core areas or risky areas (such as vaults and data centers) should be ≤5 meters, and in common areas (such as corridors and perimeters) ≤15 meters.

[0073] Each HD camera analyzes the front-facing main camera data at a default 15fps full frame rate. When a suspicious moving target is detected, 50% of the computing power is dynamically allocated to the associated side / rear-view cameras to improve the analysis accuracy of the local area.

[0074] Based on the motion vector and ROI detection algorithm, image frames containing dynamic targets are extracted (at an interval of 0.5s±0.1s), with the resolution maintained at 1080P;

[0075] Upload key image frames (0.5-second intervals) to the cloud in real time via the 4G / 5G module, and store all videos by event level (normal events are retained for 7 days, serious events are retained for 90 days);

[0076] Finally, the perception layer achieves millisecond-level response between devices based on the IoT protocol (MQTT+HTTP / 2), ensuring data consistency among cameras.

[0077] As an example, there may be multiple dynamic targets in the monitoring data. These dynamic targets can be movable targets such as people and animals. These dynamic targets will have certain behavioral characteristics. Taking people as an example, the person in the monitoring data may be going to buy clothes, so his behavioral characteristic is going to a clothing store, and his movement trajectory is also around the store selling clothes.

[0078] As an example, the behavioral purpose index represents an indicator related to the movement purpose of a dynamic target. When a person has abnormal behavior (such as theft), the criminal will evade monitoring or quickly approach the target. If the trajectory continuity can still be maintained in an occluded scenario, it means that the movement behavior of the dynamic target is highly purposeful, rather than randomly wandering in the monitoring area.

[0079] The step S10 of the real-time video anti-theft monitoring based on the Internet of Things further includes steps S11 to S14, including:

[0080] In step S11 , based on a preset target detection algorithm, each static frame in the monitoring data is marked and selected to obtain multiple dynamic targets.

[0081] As an example, the preset target detection algorithm can be the YOLO target detection algorithm. The YOLO target detection algorithm is a real-time target detection and tracking method based on deep learning. According to this algorithm, the targets in each static frame corresponding to the monitoring data can be identified and labeled. Specifically, multiple target frames are obtained, and each target frame corresponds to a dynamic target. At the same time, the target can be tracked and the target frames corresponding to the same dynamic target in different static frames can be determined.

[0082] As an example, the diagram after marking and selecting each dynamic target is as follows: Figure 2 As shown by Figure 2 It can be seen that in a static frame in the monitoring data, each person has a label box.

[0083] Step S12: Calculate the activity level of each dynamic target and filter the scene data frames when the dynamic target is blocked in the monitoring data.

[0084] As an example, behavioral activity indicates the activity level of each dynamic target. For example, when a person moves a long distance within a period of time or the amplitude of the movement is too large, it can be determined that the behavioral activity of the dynamic target is high. Behavioral activity includes the activity of limb changes and the activity of displacement changes. By considering these two aspects, the behavioral activity of the dynamic target can be calculated.

[0085] As an example, due to site limitations, it is inevitable that cameras will still have blind spots after deployment. At this time, data frames in which the target is obscured will appear in the monitoring data, which are also scene data frames. The obstruction can be half of the target area or all of it, without specific limitation.

[0086] The step S12 of calculating the activity level of each dynamic target includes:

[0087] Determine the joint point coordinate data of each dynamic target.

[0088] As an example, the joint point coordinate data can be the coordinate points of each joint point of the dynamic target within the selected area. The joint point can be a wrist joint, knee joint, etc. The coordinate data set of each joint point is output through the HRNet model (a deep learning model for computer vision tasks).

[0089] The mean square error of the joint point coordinate data of the same dynamic target in adjacent static frames is calculated to obtain the morphological change activity of the dynamic target.

[0090] As an example, the purpose of calculating the mean square error is to calculate the degree of morphological change of the dynamic target between the current static frame and the adjacent static frame (such as the previous static frame), and to reflect the change of the joint point coordinate data through the mean square error. a ,D b ) represents the mean square error, where a and b represent the adjacent static frame (frame a) and the current static frame (frame b) in the joint point coordinate data, respectively, and D a Represents the coordinate position set of each joint point coordinate data in frame a, D b Represents the coordinate position set of each joint point coordinate data in the b-frame. This value is used to preliminarily judge the activeness of the dynamic target's action form changes in the current static frame (b-frame), that is, the form change activity.

[0091] The center-of-mass coordinates of the dynamic target in each static frame are extracted, and the first displacement of the center-of-mass coordinates of the same dynamic target between the current static frame and the adjacent static frame is calculated.

[0092] As an example, the center of mass coordinates mainly quantify each dynamic target into a coordinate point. The purpose of extracting the center of mass coordinates is to calculate the displacement distance between two static frames. The first displacement is the displacement distance moved by the dynamic target between two different static frames.

[0093] Specifically, the centroid coordinates of the selected area corresponding to the dynamic target in each static frame are obtained based on the spatial coordinate system, and the displacement d of the centroid coordinates of the same dynamic target in the current static frame and the adjacent static frame (such as the previous static frame) is calculated.

[0094] Based on the morphological change activity and the first displacement, the instantaneous behavioral activity of the dynamic target in each static frame is calculated.

[0095] As an example, instantaneous behavioral activity is used to represent the behavioral activity of a dynamic target in a short period of time (for example, between two static frames). Before calculating the instantaneous behavioral activity, it is desired to obtain the instantaneous displacement change of the dynamic target based on the first displacement. The instantaneous displacement change can be calculated as follows:

[0096]

[0097] Among them, d t represents the time difference between adjacent static frames, and v represents the instantaneous displacement change of the region. The larger the value, the greater the instantaneous displacement of the target.

[0098] Furthermore, the MSE (D a ,D b ) and the product of the regional instantaneous displacement change v: E = MSE (D a ,D b )×v, and obtain the instantaneous behavioral activity E of the dynamic target.

[0099] The instantaneous behavioral activity corresponding to each static frame is integrated to obtain the behavioral activity of each dynamic target.

[0100] As an example, the way to integrate each instantaneous behavioral activity can be to calculate the instantaneous behavioral activity E of the dynamic target corresponding to all static frame images, and calculate its mean μ(E), which is regarded as the behavioral activity of the dynamic target in the current stage. The larger the value, the more active the dynamic target is in the current stage, such as fast running, strenuous exercise, etc.

[0101] Step S13: When the area of ​​the dynamic target blocked in the scene data frame is greater than a preset area threshold, it is determined that the dynamic target is in an occlusion obstacle scene, and a motion trajectory continuity index of the dynamic target in the occlusion obstacle scene is calculated.

[0102] As an example, in the actual monitoring process, dynamic targets are often partially or completely blocked by other objects (such as pedestrians, decorations, etc.), and fixed cameras cannot penetrate the obstructions, resulting in fragmentation of target behavior data and affecting the integrity of subsequent analysis. Therefore, it is necessary to analyze the scene data frames in the obstruction scene and track the bounding box of the dynamic target through the YOLO target detection algorithm. When the area of ​​the target bounding box decreases (such as by 50%, where 50% can be a preset area threshold) or the bounding box disappears (the target is lost in the continuous image frames), it is determined that the dynamic target is in an occlusion obstacle scene.

[0103] As an example, the motion trajectory continuity index is used to represent the motion continuity of a dynamic target, and is used to analyze whether it deliberately maintains its motion trajectory in an occlusion obstacle scenario, thereby determining whether its behavior has the intention of theft.

[0104] The step S13 of calculating the motion trajectory continuity index of the dynamic target in the occlusion obstacle scene includes:

[0105] The last first data frame of the dynamic target before being blocked and the first second data frame of the dynamic target after the blockage disappears are extracted.

[0106] As an example, the first data frame is the last data frame before the dynamic target is blocked, and the second data frame is the first data frame in which the dynamic target appears after the blockage disappears.

[0107] A first motion direction of the dynamic target in the first data frame and a second motion direction of the dynamic target in the second data frame are calculated respectively.

[0108] As an example, the first movement direction is the movement direction of the dynamic target before being blocked, and the second movement direction is the movement direction of the dynamic target after being blocked. By comparing the changes in the movement direction of the dynamic target before and after being blocked, the continuity of the target movement is determined.

[0109] The step of calculating the first motion direction of the dynamic target in the first data frame includes:

[0110] A plurality of optical flow vectors in the first data frame is extracted.

[0111] As an example, the optical flow method is used to extract the optical flow direction set of the area corresponding to the dynamic target in the first data frame. The optical flow vector set is represented as: {o1, o2, ..., o n}, where o n The optical flow method is well known to those skilled in the art and will not be described in detail here.

[0112] Calculate the average angle between each optical flow vector and the horizontal direction, and use the average angle as the first motion direction before occlusion.

[0113] As an example, the average of the included angles of all the optical flow vectors with the horizontal direction is calculated: θ, as the first motion direction of the dynamic target before the occlusion, and the second motion direction θ' can be calculated in the same way.

[0114] A direction deviation coefficient between the first motion direction and the second motion direction is determined, and a time deviation coefficient between the first data frame and the second data frame is calculated.

[0115] As an example, the direction deviation coefficient between the first motion direction and the second motion direction is represented as: the direction deviation before and after the occlusion of the dynamic target: |θ-θ'|, taking the value of Δθ (normalized by the norm function), defined as the direction deviation coefficient before and after the occlusion of the dynamic target, quantifying the stability of the motion direction of the target before and after the occlusion, the smaller the value, the higher the continuity of the motion intention of the target when encountering an occlusion obstacle.

[0116] As an example, the time deviation coefficient represents the time deviation of the dynamic target before and after being occluded, quantifying the motion stability of crossing the occlusion area, the smaller the value, the better the continuity of the motion trajectory of the dynamic target.

[0117] The step of calculating the time deviation coefficient between the first data frame and the second data frame includes:

[0118] The instantaneous speed of the dynamic target in the first data frame and the area width of the occlusion area are obtained;

[0119] As an example, in the process of extracting the first data frame, the instantaneous speed v of the dynamic target in the image data before the occlusion can be obtained t , and the area width w of the occlusion area (converted to the actual distance by camera calibration).

[0120] Based on the area width and the instantaneous speed, the first time period required for the dynamic target to cross the occlusion area is determined;

[0121] As an example, according to the area width and the instantaneous speed, the time required for the dynamic target to cross the occlusion area, i.e., the first time period, is calculated, and the calculation method of the first time period is:

[0122]

[0123] where t pred represents the first time period, w is the area width, and v t is the instantaneous speed.

[0124] Based on the first time point corresponding to the first data frame and the first time period, the expected appearance time point of the dynamic target is determined;

[0125] As an example, the first time point is added to the first time period to obtain the expected appearance time point H of the dynamic target. pred , H pred =t+t pred , where t represents the first time point.

[0126] The time difference between the expected occurrence time point and the second time point corresponding to the second data frame is calculated to obtain a time deviation coefficient.

[0127] As an example, calculate the time difference between the estimated appearance time of a dynamic target and the actual appearance time: |H pred -l|, where l represents the time point when the dynamic target appears after the occlusion disappears. The value ΔT is defined as the time deviation coefficient before and after the dynamic target is occluded, which quantifies the motion stability of the target crossing the occluded area. The smaller the value, the more likely it is that the target strictly follows the uniform linear motion model when crossing the occluded area. Its motion trajectory is highly predictable, which indirectly reflects that the target's motion trajectory has good continuity.

[0128] Based on the direction deviation coefficient and the time deviation coefficient, the motion trajectory continuity index is calculated.

[0129] As an example, the motion trajectory continuity index represents the motion continuity over a period of time. The continuity index between two data frames can be calculated first. The calculation method of the continuity index can be:

[0130]

[0131] Where R represents the continuity index, Δθ is the direction deviation coefficient, and ΔT is the time deviation coefficient; ∈ represents an infinite decimal greater than 0, which is used to prevent the denominator from being 0.

[0132] As an example, the R value is a comprehensive quantitative indicator of the continuity of the target motion trajectory, which directly reflects the ability of the dynamic target to maintain the original motion intention in an occlusion scenario.

[0133] As an example, the mean value μ(R) of the continuity index of the motion trajectory corresponding to each occlusion before and after each occlusion in the real-time image dataset of the dynamic target with a history of ten minutes (including at least one occlusion phenomenon, otherwise the historical dataset is extended until it includes one occlusion phenomenon) is extracted and regarded as the motion trajectory continuity index of the target at the current stage.

[0134] Step S14: determining the behavioral purposefulness index of each dynamic target in the monitoring data based on the motion trajectory continuity index and the behavioral activity.

[0135] As an example, the behavioral purposefulness metric could be calculated as:

[0136] M = norm(μ(R) + μ(E)), where M is defined as the behavioral purposefulness index of the dynamic goal;

[0137] Among them, μ(R) represents the continuity index of the motion trajectory, μ(E) represents the activity of the behavior, and norm() represents the normalization calculation.

[0138] Step S20 : ​​determining suspicious targets among the dynamic targets based on the behavior purpose index, and calculating the risk mapping index of the movement behavior of the suspicious targets to the risk area.

[0139] As an example, when a dynamic target is determined to have strong purpose, the suspicious targets among the dynamic targets are determined based on the behavioral purpose index corresponding to the dynamic target. For example, when the behavioral purpose index of dynamic target A is greater than 0.7, it is marked as a suspicious target.

[0140] As an example, after identifying a suspicious target, computing power is dynamically allocated to key node cameras (such as vaults, data centers, etc.) based on the global perception layer to enhance further decoupling analysis of the behavioral characteristics of the suspicious target, and then determine the relevance of the suspicious target's movement behavior to the risk area, that is, to determine whether the suspicious target has a tendency to move towards the risk area.

[0141] As an example, the risk mapping index is determined by two parts: movement direction and movement activity. Movement direction is mainly used to determine whether the suspicious target has a tendency to move towards the risk area, and movement activity is used to determine the abnormal behavior of the suspicious target (such as fast movement, short stay). Based on these two indicators, the theft risk of the suspicious target is determined, thereby obtaining the risk mapping index.

[0142] Step S30: determining the theft risk assessment value of the suspicious target for the risk area based on the risk mapping index and the behavior purpose index.

[0143] As an example, the theft risk assessment value may be calculated as follows:

[0144]

[0145] Where norm is the normalization function, ρ is the risk mapping indicator, KPI represents the theft risk assessment value of a specific dynamic target, and M is the behavior purpose indicator, which comprehensively quantifies the abnormality of the current dynamic target's motion trajectory in an open scene. The larger the value, the greater the suspicion that the corresponding dynamic target is involved in theft.

[0146] Step S40: performing anti-theft monitoring on the suspicious target based on the theft risk assessment value.

[0147] As an example, after determining the theft risk assessment value of a suspicious target, it is determined whether to continuously track and monitor the suspicious target or issue an early warning based on the size of the theft risk assessment value, and then proceed to the next step of anti-theft processing.

[0148] The step S40 of performing anti-theft monitoring on a suspicious target based on the theft risk assessment value includes:

[0149] Step S41: If the theft risk assessment value is greater than a preset risk threshold, it is determined that the suspicious target has a theft risk and an early warning prompt is issued.

[0150] As an example, the preset risk threshold may be 0.6, 0.7, etc., without specific limitation.

[0151] As an example, taking the preset risk threshold of 0.7 as an example, when the theft risk assessment value is greater than 0.7, it is determined that the suspicious target is at risk of theft. The system immediately activates the third-level emergency response mechanism and issues an early warning.

[0152] Step S42: Track the suspicious target based on the early warning prompt and generate a suspicious target behavior trajectory.

[0153] Specifically, the system uses the alarm mechanism to issue early warnings to the security system in real time, and arranges security personnel to check for information on stolen items; simultaneously, the system pushes target feature vectors (including HSV color histograms, motion acceleration vectors, and skeletal joint topology data) to cameras within a radius of 200 meters through the LoRaWAN broadcast protocol, and starts the cluster collaborative tracking mode. Then, the computing power allocation strategy is dynamically adjusted according to the network topology structure, and the frame rate of the associated cameras is increased from 15fps to 30fps (using inter-frame interpolation technology to compensate for frame loss), and the bit rate is increased from 4Mbps to 12Mbps (enabling the CU extended partitioning mode of H.266VVC encoding); in addition, to ensure accurate target locking, the system allocates an additional 23% of GPU computing power resources to the cameras in the target area, and uses containerization technology to achieve millisecond-level resource preemption, ensuring that at least three cameras continuously track the target contour with an accuracy of 0.5°, accurately obtain the suspicious person's behavior trajectory, and then generate the suspicious target behavior trajectory.

[0154] Step S43: Send the suspicious target's behavior trajectory to relevant security personnel so that the relevant security personnel can track the suspicious target.

[0155] As an example, after the relevant security personnel determine that the item is stolen, the suspicious target's behavior trajectory is sent to the relevant security personnel so that the relevant security personnel can track the suspicious target.

[0156] The present application provides a real-time video anti-theft monitoring method and system based on the Internet of Things. Compared with the related art, in which the security risk level is dynamically evaluated by a deep learning model, which is limited by the highly nonlinear and coupled relationship between the behavioral characteristics of each target and the security risk, resulting in a high false alarm rate and missed alarm rate of the monitoring system, in the present application, by obtaining monitoring data within a historical preset time period and calculating the behavioral purpose index of each dynamic target in the monitoring data, suspicious targets with suspicious behavior are determined based on the behavioral purpose index, and then the risk mapping index of the movement behavior of the suspicious target and the risk area is calculated, thereby establishing a linear correlation between the movement behavior of the suspicious target and the risk area, and then determining the theft risk assessment value of the suspicious target for the risk area based on the risk mapping index and the behavioral purpose index. Based on the theft risk assessment value, the corresponding suspicious target is selected for anti-theft monitoring, thereby improving the alarm accuracy of the monitoring system and reducing the false alarm rate and missed alarm rate.

[0157] Further, refer to Figure 3 Based on the first embodiment of the present application, another embodiment of the present application is provided. In this embodiment, step S20 of calculating the risk mapping index of the movement behavior of the suspicious target for the risk area includes:

[0158] Step S21 : when a suspicious target appears within a monitoring range corresponding to a risk area, a plurality of real-time motion data frames of the suspicious target are acquired.

[0159] As an example, when a suspicious target appears within the corresponding monitoring range of the risk area (identified by facial or behavioral features M), the module further predicts its direction of movement, determines whether it moves towards sensitive areas (such as vaults, data centers, etc.), and obtains multiple real-time motion data frames of the suspicious target.

[0160] Step S22 , calculating a third motion direction of the suspicious target in the real-time motion data frame and a fourth motion direction relative to the risk area.

[0161] As an example, the optical flow method is used to extract the set of optical flow vectors of adjacent static frames during the movement of the suspicious target, and the average angle between all optical flow vectors and the horizontal direction is calculated (note that the angle range is: 0°<θ1<90°): θ1, which is regarded as the third movement direction of the suspicious target.

[0162] As an example, connect the centroid of the selected area corresponding to the suspicious target and the centroid of the risk area, and obtain the angle between the line connecting the two points and the horizontal direction: θ2, (0°<θ2<90°)) and use θ2 as the fourth movement direction of the suspicious target relative to the risk area.

[0163] Step S23: determining a motion direction sensitivity factor of the suspicious target based on the third motion direction and the fourth motion direction.

[0164] As an example, the interaction direction of the suspicious target is quantified: Q = 1 - cos | θ1- θ2|, where Q defines the motion direction sensitive factor of the suspicious target at the current moment, and quantifies the interaction between the motion behavior of the suspicious target at the current stage and the risk region. When Q is 0, the included angle corresponding to θ1 and θ2 is the same, which determines that the suspicious target is moving in the direction of the risk region. When the suspicious target leaves the risk region after the theft behavior is completed, the included angle corresponding to θ1 and θ2 is the same, and the value of Q is also 0. That is, when the suspicious target has a theft behavior, the motion direction sensitive factors corresponding to the approaching behavior and the moving away behavior are both small.

[0165] When other targets accidentally approach or deviate from the risk region, the motion behavior of the target is uncertain. At this time, the greater the value of Q, the greater the included angle between θ1 and θ2, which reflects that the motion behavior of the suspicious target at the current stage is moving away from the risk region. The smaller the value of Q, the closer the suspicious target is to the risk region.

[0166] Step S24, obtaining the first instantaneous behavior activity of the suspicious target corresponding to each static frame in the monitoring data, and calculating the motion activity feature based on the first instantaneous behavior activity.

[0167] As an example, generally, when analyzing the motion trajectory of a criminal suspect, if the target and the sensitive region do not show significant approaching or moving away behavior, it can be inferred that the suspect is performing “scouting” or observing the withdrawal behavior.

[0168] Therefore, further analysis is performed on the combination of “short stay” and “fast movement” behaviors in the complete motion trajectory of the target to identify potential criminal signs. This behavior pattern can correspond to the “scouting” preparation activities before theft or the withdrawal action after theft of the suspect.

[0169] As an example, the first instantaneous behavior activity E of the suspicious target corresponding to each static frame in the monitoring data is obtained, and the instantaneous activity difference ΔE of the first instantaneous behavior activity of adjacent static frames is calculated. Each static frame with an instantaneous activity difference less than a preset threshold is divided into the same similar motion activity stage, and the mean value μ(E) of the instantaneous behavior activity of all static frames in all similar motion activity stages is calculated as the motion activity feature of the suspicious target in the current similar motion activity stage.

[0170] Step S25, calculating the risk mapping index based on the motion direction sensitive factor and the motion activity feature.

[0171] In step S25 of calculating the risk mapping index based on the motion direction sensitive factor and the motion activity feature, the following steps are included:

[0172] Calculate the instantaneous activity difference of the first instantaneous behavior activity in adjacent static frames.

[0173] The static frames whose instantaneous activity difference is less than a preset activity threshold are divided into the same similar motion activity stage.

[0174] As an example, the preset activity threshold may be 0.5, or other values, which are not specifically limited.

[0175] As an example, static frames whose instantaneous activity difference ΔE is less than a preset activity threshold are divided into the same similar motion activity stage. That is, when a static frame whose ΔE is greater than or equal to the preset activity threshold is traversed, the division is terminated and the stage division is restarted from the current static frame until the complete data set of monitoring data is traversed, and multiple similar motion activity stages are obtained.

[0176] Based on the motion activity characteristics, the activity difference value between the corresponding static frames of the suspicious target at similar motion activity stages is calculated;

[0177] As an example, the instantaneous behavioral activity mean μ(E) of all static frames in all similar motion activity stages is calculated respectively and regarded as the motion activity feature of that stage. The activity mean values ​​corresponding to different similar motion activity stages are also different.

[0178] As an example, based on the activity differences between adjacent similar motion activity stages, the target's "short stay" and "fast movement" behavior combinations are identified. Before the target prepares to perform a theft operation, when approaching or leaving the risk area, there will be a "short stay" and "fast movement" behavior combination. The activity difference between adjacent similar motion activity stages is recorded as the activity difference value. The activity difference value can be calculated as follows:

[0179]

[0180] Where i represents the i-th similar motion activity stage in the monitoring data set collected by the system, i+1 and i-1 represent the similar motion activity stages on the left and right sides of stage i, respectively, and μ(E) i-1 ,μ(E) i ,μ(E) i+1 Represents the mean activity of stages i-1, i, and i+1, ε=1e-5, W i Represents the activity difference value of the i-th similar sports activity stage; the formula value W iThe larger the value is, the similar motion activity exists in the target area of ​​stages i+1 and i-1 in the monitoring data set, while there is a significant difference in motion activity between stages i+1 and i, and the motion activity of stage i+1 is significantly greater than that of stage i, which corresponds to the combination of "short stay (stage i)" and "fast movement (stage i-1, stage i+1)" behaviors in the motion trajectory of suspicious targets.

[0181] Constructing a first data sequence corresponding to the activity difference value and a second data sequence corresponding to the motion direction sensitivity factor;

[0182] As an example, the statistical system collects the motion direction sensitivity factor Q corresponding to each static frame in the image data set, and the activity difference value W corresponding to the similar motion activity stage of the static frame. i , respectively construct data sequences A(Q) (second data sequence), B(W i )(first data sequence).

[0183] The Spearman correlation coefficient between the first data series and the second data series is calculated and used as a risk mapping indicator.

[0184] As an example, calculate the data sequence A(Q), B(W i ) between them: ρ, which is regarded as the risk mapping indicator of the target. The value range is -1 to +1. The closer the value is to -1, the better the data sets A(Q), B(W i ) is stronger, corresponding to when the suspicious target has a combination of "short stay" and "fast movement" behaviors, that is, when it approaches or moves away from the risk area, W i When the Q value increases and decreases, it means that the motion trajectory of the dynamic target is deviating from the trajectory close to or away from the risk area, indicating that the current dynamic target has a high theft risk behavior.

[0185] In this embodiment, the risk mapping index is calculated by combining the motion direction sensitivity factor and activity difference value of the suspicious target, thereby determining whether the dynamic target has theft risk.

[0186] In an embodiment of the present application, a real-time video anti-theft monitoring system based on the Internet of Things is further provided. The real-time video anti-theft monitoring system based on the Internet of Things includes:

[0187] The acquisition module is used to obtain monitoring data within a preset historical time period and calculate the behavioral purpose index of each dynamic target in the monitoring data;

[0188] A calculation module is used to determine suspicious targets among dynamic targets based on the behavioral purpose index, and calculate the risk mapping index of the movement behavior of the suspicious targets for the risk area;

[0189] The determining module is configured to determine a theft risk assessment value of the suspicious target for the risk area based on the risk mapping index and the behavior purpose index.

[0190] The monitoring module is configured to perform anti-theft monitoring on the suspicious target based on the theft risk assessment value.

[0191] Referring to Figure 4 , Figure 4 is a device structure schematic diagram of a hardware running environment involved in the embodiment of the present application.

[0192] As Figure 4 shown, the real-time video anti-theft monitoring device based on the Internet of Things can include a processor 1001, a memory 1005, and a communication bus 1002. The communication bus 1002 is configured to realize the connection communication between the processor 1001 and the memory 1005.

[0193] Optionally, the real-time video anti-theft monitoring device based on the Internet of Things can further include a user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, a WiFi module, and the like. The user interface can include a display screen (Display), an input sub-module such as a keyboard (Keyboard), and the optional user interface can further include a standard wired interface, a wireless interface. The network interface can include a standard wired interface, a wireless interface (such as a WI-FI interface).

[0194] Those skilled in the art can understand that Figure 4 the real-time video anti-theft monitoring device structure based on the Internet of Things shown in the figure does not constitute a limitation on the real-time video anti-theft monitoring device based on the Internet of Things, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0195] As Figure 4 shown, the memory 1005 as a storage medium can include an operating system, a network communication module, and a real-time video anti-theft monitoring program based on the Internet of Things. The operating system is a program that manages and controls the hardware and software resources of the real-time video anti-theft monitoring device based on the Internet of Things, supports the running of the real-time video anti-theft monitoring program based on the Internet of Things and other software and / or programs. The network communication module is configured to realize the communication between the components in the memory 1005, and the communication between other hardware and software in the real-time video anti-theft monitoring system based on the Internet of Things.

[0196] In Figure 4In the IoT-based real-time video anti-theft monitoring device shown, the processor 1001 is used to execute the IoT-based real-time video anti-theft monitoring program stored in the memory 1005 to implement any of the steps of the IoT-based real-time video anti-theft monitoring method described above.

[0197] The specific implementation of the real-time video anti-theft monitoring device based on the Internet of Things in this application is basically the same as the above-mentioned embodiments of the real-time video anti-theft monitoring method based on the Internet of Things, and will not be repeated here.

[0198] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0199] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0200] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as mentioned above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of each embodiment of the present application.

[0201] The above are only preferred embodiments of the present application and do not limit the scope of application of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application description and drawings, or directly or indirectly applied in other related technical fields, are also included in the scope of protection of the present application.

[0202] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0203] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A real-time video anti-theft monitoring method based on the Internet of Things, characterized in that: The method comprises: Obtain monitoring data within a preset historical time period and calculate the behavioral purposefulness index of each dynamic target in the monitoring data; Based on the behavioral purpose index, determining suspicious targets among the dynamic targets, and calculating a risk mapping index of the movement behavior of the suspicious targets for the risk area; Determining a theft risk assessment value of the suspicious target for the risk area based on the risk mapping indicator and the behavior purpose indicator; Based on the theft risk assessment value, anti-theft monitoring is performed on the suspicious target.

2. The real-time video anti-theft monitoring method based on the Internet of Things according to claim 1, characterized in that: The calculating of the behavioral purpose index of each dynamic target in the monitoring data includes: Based on a preset target detection algorithm, each static frame in the monitoring data is marked and framed to obtain multiple dynamic targets; Calculating the behavioral activity of each of the dynamic targets, and screening the scene data frames when the dynamic targets are blocked in the monitoring data; When the area blocked by the dynamic target in the scene data frame is greater than a preset area threshold, determining that the dynamic target is in an occlusion obstacle scene, and calculating a motion trajectory continuity index of the dynamic target in the occlusion obstacle scene; Based on the motion trajectory continuity index and the behavioral activity, a behavioral purpose index of each dynamic target in the monitoring data is determined.

3. The real-time video anti-theft monitoring method based on the Internet of Things according to claim 2, characterized in that: The calculating of the behavioral activity of each dynamic target includes: Determining the joint point coordinate data of each of the dynamic targets; Calculating the mean square error of the joint point coordinate data of the same dynamic object in adjacent static frames to obtain the morphological change activity of the dynamic object; Extracting the center-of-mass coordinates of the dynamic target in each of the static frames, and calculating a first displacement of the center-of-mass coordinates of the same dynamic target between the current static frame and the adjacent static frame; Calculating the instantaneous behavior activity of the dynamic target in each static frame based on the morphological change activity and the first displacement; The instantaneous behavioral activity corresponding to each of the static frames is integrated to obtain the behavioral activity of each of the dynamic targets.

4. The real-time video anti-theft monitoring method based on the Internet of Things according to claim 2, characterized in that: The calculating of the motion trajectory continuity index of the dynamic target in the occlusion obstacle scene includes: Extracting the last first data frame of the dynamic target before being blocked and the first second data frame of the dynamic target after the blockage disappears; respectively calculating a first motion direction of the dynamic target in the first data frame and a second motion direction of the dynamic target in the second data frame; determining a direction deviation coefficient between the first motion direction and the second motion direction, and calculating a time deviation coefficient between the first data frame and the second data frame; Based on the direction deviation coefficient and the time deviation coefficient, a motion trajectory continuity index is calculated.

5. The real-time video anti-theft monitoring method based on the Internet of Things according to claim 4, characterized in that: The calculating the first motion direction of the dynamic target in the first data frame includes: Extracting a plurality of optical flow vectors from the first data frame; The average of the angles between each of the optical flow vectors and the horizontal direction is calculated, and the average of the angles is used as the first movement direction before occlusion.

6. The real-time video anti-theft monitoring method based on the Internet of Things according to claim 4, characterized in that: The calculating a time deviation coefficient between the first data frame and the second data frame includes: Obtaining the instantaneous speed of the dynamic target and the width of the occluded area in the first data frame; Determining a first time period required for the dynamic target to pass through the obstruction area based on the area width and the instantaneous speed; Determining an expected appearance time point of a dynamic target based on a first time point corresponding to the first data frame and the first time period; A time difference between the expected occurrence time point and a second time point corresponding to the second data frame is calculated to obtain a time deviation coefficient.

7. The real-time video anti-theft monitoring method based on the Internet of Things according to claim 1, characterized in that: The calculating of the risk mapping index of the movement behavior of the suspicious target for the risk area includes: When the suspicious target appears within the monitoring range corresponding to the risk area, multiple real-time motion data frames of the suspicious target are acquired; Calculating a third motion direction of the suspicious target in the real-time motion data frame and a fourth motion direction relative to the risk area; Determining a motion direction sensitivity factor of the suspicious target based on the third motion direction and the fourth motion direction; Obtaining a first instantaneous behavioral activity corresponding to a suspicious target in each static frame in the monitoring data, and calculating a motion activity feature based on each of the first instantaneous behavioral activities; A risk mapping index is calculated based on the movement direction sensitivity factor and the movement activity characteristic.

8. The real-time video anti-theft monitoring method based on the Internet of Things according to claim 7, characterized in that: The calculating of the risk mapping index based on the motion direction sensitivity factor and the motion activity characteristic includes: Calculating instantaneous activity differences of the first instantaneous activity in adjacent static frames; Classifying the static frames whose instantaneous activity difference is less than a preset activity threshold into the same similar motion activity stage; Calculating, based on the motion activity characteristics, activity difference values ​​of the suspicious target between the similar motion activity stages; Constructing a first data sequence corresponding to the activity difference value and a second data sequence corresponding to the motion direction sensitivity factor; Calculate the Spearman correlation coefficient between the first data sequence and the second data sequence, and use the Spearman correlation coefficient as a risk mapping indicator.

9. The real-time video anti-theft monitoring method based on the Internet of Things according to claim 1, characterized in that: The anti-theft monitoring of the suspicious target based on the theft risk assessment value includes: If the theft risk assessment value is greater than a preset risk threshold, the suspicious target is determined to be at risk of theft and an early warning is issued; Tracking the suspicious target based on the early warning prompt to generate a suspicious target behavior trajectory; The behavior trajectory of the suspicious target is sent to relevant security personnel so that the relevant security personnel can track the suspicious target.

10. A real-time video anti-theft monitoring system based on the Internet of Things, characterized in that: The real-time video anti-theft monitoring system based on the Internet of Things includes: An acquisition module, configured to acquire monitoring data within a preset historical time period and calculate a behavioral purpose index for each dynamic target in the monitoring data; a calculation module, the calculation module being configured to determine a suspicious target among the dynamic targets based on the behavior purpose index, and calculate a risk mapping index of the movement behavior of the suspicious target with respect to a risk area; a determination module configured to determine a theft risk assessment value of the suspicious target for the risk area based on the risk mapping indicator and the behavior purpose indicator; The monitoring module is used to perform anti-theft monitoring on the suspicious target based on the theft risk assessment value.

Images