Real-time video theftproof monitoring method and system based on internet of things

Abstract

The application discloses a real-time video theftproof monitoring method and system based on Internet of Things, and relates to the technical field of data processing. The method comprises the following steps: acquiring monitoring data in a historical preset time period, and calculating a behavior purpose index of each dynamic target in the monitoring data; determining suspicious targets in the dynamic targets based on the behavior purpose index, and calculating a risk mapping index of a motion behavior of the suspicious targets to a risk area; determining a theft risk evaluation value of the suspicious targets to the risk area based on the risk mapping index and the behavior purpose index; and performing theftproof monitoring on the suspicious targets based on the theft risk evaluation value. The application reduces the false positive rate and the false negative rate of the monitoring system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a real-time video anti-theft monitoring method and system based on Internet of Things. BACKGROUND

[0002] Nowadays, intelligent devices and sensors combined with Internet of Things technology can collect video and environmental data in real time, timely alarm and handle potential security threats, thereby improving monitoring efficiency and security, and are widely used to solve public security problems.

[0003] At present, the video anti-theft monitoring system based on Internet of Things in the market mainly fuses multi-source data, uses a deep learning model (such as target detection and behavior recognition) to dynamically evaluate the security risk level, so as to realize accurate alarm. However, the actual security risk evaluation is limited by the high nonlinearity and coupling between the behavior characteristics (walking, staying and carrying objects, etc.) of each target and the security risk, which will lead to high false alarm rate and missed alarm rate of the monitoring system. SUMMARY

[0004] The main purpose of the present application is to provide a real-time video anti-theft monitoring method and system based on Internet of Things, which aims to solve the technical problem in the related art that the false alarm rate and missed alarm rate of the monitoring system are high due to the high nonlinearity and coupling between the behavior characteristics of each target and the security risk when the security risk level is dynamically evaluated by a deep learning model.

[0005] To achieve the above-mentioned purpose, the embodiments of the present application provide a real-time video anti-theft monitoring method based on Internet of Things, comprising:

[0006] acquiring monitoring data in a historical preset time period, and calculating a behavior purpose index of each dynamic target in the monitoring data;

[0007] determining a suspicious target in each dynamic target based on the behavior purpose index, and calculating a risk mapping index of the motion behavior of the suspicious target to a risk area;

[0008] determining a theft risk evaluation value of the suspicious target to the risk area based on the risk mapping index and the behavior purpose index;

[0009] based on the theft risk evaluation value, performing anti-theft monitoring on the suspicious target.

[0010] In a possible implementation manner of the present application, the behavior purpose index of each dynamic target in the monitoring data is calculated, comprising:

[0011] based on a preset target detection algorithm, performing frame labeling and frame selection on each static frame in the monitoring data to obtain a plurality of dynamic targets;

[0012] calculate behavior activity of each dynamic target, and screen scene data frames in which the dynamic target is blocked in the monitoring data;

[0013] In a case where an area in which the dynamic target is blocked in the scene data frame is greater than a preset area threshold, it is determined that the dynamic target is in an occlusion obstacle scene, and a motion trajectory continuity index of the dynamic target in the occlusion obstacle scene is calculated;

[0014] Based on the motion trajectory continuity index and the behavior activity, a behavior purpose index of each dynamic target in the monitoring data is determined.

[0015] In a possible implementation of the present application, the behavior activity of each dynamic target is calculated, including:

[0016] The joint coordinate data of each dynamic target is determined;

[0017] The mean square error of the joint coordinate data of the same dynamic target in adjacent static frames is calculated to obtain a shape change activity of the dynamic target;

[0018] The centroid coordinates of the dynamic target in each static frame are extracted, and a first displacement of the centroid coordinates of the same dynamic target between the current static frame and the adjacent static frame is calculated;

[0019] Based on the shape change activity and the first displacement, an instantaneous behavior activity of the dynamic target in each static frame is calculated;

[0020] The instantaneous behavior activities corresponding to each static frame are integrated to obtain the behavior activity of each dynamic target.

[0021] In a possible implementation of the present application, the motion trajectory continuity index of the dynamic target in the occlusion obstacle scene is calculated, including:

[0022] The last first data frame before the dynamic target is blocked and the first second data frame after the occlusion disappears are extracted;

[0023] The first motion direction of the dynamic target in the first data frame and the second motion direction of the dynamic target in the second data frame are calculated respectively;

[0024] A direction deviation coefficient between the first motion direction and the second motion direction is determined, and a time deviation coefficient between the first data frame and the second data frame is calculated;

[0025] Based on the direction deviation coefficient and the time deviation coefficient, the motion trajectory continuity index is calculated.

[0026] In a possible implementation of the present application, the first motion direction of the dynamic target in the first data frame is calculated, including:

[0027] extracting a plurality of optical flow vectors in the first data frame;

[0028] calculating an average of angles between the plurality of optical flow vectors and a horizontal direction, and taking the average of the angles as a first motion direction before the occlusion.

[0029] In a possible implementation of the present application, the time deviation coefficient between the first data frame and the second data frame is calculated, comprising:

[0030] obtaining an instantaneous speed of the dynamic target in the first data frame and a region width of the occlusion region;

[0031] determining a first time period required by the dynamic target to pass through the occlusion region based on the region width and the instantaneous speed;

[0032] determining a predicted appearance time point of the dynamic target based on the first time point corresponding to the first data frame and the first time period;

[0033] calculating a time difference value between the predicted appearance time point and a second time point corresponding to the second data frame to obtain the time deviation coefficient.

[0034] In a possible implementation of the present application, the risk mapping index of the motion behavior of the suspicious target to the risk region is calculated, comprising:

[0035] when the suspicious target appears in a monitoring range corresponding to the risk region, obtaining a plurality of real-time motion data frames of the suspicious target;

[0036] calculating a third motion direction of the suspicious target in the real-time motion data frames and a fourth motion direction relative to the risk region;

[0037] determining a motion direction sensitivity factor of the suspicious target based on the third motion direction and the fourth motion direction;

[0038] obtaining a first instantaneous behavior activity of the suspicious target in each static frame in the monitoring data, and calculating a motion activity feature based on each first instantaneous behavior activity;

[0039] calculating the risk mapping index based on the motion direction sensitivity factor and the motion activity feature.

[0040] In a possible implementation of the present application, the risk mapping index is calculated based on the motion direction sensitivity factor and the motion activity feature, comprising:

[0041] calculating a difference value of instantaneous activity of the first instantaneous behavior activity in adjacent static frames;

[0042] dividing the static frames with the difference value of instantaneous activity less than a preset activity threshold to a same similar motion activity stage;

[0043] Based on the motion activity feature, the activity difference value between similar motion activity stages of the suspicious target is calculated;

[0044] The first data sequence corresponding to the activity difference value and the second data sequence corresponding to the motion direction sensitive factor are constructed;

[0045] The Spearman correlation coefficient between the first data sequence and the second data sequence is calculated, and the Spearman correlation coefficient is taken as a risk mapping index.

[0046] In a possible implementation of the present application, based on the theft risk assessment value, the suspicious target is monitored against theft, comprising:

[0047] If the theft behavior risk assessment value is greater than a preset risk threshold, it is determined that the suspicious target has a theft risk, and a warning prompt is issued;

[0048] Based on the warning prompt, the suspicious target is tracked to generate a suspicious target behavior trajectory;

[0049] The suspicious target behavior trajectory is sent to relevant security personnel, so that the relevant security personnel track the suspicious target.

[0050] The present application also provides a real-time video theft prevention monitoring system based on the Internet of Things, comprising:

[0051] An acquisition module is configured to acquire monitoring data in a historical preset time period and calculate a behavior purpose index of each dynamic target in the monitoring data;

[0052] A calculation module is configured to determine a suspicious target among the dynamic targets based on the behavior purpose index and calculate a risk mapping index of motion behavior of the suspicious target to a risk area;

[0053] A determination module is configured to determine a theft risk assessment value of the suspicious target to the risk area based on the risk mapping index and the behavior purpose index;

[0054] A monitoring module is configured to monitor the suspicious target against theft based on the theft risk assessment value.

[0055] The application provides a real-time video anti-theft monitoring method and system based on Internet of Things, and in the related art, a safety risk level is dynamically evaluated through a deep learning model, and due to the high nonlinearity and coupling between the behavior characteristics of each target and the safety risk, the false positive rate and the false negative rate of the monitoring system are relatively high, in the application, monitoring data in a historical preset time period is acquired, and a behavior purpose index of each dynamic target in the monitoring data is calculated, a suspicious target with suspicious behavior is determined according to the behavior purpose index, a risk mapping index of the motion behavior of the suspicious target to a risk area is calculated, thereby a linear correlation between the motion behavior of the suspicious target and the risk area is established, and then the risk mapping index and the behavior purpose index are used to determine a theft risk evaluation value of the suspicious target to the risk area, and the corresponding suspicious target is selected for anti-theft monitoring according to the theft risk evaluation value, thereby the alarm accuracy of the monitoring system is improved, and the false positive rate and the false negative rate are reduced. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 A flowchart of a first embodiment of the real-time video anti-theft monitoring method based on Internet of Things of the application;

[0057] Figure 2 A dynamic target labeling diagram related to the real-time video anti-theft monitoring method based on Internet of Things of the application;

[0058] Figure 3 A flowchart of a second embodiment of the real-time video anti-theft monitoring method based on Internet of Things of the application;

[0059] Figure 4 A device structure diagram of a hardware running environment related to the embodiment scheme of the application. DETAILED DESCRIPTION

[0060] It should be understood that the specific embodiments described herein are merely intended to explain the application, and are not intended to limit the application.

[0061] The application provides a real-time video anti-theft monitoring method based on Internet of Things, and in the first embodiment of the real-time video anti-theft monitoring method based on Internet of Things of the application, referring to Figure 1 , the method comprises:

[0062] Step S10, acquiring monitoring data in a historical preset time period, and calculating a behavior purpose index of each dynamic target in the monitoring data;

[0063] Step S20, determining a suspicious target in each dynamic target based on the behavior purpose index, and calculating a risk mapping index of the motion behavior of the suspicious target to a risk area;

[0064] Step S30: Based on risk mapping indicators and behavioral purpose indicators, determine the theft risk assessment value of suspicious targets for risk areas;

[0065] Step S40: Based on the theft risk assessment value, conduct anti-theft monitoring on suspicious targets.

[0066] This embodiment aims to improve the alarm accuracy of the monitoring system and reduce the false alarm rate and missed alarm rate.

[0067] The specific steps are as follows:

[0068] Step S10: Obtain monitoring data within a historical preset time period and calculate the behavioral purpose indicators of each dynamic target in the monitoring data.

[0069] As an example, the IoT-based real-time video anti-theft monitoring method can be applied to IoT-based real-time video anti-theft monitoring devices, which belong to IoT-based real-time video anti-theft monitoring systems, and these systems belong to IoT-based real-time video anti-theft monitoring equipment.

[0070] As an example, the preset historical time period can be a collection of real-time images from the previous 1 minute, 5 minutes, or 10 minutes. This application is mainly applied to a real-time video anti-theft monitoring system based on the Internet of Things (IoT). This system adopts a three-level deployment architecture of "three-dimensional coverage + dynamic focusing + multi-camera collaboration," and combines IoT edge computing nodes to build an intelligent data acquisition network. The specific deployment strategy is as follows:

[0071] First, 1080P high-definition cameras are deployed at each entrance and passageway hub in open areas, covering the front view (120° ultra-wide angle horizontally), side view (90° rotatable pan-tilt head, vertical adjustment range ±30°) and rear view (80° fixed viewing angle) areas respectively, to achieve 360° surround view monitoring.

[0072] The deployment density of cameras is dynamically adjusted according to the risk level of the monitored area. The deployment spacing in core areas or high-risk areas (such as vaults and data centers) is ≤5 meters, and the deployment spacing in ordinary areas (such as corridors and perimeters) is ≤15 meters.

[0073] Each high-definition camera analyzes the data from the front-view main camera at a default full frame rate of 15fps. When a suspicious moving target is detected, 50% of the computing power is dynamically allocated to the associated side-view / rear-view cameras to improve the analysis accuracy of local areas.

[0074] Based on motion vector and ROI detection algorithms, image frames containing dynamic targets are extracted (at intervals of 0.5s ± 0.1s), while maintaining a resolution of 1080P.

[0075] Key video frames are uploaded to the cloud in real time via 4G / 5G modules (at 0.5-second intervals), and all videos are stored in a hierarchical manner according to events (ordinary events are retained for 7 days, and serious events are retained for 90 days).

[0076] Ultimately, the perception layer uses IoT protocols (MQTT+HTTP / 2) to achieve millisecond-level responses between devices, ensuring data consistency across all cameras.

[0077] As an example, there may be multiple dynamic targets in the monitoring data. These dynamic targets can be people, animals, or other mobile targets. These dynamic targets will have certain behavioral characteristics. For example, if a person in the monitoring data is going to buy clothes, then their behavioral characteristics will be that they go to the clothing store and their movement trajectory will be around the clothing store.

[0078] As an example, the behavioral purposefulness index represents an indicator related to the purpose of movement of a dynamic target. When a person exhibits abnormal behavior (such as theft), the criminal will evade surveillance or quickly approach the target. If the trajectory can still maintain continuity in an occluded scenario, it indicates that the movement behavior of the dynamic target is highly purposeful, rather than randomly wandering in the monitored area.

[0079] Among them, step S10 of the real-time video anti-theft monitoring based on the Internet of Things also includes steps S11 to S14, including:

[0080] Step S11: Based on the preset target detection algorithm, mark and select each static frame in the monitoring data to obtain multiple dynamic targets.

[0081] As an example, the preset target detection algorithm could be the YOLO target detection algorithm, which is a real-time target detection and tracking method based on deep learning. According to this algorithm, targets in each static frame corresponding to the monitoring data can be identified and labeled, resulting in multiple target boxes. Within each target box, there is a corresponding dynamic target. At the same time, target tracking can be achieved, and the target boxes corresponding to the same dynamic target in different static frames can be determined.

[0082] As an example, the schematic diagram after labeling and selecting each dynamic target is as follows: Figure 2 As shown, by Figure 2 Thus, in a static frame of the monitoring data, each person has a labeled box.

[0083] Step S12: Calculate the activity level of each dynamic target and filter the scene data frames in the monitoring data when the dynamic target is occluded.

[0084] As an example, behavioral activity represents the level of activity of each dynamic target. For instance, if a person moves a long distance or makes large movements over a period of time, the behavioral activity of that dynamic target can be considered high. Behavioral activity includes the activity of limb changes and the activity of displacement changes. By considering both aspects, the behavioral activity of the dynamic target can be calculated.

[0085] As an example, due to site limitations, blind spots will inevitably appear after the cameras are deployed. In this case, the monitoring data will contain data frames where the target is obscured, which are also scene data frames. The obscuration can be half or all of the target's area, and there is no specific limitation.

[0086] The step S12, which calculates the behavioral activity of each dynamic target, includes:

[0087] Determine the key point coordinates of each dynamic target.

[0088] As an example, the key point coordinate data can be the coordinates of each key point of a dynamic target within the selected area. Key points can be joints such as wrist joints and knee joints. The coordinate data set of each key point is output by the HRNet model (a deep learning model for computer vision tasks).

[0089] The mean square error of the joint coordinate data of the same dynamic target in adjacent static frames is calculated to obtain the activity level of the dynamic target's morphological changes.

[0090] As an example, the purpose of calculating the mean square error (MSE) is to: calculate the degree of morphological change of the dynamic target between the current static frame and adjacent static frames (such as the previous static frame), reflect the changes in the keypoint coordinate data through the MSE, and measure the change through the mean square error (MSE). a D b ) represents the mean square error, where a and b represent the adjacent static frame (frame a) and the current static frame (frame b) in the joint coordinate data, respectively. a D represents the set of coordinate positions of each key point in frame a. b This represents the set of coordinate positions of each key point in frame b. This value is used to initially determine the activity level of the dynamic target's movement in the current static frame (frame b), that is, the activity level of its movement changes.

[0091] Extract the centroid coordinates of the dynamic target in each static frame, and calculate the first displacement of the centroid coordinates of the same dynamic target between the current static frame and the adjacent static frames.

[0092] As an example, centroid coordinates mainly quantize each dynamic target into a coordinate point. The purpose of extracting centroid coordinates is to calculate the displacement distance between two static frames. The first displacement is the displacement distance moved by the dynamic target between two different static frames.

[0093] Specifically, the centroid coordinates of the selected area corresponding to the dynamic target in each static frame are obtained based on the spatial coordinate system, and the displacement d of the centroid coordinates of the same dynamic target in the current static frame and the adjacent static frame (such as the previous static frame) is calculated.

[0094] Based on the morphological change activity and the first displacement, the instantaneous behavioral activity of the dynamic target in each static frame is calculated.

[0095] As an example, instantaneous behavioral activity is used to represent the behavioral activity of a dynamic target over a short period of time (e.g., between two static frames). Before calculating instantaneous behavioral activity, it is desirable to obtain the instantaneous displacement change of the dynamic target based on the first displacement. The instantaneous displacement change can be calculated as follows:

[0096]

[0097] Where, d t The value represents the time difference between adjacent static frames, and v represents the instantaneous displacement change of the region. The larger the value, the greater the instantaneous displacement of the target.

[0098] Furthermore, the morphological change activity of dynamic targets in adjacent static frames (MSE(D)) is combined with the morphological change activity of dynamic targets in adjacent static frames. a D b The product of the instantaneous displacement change v in the region: E = MSE(D) a D b )×v, to obtain the instantaneous behavioral activity E of the dynamic target.

[0099] The instantaneous behavioral activity corresponding to each static frame is integrated to obtain the behavioral activity of each dynamic target.

[0100] As an example, one way to integrate the various instantaneous behavioral activity levels is to calculate the instantaneous behavioral activity level E of the dynamic target corresponding to all static frame images, and calculate its mean μ(E), which is regarded as the behavioral activity level of the dynamic target in the current stage. The larger the value, the more active the dynamic target is in the current stage, such as running fast or vigorous exercise.

[0101] Step S13: If the area of ​​the dynamic target being occluded in the scene data frame is greater than a preset area threshold, determine that the dynamic target is in an occlusion obstacle scene, and calculate the motion trajectory continuity index of the dynamic target in the occlusion obstacle scene.

[0102] As an example, in actual monitoring, dynamic targets are often partially or completely obscured by other objects (such as pedestrians, decorations, etc.), and fixed cameras cannot penetrate the obstructions, resulting in fragmented target behavior data and affecting the integrity of subsequent analysis. Therefore, it is necessary to analyze scene data frames in occluded scenarios and track the bounding boxes of dynamic targets using the YOLO target detection algorithm. When the area of ​​the target bounding box decreases (e.g., by 50%, where 50% can be a preset area threshold) or the bounding box disappears (the target is lost in consecutive image frames), it is determined that the dynamic target is in an occluded obstacle scenario.

[0103] As an example, the motion trajectory continuity index is used to represent the motion continuity of dynamic targets. It is used to analyze whether the target deliberately maintains its motion trajectory in occluded obstacle scenarios, thereby determining whether its behavior has the intent to steal.

[0104] Step S13, which calculates the continuity index of the motion trajectory of a dynamic target in an occluded obstacle scenario, includes:

[0105] Extract the last first data frame of the dynamic target before it is occluded and the first second data frame of the dynamic target after the occlusion is removed.

[0106] As an example, the first data frame is the last data frame of the dynamic target before it is occluded, and the second data frame is the first data frame of the dynamic target after the occlusion is removed.

[0107] Calculate the first direction of motion of the dynamic target in the first data frame and the second direction of motion of the dynamic target in the second data frame.

[0108] As an example, the first direction of motion is the direction of motion of the dynamic target before it is occluded, and the second direction of motion is the direction of motion of the dynamic target after it is occluded. By comparing the changes in the direction of motion of the dynamic target before and after it is occluded, the continuity of the target's motion is determined.

[0109] The step of calculating the first direction of motion of the dynamic target in the first data frame includes:

[0110] Extract multiple optical flow vectors from the first data frame.

[0111] As an example, the optical flow method is used to extract the set of optical flow vectors corresponding to the dynamic target region in the first data frame. This set of optical flow vectors is represented as: {o1, o2, ..., o n}, where o n This represents the nth optical flow vector. Optical flow is a technique well-known to those skilled in the art and will not be elaborated upon here.

[0112] Calculate the average angle between each optical flow vector and the horizontal direction, and use the average angle as the first direction of motion before occlusion.

[0113] As an example, the mean angle between all optical flow vectors and the horizontal direction is calculated: θ, which is the first direction of motion of the dynamic target before occlusion. Similarly, the second direction of motion θ′ can be calculated.

[0114] Determine the directional deviation coefficient between the first and second motion directions, and calculate the time deviation coefficient between the first and second data frames.

[0115] As an example, the directional deviation coefficient between the first and second motion directions is expressed as: directional deviation before and after dynamic target occlusion: |θ-θ′|, with a value of Δθ (normalized simultaneously using the norm function). It is defined as the directional deviation coefficient before and after dynamic target occlusion, which quantifies the stability of the target's motion direction before and after occlusion. The smaller the value, the more consistent the target's motion intention is when it encounters an obstacle and maintains its original direction of motion.

[0116] As an example, the time deviation coefficient represents the time deviation of a dynamic target before and after being occluded, quantifying the motion stability when traversing the occluded area. The smaller the value, the better the continuity of the dynamic target's trajectory.

[0117] The step of calculating the time deviation coefficient between the first data frame and the second data frame includes:

[0118] Obtain the instantaneous velocity of the dynamic target and the width of the occluded area in the first data frame;

[0119] As an example, during the extraction of the first data frame, the instantaneous velocity v of the moving target in the previous frame of image data that is occluded can be obtained. t And the width w of the occluded area (converted to actual distance via camera calibration).

[0120] Based on the area width and instantaneous velocity, determine the first time period required for a dynamic target to traverse the occluded area;

[0121] As an example, the time required for a dynamic target to traverse the occlusion area, i.e., the first time period, is calculated based on the area width and instantaneous velocity. The first time period is calculated as follows:

[0122]

[0123] Among them, t pred This represents the first time period, where w is the width of the region, and v t This refers to instantaneous velocity.

[0124] Based on the first time point and the first time period corresponding to the first data frame, the expected time point of appearance of the dynamic target is determined;

[0125] As an example, adding the first time point to the first time period yields the expected occurrence time H of the dynamic target. pred H pred =t+t pred , where t represents the first time point.

[0126] The time difference between the expected occurrence time and the second time point corresponding to the second data frame is calculated to obtain the time deviation coefficient.

[0127] As an example, calculate the time difference between the estimated occurrence time and the actual occurrence time of a dynamic target: |H pred -l|, where l represents the time point when the dynamic target appears after the occlusion disappears, and the value ΔT is defined as the time deviation coefficient before and after the dynamic target is occluded. It quantifies the motion stability of the target when crossing the occluded area. The smaller the value, the more it reflects that the target strictly follows the uniform linear motion model when crossing the occluded area, and its motion trajectory is highly predictable, which indirectly reflects that the motion trajectory of the target is relatively continuous.

[0128] The continuity index of the motion trajectory is calculated based on the directional deviation coefficient and the time deviation coefficient.

[0129] As an example, the motion trajectory continuity index represents the continuity of motion over a period of time. The continuity index between two data frames can be calculated first. This continuity index can be calculated in the following ways:

[0130]

[0131] Where R represents the continuity index, Δθ is the direction deviation coefficient, and ΔT is the time deviation coefficient; ∈ represents an infinite decimal greater than 0, used to prevent the denominator from being 0.

[0132] As an example, the R value is a comprehensive quantitative index of the continuity of the target's motion trajectory, which directly reflects the ability of a dynamic target to maintain its original motion intention in an occluded scene.

[0133] As an example, the mean value μ(R) of the continuity index of the motion trajectory before and after each occlusion is extracted from the real-time image dataset of the dynamic target over a historical ten-minute period (containing at least one occurrence of occlusion; otherwise, the historical dataset is extended until it contains one occlusion). This value is regarded as the continuity index of the target's motion trajectory in the current stage.

[0134] Step S14: Based on the motion trajectory continuity index and behavioral activity, determine the behavioral purpose index of each dynamic target in the monitoring data.

[0135] As an example, the calculation method for behavioral purposefulness indicators can be:

[0136] M = norm(μ(R) + μ(E)), where M is defined as the behavioral purposefulness index of a dynamic objective;

[0137] Where μ(R) represents the continuity index of the motion trajectory, μ(E) represents the activity level of the behavior, and norm() represents the normalization calculation.

[0138] Step S20: Based on the behavioral purpose index, identify suspicious targets among the dynamic targets and calculate the risk mapping index of the movement behavior of the suspicious targets to the risk area.

[0139] As an example, when a dynamic target is determined to have strong purposefulness, suspicious targets among the dynamic targets are identified based on the behavioral purposefulness index corresponding to the dynamic target. For example, if the behavioral purposefulness index of dynamic target A is greater than 0.7, it is marked as a suspicious target.

[0140] As an example, after identifying a suspicious target, computing power is dynamically allocated to key node cameras (such as vaults, data centers, etc.) based on the global perception layer to enhance the further decoupling analysis of the suspicious target's behavioral characteristics. In turn, the correlation between the suspicious target's movement behavior and the risk area is determined, that is, whether the suspicious target has a tendency to move towards the risk area.

[0141] As an example, the risk mapping index is determined by two parts: direction of movement and activity level. The direction of movement is mainly used to determine whether the suspicious target has a tendency to move towards the risk area, while the activity level is used to determine the abnormal behavior of the suspicious target (such as rapid movement or short stay). The risk of theft by the suspicious target is determined based on these two indicators, thus obtaining the risk mapping index.

[0142] Step S30: Based on risk mapping indicators and behavioral purpose indicators, determine the theft risk assessment value of suspicious targets for risk areas.

[0143] As an example, the theft risk assessment value can be calculated as follows:

[0144]

[0145] In the formula, norm is the normalization function, ρ represents the risk mapping index, KPI represents the risk assessment value of the theft behavior of a specific dynamic target, and M represents the behavior purpose index. This value comprehensively quantifies the anomaly of the current dynamic target's movement trajectory in an open scene. The larger the value, the greater the suspicion that the corresponding dynamic target has theft behavior.

[0146] Step S40: Based on the theft risk assessment value, conduct anti-theft monitoring on suspicious targets.

[0147] As an example, after determining the theft risk assessment value of a suspicious target, the decision is made based on the magnitude of the theft risk assessment value to determine whether to continuously track and monitor the suspicious target or to issue an early warning, and then to carry out the next step of theft prevention measures.

[0148] The step S40, which involves monitoring suspicious targets for theft based on a theft risk assessment value, includes:

[0149] Step S41: If the risk assessment value of the theft behavior is greater than the preset risk threshold, it is determined that the suspicious target poses a theft risk and an early warning is issued.

[0150] As an example, the preset risk threshold can be 0.6, 0.7, etc., and there is no specific limitation.

[0151] As an example, with a preset risk threshold of 0.7, when the risk assessment value of theft behavior is greater than 0.7, it is determined that the suspicious target poses a theft risk. The system immediately activates the three-level emergency response mechanism and issues an early warning.

[0152] Step S42: Track suspicious targets based on early warning prompts and generate suspicious target behavior trajectories.

[0153] Specifically, the system sends real-time alerts to the security system via an alarm mechanism, arranging for security personnel to check for stolen items. Simultaneously, it pushes target feature vectors (including HSV color histograms, motion acceleration vectors, and skeletal joint topology data) to cameras within a 200-meter radius via the LoRaWAN broadcast protocol, initiating a cluster collaborative tracking mode. Then, it dynamically adjusts the computing power allocation strategy based on the network topology, increasing the frame rate of associated cameras from 15fps to 30fps (using inter-frame interpolation technology to compensate for dropped frames) and the bitrate from 4Mbps to 12Mbps (enabling H.266VVC encoded CU extended partitioning mode). In addition, to ensure accurate target locking, the system allocates an additional 23% of GPU computing power to cameras within the target area and uses containerization technology to achieve millisecond-level resource preemption, ensuring that at least three cameras continuously track the target contour with 0.5° accuracy, accurately acquiring the behavioral trajectory of suspicious individuals, and thus generating the behavioral trajectory of the suspicious target.

[0154] Step S43: Send the suspicious target's behavioral trajectory to the relevant security personnel so that they can track the suspicious target.

[0155] As an example, after security personnel determine that an item has been stolen, the behavioral trajectory of the suspicious target is sent to them so that they can track the suspicious target.

[0156] This application provides a real-time video anti-theft monitoring method and system based on the Internet of Things. Compared with related technologies that use deep learning models to dynamically assess security risk levels, which are limited by the highly nonlinear and coupled relationship between the behavioral characteristics of each target and security risks, resulting in a high false alarm rate and missed alarm rate, this application obtains monitoring data within a preset historical time period and calculates the behavioral purpose index of each dynamic target in the monitoring data. Based on the behavioral purpose index, suspicious targets with suspicious behavior are identified. Then, the risk mapping index between the movement behavior of suspicious targets and risk areas is calculated to establish a linear correlation between the movement behavior of suspicious targets and risk areas. Based on the risk mapping index and the behavioral purpose index, the theft risk assessment value of the suspicious target for the risk area is determined. Using the theft risk assessment value as a benchmark, the corresponding suspicious targets are selected for anti-theft monitoring, thereby improving the alarm accuracy of the monitoring system and reducing the false alarm rate and missed alarm rate.

[0157] Furthermore, referring to Figure 3 Based on the first embodiment of this application, another embodiment of this application is provided. In this embodiment, step S20, which calculates the risk mapping index of the movement behavior of a suspicious target for a risk area, includes:

[0158] Step S21: When a suspicious target appears within the monitoring range corresponding to the risk area, acquire multiple real-time motion data frames of the suspicious target.

[0159] As an example, when a suspicious target appears within the monitoring range corresponding to a risk area (identified by facial or behavioral features M), the module further predicts its direction of movement, determines whether it is moving towards a sensitive area (such as a vault, data center, etc.), and obtains multiple real-time motion data frames of the suspicious target.

[0160] Step S22: Calculate the third motion direction of the suspicious target in the real-time motion data frame and the fourth motion direction relative to the risk area.

[0161] As an example, the optical flow vector set of adjacent static frames during the motion process of the suspected target is extracted by optical flow method, and the mean value of the angle between all optical flow vectors and the horizontal direction (note that the value range of this angle is 0°<θ1<90°) is calculated as θ1, which is regarded as the third motion direction of the suspected target.

[0162] As an example, connect the centroid of the selected area corresponding to the suspicious target with the centroid of the risk area, and obtain the angle between the line connecting the two points and the horizontal direction: θ2, (0°<θ2<90°). θ2 is used as the fourth direction of movement of the suspicious target relative to the risk area.

[0163] Step S23: Based on the third and fourth motion directions, determine the motion direction sensitivity factor of the suspected target.

[0164] As an example, the interaction direction of a suspicious target is quantified as follows: Q = 1 - cos|θ1 - θ2|. The value Q is defined as the sensitivity factor of the suspicious target's current movement direction, quantifying the interaction between the suspicious target's movement behavior and the risk area at the current stage. When Q is 0, the angles corresponding to θ1 and θ2 are the same, indicating that the suspicious target is moving towards the risk area. When the suspicious target leaves the risk area after completing the theft, the angles corresponding to θ1 and θ2 are the same, and the value of Q is also 0. In other words, when a suspicious target is engaged in theft, the sensitivity factors of its movement direction corresponding to its approach and departure behaviors are both small.

[0165] When other targets inadvertently approach or deviate from the risk area, the target's movement behavior is uncertain. In this case, the larger the value of Q, the larger the angle between θ1 and θ2, which reflects that the movement behavior of the suspicious target at the current stage is moving away from the risk area. The smaller the value of Q, the more likely the suspicious target is approaching the risk area.

[0166] Step S24: Obtain the first instantaneous behavioral activity level of the suspicious target in each static frame of the monitoring data, and calculate the motion activity level feature based on each first instantaneous behavioral activity level.

[0167] As an example, when analyzing the movement trajectory of a criminal suspect, if the target does not show significant approach or departure from a sensitive area, it can be inferred that the suspect is "reconnaissance" or observing the withdrawal behavior.

[0168] Based on this, further analysis is conducted using the combination of "short-term pauses" and "rapid movements" in the target's complete movement trajectory to identify potential signs of crime. This behavioral pattern can correspond to a suspect's "reconnaissance" preparation activities before a theft or their escape actions after the theft.

[0169] As an example, the first instantaneous behavioral activity level E corresponding to the suspicious target in each static frame of the monitoring data is obtained, and the instantaneous activity level difference ΔE between the first instantaneous behavioral activity levels of adjacent static frames is calculated. Static frames with instantaneous activity level differences less than a preset threshold are divided into the same similar motion activity level stage. Then, the mean instantaneous behavioral activity level μ(E) of all static frames in all similar motion activity level stages is calculated as the motion activity feature of the suspicious target in the current similar motion activity level stage.

[0170] Step S25: Based on the motion direction sensitivity factor and motion activity characteristics, the risk mapping index is calculated.

[0171] Step S25, which calculates the risk mapping index based on the motion direction sensitivity factor and motion activity characteristics, includes:

[0172] Calculate the instantaneous activity difference of the first instantaneous behavior activity in adjacent static frames.

[0173] Static frames whose instantaneous activity difference is less than a preset activity threshold are classified into the same similar motion activity stage.

[0174] As an example, the preset activity threshold can be 0.5 or other values, without any specific limitation.

[0175] As an example, static frames with an instantaneous activity difference ΔE less than a preset activity threshold are grouped into the same similar motion activity stage. That is, when a static frame with ΔE greater than or equal to the preset activity threshold is encountered, the division is terminated, and the stage division is restarted from the current static frame until the entire dataset of monitoring data is traversed, resulting in multiple similar motion activity stages.

[0176] Based on motion activity characteristics, calculate the activity difference value between static frames corresponding to similar motion activity stages of suspicious targets;

[0177] As an example, the mean instantaneous behavioral activity of all static frames within all similar motion activity stages is calculated and regarded as the motion activity feature of that stage. The mean activity values ​​corresponding to different similar motion activity stages are also different.

[0178] As an example, based on the activity difference between adjacent similar motion activity stages, the combination of "short-term stay" and "rapid movement" behavior of the target is identified. Before the target prepares to carry out a theft operation, this combination of "short-term stay" and "rapid movement" behavior will exist when approaching or moving away from the risk area. The activity difference between adjacent similar motion activity stages is recorded as the activity difference value. The activity difference value can be calculated as follows:

[0179]

[0180] Where i represents the i-th similar motion activity stage in the monitoring dataset collected by the system, i+1 and i-1 represent the similar motion activity stages on the left and right sides of stage i, respectively, and μ(E) i-1 ,μ(E) i ,μ(E) i+1 Let W represent the mean activity levels at stages i-1, i, and i+1, respectively, where ε = 1e-5. i This represents the difference in activity level between the i-th and i-th similar activity levels; the formula takes the value W. iThe larger the value, the more likely it is that the target areas in the i+1 and i-1 stages of the monitoring dataset have similar levels of motion activity, while there are significant differences in motion activity between i+1 and i stages. Moreover, the motion activity in i+1 stage is significantly greater than that in i stage, which corresponds to the combination of "short stay (i stage)" and "rapid movement (i-1 stage, i+1 stage)" behaviors in the movement trajectory of the suspicious target.

[0181] Construct a first data sequence corresponding to the activity difference value and a second data sequence corresponding to the motion direction sensitivity factor;

[0182] As an example, the statistical system collects the motion direction sensitivity factor Q for each static frame in the image dataset, and the activity difference value W corresponding to the similar motion activity stage of that static frame. i Construct data sequences A(Q) (the second data sequence) and B(W) respectively. i (First data sequence).

[0183] Calculate the Spearman correlation coefficient between the first and second data sequences, and use the Spearman correlation coefficient as a risk mapping indicator.

[0184] As an example, calculate the data sequences A(Q) and B(W). i The Spearman correlation coefficient between data sets A(Q) and B(W) is ρ, considered as a risk mapping indicator for the target. Its value ranges from -1 to +1. The closer the value is to -1, the more sensitive the risk is to the target. i The stronger the negative correlation between W and W, the more likely it is that when a suspicious target exhibits a combination of "short-term stay" and "rapid movement," that is, when approaching or moving away from a risk area, W will be stronger. i When the Q value increases and decreases, it indicates that the trajectory of the dynamic target is deviating from the trajectory of approaching or moving away from the risk area, suggesting that the current dynamic target poses a high risk of theft.

[0185] In this embodiment, a risk mapping index is calculated by combining the motion direction sensitivity factor and activity difference value of the suspected target, thereby determining whether the dynamic target poses a theft risk.

[0186] This application also provides an IoT-based real-time video anti-theft monitoring system, which includes:

[0187] The acquisition module is used to acquire monitoring data within a preset historical time period and calculate the behavioral purpose indicators of each dynamic target in the monitoring data.

[0188] The calculation module is used to identify suspicious targets among various dynamic targets based on behavioral purpose indicators, and to calculate the risk mapping indicators of the movement behavior of suspicious targets for risk areas.

[0189] The determination module is used to determine the theft risk assessment value of suspicious targets for risk areas based on risk mapping indicators and behavioral purpose indicators.

[0190] The monitoring module and the acquisition module are used to monitor suspicious targets for theft prevention based on theft risk assessment values.

[0191] Reference Figure 4 , Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application.

[0192] like Figure 4 As shown, the IoT-based real-time video anti-theft monitoring device may include: a processor 1001, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the processor 1001 and the memory 1005.

[0193] Optionally, the IoT-based real-time video surveillance device may also include a user interface, a network interface, a camera, RF (Radio Frequency) circuitry, sensors, a WiFi module, etc. The user interface may include a display screen and an input submodule such as a keyboard; optional user interfaces may also include standard wired or wireless interfaces. The network interface may include standard wired or wireless interfaces (such as a Wi-Fi interface).

[0194] Those skilled in the art will understand that Figure 4 The structure of the IoT-based real-time video anti-theft monitoring device shown in the figure does not constitute a limitation on the IoT-based real-time video anti-theft monitoring device. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0195] like Figure 4 As shown, the memory 1005, serving as a storage medium, may include an operating system, a network communication module, and an IoT-based real-time video surveillance program. The operating system is a program that manages and controls the hardware and software resources of the IoT-based real-time video surveillance device, supporting the operation of the IoT-based real-time video surveillance program and other software and / or programs. The network communication module is used to enable communication between the various components within the memory 1005, as well as communication with other hardware and software in the IoT-based real-time video surveillance system.

[0196] exist Figure 4In the IoT-based real-time video anti-theft monitoring device shown, the processor 1001 is used to execute the IoT-based real-time video anti-theft monitoring program stored in the memory 1005 to implement the steps of any of the above-mentioned IoT-based real-time video anti-theft monitoring methods.

[0197] The specific implementation method of the IoT-based real-time video anti-theft monitoring device in this application is basically the same as the embodiments of the IoT-based real-time video anti-theft monitoring method described above, and will not be repeated here.

[0198] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0199] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0200] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.

[0201] The above are merely preferred embodiments of this application and do not limit the scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the scope of protection of this application.

[0202] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0203] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A real-time video anti-theft monitoring method based on the Internet of Things, characterized in that, The method includes: Acquire monitoring data within a preset historical time period and calculate the behavioral purpose indicators of each dynamic target in the monitoring data; The calculation of the behavioral purpose indicators of each dynamic target in the monitoring data includes: Based on a preset target detection algorithm, each static frame in the monitoring data is marked and selected to obtain multiple dynamic targets; Calculate the behavioral activity of each dynamic target and filter the scene data frames in the monitoring data when the dynamic targets are occluded; If the area of ​​the dynamic target being occluded in the scene data frame is greater than a preset area threshold, it is determined that the dynamic target is in an occlusion obstacle scene, and the motion trajectory continuity index of the dynamic target in the occlusion obstacle scene is calculated. Based on the motion trajectory continuity index and the behavioral activity level, determine the behavioral purpose index of each dynamic target in the monitoring data; Based on the behavioral purpose index, suspicious targets among the dynamic targets are identified, and the risk mapping index of the movement behavior of the suspicious targets to the risk area is calculated. Based on the risk mapping index and the behavioral purpose index, the theft risk assessment value of the suspicious target for the risk area is determined; Based on the theft risk assessment value, the suspicious target is monitored for theft prevention.

2. The real-time video anti-theft monitoring method based on the Internet of Things as described in claim 1, characterized in that, The calculation of the behavioral activity of each of the dynamic targets includes: Determine the key point coordinate data of each dynamic target; The mean square error of the joint coordinate data of the same dynamic target in adjacent static frames is calculated to obtain the morphological change activity of the dynamic target. Extract the centroid coordinates of the dynamic target in each static frame, and calculate the first displacement of the centroid coordinates of the same dynamic target between the current static frame and the adjacent static frames; Based on the morphological change activity and the first displacement, the instantaneous behavioral activity of the dynamic target in each static frame is calculated; The instantaneous behavioral activity corresponding to each static frame is integrated to obtain the behavioral activity of each dynamic target.

3. The real-time video anti-theft monitoring method based on the Internet of Things as described in claim 1, characterized in that, The calculation of the motion trajectory continuity index of the dynamic target in the occlusion obstacle scenario includes: Extract the last first data frame of the dynamic target before it is occluded and the first second data frame of the dynamic target after the occlusion is removed; Calculate the first direction of motion of the dynamic target in the first data frame and the second direction of motion of the dynamic target in the second data frame, respectively. Determine the directional deviation coefficient between the first motion direction and the second motion direction, and calculate the time deviation coefficient between the first data frame and the second data frame; Based on the directional deviation coefficient and the time deviation coefficient, the motion trajectory continuity index is calculated.

4. The real-time video anti-theft monitoring method based on the Internet of Things as described in claim 3, characterized in that, The calculation of the first motion direction of the dynamic target in the first data frame includes: Extract multiple optical flow vectors from the first data frame; Calculate the average angle between each optical flow vector and the horizontal direction, and use the average angle as the first direction of motion before occlusion.

5. The real-time video anti-theft monitoring method based on the Internet of Things as described in claim 3, characterized in that, The calculation of the time deviation coefficient between the first data frame and the second data frame includes: Obtain the instantaneous velocity of the dynamic target and the width of the occluded area in the first data frame; Based on the width of the region and the instantaneous velocity, the first time period required for the dynamic target to traverse the occlusion region is determined; Based on the first time point corresponding to the first data frame and the first time period, the expected time point of appearance of the dynamic target is determined; The time difference between the expected occurrence time and the second time point corresponding to the second data frame is calculated to obtain the time deviation coefficient.

6. The real-time video anti-theft monitoring method based on the Internet of Things as described in claim 2, characterized in that, The calculation of the risk mapping index of the movement behavior of the suspicious target to the risk area includes: When the suspicious target appears within the monitoring range corresponding to the risk area, multiple real-time motion data frames of the suspicious target are acquired; Calculate the third motion direction of the suspected target in the real-time motion data frame and the fourth motion direction relative to the risk area; Based on the third and fourth directions of motion, the motion direction sensitivity factor of the suspected target is determined; The instantaneous behavioral activity level of the suspicious target in each static frame of the monitoring data is obtained, and the motion activity level feature is calculated based on each instantaneous behavioral activity level. The motion activity characteristics calculated based on the instantaneous behavioral activity include: Calculate the instantaneous activity difference between adjacent static frames, divide each static frame whose instantaneous activity difference is less than a preset threshold into the same similar motion activity stage, calculate the average instantaneous activity of all static frames in all similar motion activity stages, and use the average instantaneous activity as the motion activity feature of the suspicious target in the current similar motion activity stage; Based on the motion direction sensitivity factor and the motion activity characteristics, a risk mapping index is calculated.

7. The real-time video anti-theft monitoring method based on the Internet of Things as described in claim 6, characterized in that, The risk mapping index, calculated based on the motion direction sensitivity factor and the motion activity characteristics, includes: Based on the aforementioned motion activity characteristics, the activity difference value of the suspected target between the similar motion activity stages is calculated; Construct a first data sequence corresponding to the activity difference value and a second data sequence corresponding to the motion direction sensitivity factor; Calculate the Spearman correlation coefficient between the first data sequence and the second data sequence, and use the Spearman correlation coefficient as a risk mapping indicator.

8. The real-time video anti-theft monitoring method based on the Internet of Things as described in claim 1, characterized in that, The process of monitoring the suspicious target for theft based on the theft risk assessment value includes: If the theft risk assessment value is greater than the preset risk threshold, it is determined that the suspicious target poses a theft risk, and an early warning is issued. Based on the aforementioned warning, the suspicious target is tracked, and a behavioral trajectory of the suspicious target is generated; The suspicious target's behavioral trajectory is sent to relevant security personnel so that they can track the suspicious target.

9. A real-time video anti-theft monitoring system based on the Internet of Things, characterized in that, For performing the IoT-based real-time video anti-theft monitoring method as described in any one of claims 1 to 8, the IoT-based real-time video anti-theft monitoring system comprises: The acquisition module is used to acquire monitoring data within a preset historical time period and calculate the behavioral purpose indicators of each dynamic target in the monitoring data. The calculation module is used to determine suspicious targets among the dynamic targets based on the behavioral purposefulness index, and to calculate the risk mapping index of the movement behavior of the suspicious targets for the risk area; The determination module is used to determine the theft risk assessment value of the suspicious target for the risk area based on the risk mapping index and the behavioral purpose index; The monitoring module, wherein the acquisition module is used to perform anti-theft monitoring on the suspicious target based on the theft risk assessment value.

Images