Security and protection monitoring data security management method and system based on Internet of Things

Abstract

The invention relates to the technical field of remote monitoring, and discloses a security and protection monitoring data security management method and system based on the Internet of Things, and the method comprises the steps: extracting a dynamic object key frame and a dimension reduction audio feature of block multi-mode security and protection data; dividing the block multi-mode security and protection data into sensitive biological characteristic data and conventional monitoring metadata; storing the sensitive biological characteristic data to a private chain to obtain private chain sensitive data, storing the conventional monitoring metadata to a public chain to obtain public chain conventional data, and establishing a three-level threat detection network of the private chain sensitive data and the public chain conventional data; identifying an abnormal behavior mode of the security and protection monitoring scene by using the edge layer so as to analyze global abnormal model parameters of the security and protection monitoring scene by using the fog computing layer; and analyzing the abnormal state of the security and protection monitoring scene by using the global enhancement layer, and generating a hierarchical response strategy of the security and protection monitoring scene according to the abnormal state. According to the invention, the recognition speed and accuracy of the key threats of the security and protection monitoring scene can be improved.

Description

TECHNICAL FIELD

[0001] The application relates to a security monitoring data security management method and system based on the Internet of Things, and belongs to the technical field of remote monitoring. BACKGROUND

[0002] Security monitoring data security management refers to the protection and management of data generated in a security monitoring system throughout its life cycle to ensure the confidentiality, integrity, availability, authenticity and non-repudiation of the data, and to comply with relevant laws and regulations and privacy protection requirements. Security monitoring data security management is a comprehensive system engineering aiming to balance the needs of security monitoring and personal privacy protection, data security requirements, and prevent sensitive information leakage, misuse or destruction.

[0003] Traditional security monitoring data security management methods usually focus on physical protection (such as computer room security) and basic network security (such as firewalls, access control), and data is often stored in raw format in local servers or the cloud, and is uniformly encrypted and backed up, making it difficult to effectively balance the needs of security monitoring and personal privacy protection. Centralized storage can lead to large-scale sensitive data leakage in the event of a security incident (such as hacking, internal personnel leaks); at the same time, uniform analysis of massive raw data not only has large computational load and low efficiency, thus reducing the speed and accuracy of identifying key threats. SUMMARY

[0004] The application provides a security monitoring data security management method and system based on the Internet of Things, which aims to improve the speed and accuracy of identifying key threats in security monitoring scenarios.

[0005] To achieve the above purpose, the application provides a security monitoring data security management method based on the Internet of Things, which comprises: Collecting multi-modal security data of a security monitoring scene, and dividing the multi-modal security data into blocks to obtain block multi-modal security data, and extracting dynamic object key frames and dimensionality reduction audio features of the block multi-modal security data; Dividing the block multi-modal security data into sensitive biometric data and regular monitoring metadata through the dynamic object key frames and dimensionality reduction audio features; Storing the sensitive biometric data into a private chain to obtain private chain sensitive data, storing the regular monitoring metadata into a public chain to obtain public chain regular data, and establishing a three-level threat detection network of the private chain sensitive data and the public chain regular data; Based on the public blockchain's regular data, the edge layer in the three-level threat detection network is used to identify abnormal behavior patterns in the security monitoring scenario, and the fog computing layer in the three-level threat detection network is used to analyze the global anomaly model parameters of the security monitoring scenario. Based on the global anomaly model parameters and the private chain sensitive data, the global reinforcement layer in the three-level threat detection network is used to analyze the abnormal state of the security monitoring scenario, and a hierarchical response strategy for the security monitoring scenario is generated according to the abnormal state.

[0006] Optionally, the extraction of dynamic object keyframes and dimensionality-reduced audio features from the segmented multimodal security data includes: The video data segments corresponding to the segmented multimodal security data are decoded to obtain decoded image frames; A background model of the decoded image frame is established to identify the foreground mask of the decoded image frame; Calculate the area of ​​the white pixels in the foreground mask; Based on the area of ​​the white pixels and a preset area threshold, candidate frames for the decoded image frame are determined; Determine the dynamic object keyframes of the candidate frames; The audio data segment in the segmented multimodal security data is decoded into PCM waveform data; Extract the MFCC feature vector from the PCM waveform data; The dimensionality of the MFCC feature vector is reduced to obtain the dimensionality-reduced audio features.

[0007] Optionally, determining the dynamic object keyframes of the candidate frames includes: Define the sliding window of the candidate frames; Calculate the window foreground SSIM and the area of ​​the white pixels in the sliding window for the candidate frame; Based on the window foreground SSIM and the area of ​​the white pixels in the window, the weighted foreground SSIM of the candidate frame is calculated using the following formula: ; in, Indicates the weighted foreground SSIM of the candidate frame. Represents the basis weights. This indicates the number of sliding windows corresponding to the candidate frames. Indicates candidate frames In the The area of ​​the white pixels in a sliding window. Indicates the first The area of ​​a sliding window. Indicates candidate frames and candidate frames In the The window foreground of a sliding window is SSIM. Indicates candidate frames In the Image blocks in a sliding window, Indicates candidate frames In the Image blocks in a sliding window; Based on the weighted foreground SSIM, the dynamic object keyframes of the candidate frames are determined.

[0008] Optionally, the step of dividing the multimodal security data into blocks to obtain block multimodal security data includes: The multimodal security data is timestamped to obtain aligned multimodal security data; The aligned multimodal security data is stored in a buffer queue to perform pairing of the aligned multimodal security data, thereby obtaining paired multimodal security data; Define the segmentation boundaries of the paired multimodal security data, wherein the segmentation boundaries include fixed-duration segmentation boundaries and event-triggered segmentation boundaries; Based on the block boundaries, video data segments and audio data segments of the paired multimodal security data are generated; The video data segments and audio data segments are combined to obtain the segmented multimodal security data.

[0009] Optionally, the segmented multimodal security data is divided into sensitive biometric data and routine monitoring metadata using the dynamic object keyframes and reduced-dimensional audio features, including: Identify the target bounding box of the keyframe of the dynamic object; Based on the target bounding box, analyze the face integrity coefficient of the keyframe of the dynamic object; Based on the preset sensitive classification rules and the face integrity coefficient, the explicit sensitive data and potential sensitive data of the keyframe of the dynamic object are determined. Identify the sound type labels of the reduced-dimensional audio features; Based on the sound type label, identify audiovisual related sensitive data in the potential sensitive data and audio-only sensitive data in the segmented multimodal security data; Based on the explicitly sensitive data, the audiovisual associated sensitive data, and the audio-only sensitive data, the segmented multimodal security data is divided into sensitive biometric data and regular monitoring metadata.

[0010] Optionally, establishing a three-tiered threat detection network for the private blockchain sensitive data and the public blockchain regular data includes: A three-tiered threat detection network is defined for the sensitive data of the private chain and the regular data of the public chain, wherein the three-tiered threat detection network includes an edge layer, a fog computing layer, and a global reinforcement layer; Configure the software environment, hierarchical algorithms, and communication links of the three-level threat detection network. Calculate the link effectiveness coefficient of the communication link; When the link validity coefficient meets the preset link validity threshold, a three-level threat detection network for the private chain sensitive data and the public chain regular data is established using the three-level threat detection network hierarchy, software environment, hierarchical algorithm and communication link.

[0011] Optionally, calculating the link effectiveness coefficient of the communication link includes: Obtain the link test data of the communication link; Based on the link test data, the link bandwidth, average transmission delay, number of transmissions, and encryption strength score of the communication link are analyzed. The average transmission delay is normalized to obtain the normalized average transmission delay. Combining the link bandwidth, normalized average transmission delay, number of transmissions, and encryption strength score, the link effectiveness coefficient of the communication link is calculated using the following formula: ; in, The link effectiveness coefficient represents the link efficiency coefficient of the communication link. This indicates the link bandwidth of the communication link. This indicates the maximum bandwidth of the communication link. This represents the link bandwidth weight of the communication link. This represents the normalized average transmission delay of the communication link. Indicates the transmission delay weight of the communication link. Indicates the number of transmissions in the communication link. This represents the number of successful transmissions in the corresponding transmission count of the communication link. The weight representing the transmission success rate of the communication link. This indicates the encryption strength score of the communication link. This indicates the encryption strength weight of the communication link.

[0012] Optionally, the step of identifying abnormal behavior patterns in the security monitoring scenario based on the public blockchain's regular data and utilizing the edge layer in the three-level threat detection network includes: The keyframes of dynamic objects and the reduced-dimensional audio features corresponding to the public blockchain's regular data are reconstructed to obtain the reconstructed features of the security monitoring scene. The reconstruction error of the reconstructed feature is calculated using the edge layer; Based on the reconstruction error, candidate anomalies for the security monitoring scenario are defined; Analyze the dynamic behavior of the keyframes of the dynamic object; By combining the candidate anomalies and the dynamic behaviors, the abnormal behavior patterns of the security monitoring scenario are determined.

[0013] Optionally, the step of analyzing the global anomaly model parameters of the security monitoring scenario using the fog computing layer in the three-level threat detection network includes: Identify the abnormal behavior targets corresponding to the abnormal behavior patterns in the security monitoring scenario; The scattered trajectory fragments of the abnormal behavior target are marked to generate the complete and continuous motion trajectory of the security monitoring scene; Analyze the audio-related anomaly events associated with the complete and continuous motion trajectory; The fog computing layer is used to calculate the abnormal event heatmap, behavior pattern parameters, and correlation parameters of the audio-related abnormal events; Based on the abnormal event heatmap, behavior pattern parameters, and correlation parameters, the global anomaly model parameters for the security monitoring scenario are determined.

[0014] To address the aforementioned problems, the present invention also provides an Internet of Things-based security monitoring data security management system, the system comprising: The data feature extraction module is used to collect multimodal security data from security monitoring scenarios, divide the multimodal security data into blocks to obtain block multimodal security data, and extract dynamic object keyframes and dimensionality-reduced audio features from the block multimodal security data. The sensitive data segmentation module is used to segment the block multimodal security data into sensitive biometric data and regular monitoring metadata based on the keyframes of the dynamic objects and the reduced-dimensional audio features. The detection network construction module is used to store the sensitive biometric data in a private chain to obtain private chain sensitive data, store the regular monitoring metadata in a public chain to obtain public chain regular data, and establish a three-level threat detection network of the private chain sensitive data and the public chain regular data. The scene anomaly analysis module is used to identify abnormal behavior patterns of the security monitoring scene based on the public chain's regular data and using the edge layer in the three-level threat detection network, so as to analyze the global anomaly model parameters of the security monitoring scene using the fog computing layer in the three-level threat detection network. The response strategy construction module is used to analyze the abnormal state of the security monitoring scenario based on the global anomaly model parameters and the private chain sensitive data, using the global reinforcement layer in the three-level threat detection network, and generate a hierarchical response strategy for the security monitoring scenario according to the abnormal state.

[0015] First, by segmenting multimodal security data and extracting keyframes of dynamic objects and dimensionality-reduced audio features, efficient compression and accurate representation of monitored content are achieved, significantly reducing data redundancy and computational overhead in subsequent processing. Second, by combining keyframes and audio features to intelligently classify data into sensitive biometric features and routine monitoring metadata, this effectively protects personal privacy and biometric security while improving the compliance and precision of data processing. Furthermore, an architecture is adopted that uses a private blockchain to store sensitive data and a public blockchain to store routine data. This not only ensures the immutability and high security of sensitive information but also leverages the transparency of the public blockchain. The system's high performance and efficiency support the sharing and rapid retrieval of routine data. Finally, a three-tiered threat detection network—edge-fog-global—is constructed, enabling multi-level, progressive intelligent analysis from real-time abnormal behavior recognition and global model parameter analysis to comprehensive abnormal state determination. This architecture fully leverages the real-time performance of edge computing, the synergistic nature of fog computing, and the decision-making capabilities of the global layer. It can accurately identify potential threats in complex scenarios and generate differentiated, tiered response strategies accordingly, significantly improving the intelligence level, response speed, and handling accuracy of security systems. This provides strong technical support for building a new generation of efficient, secure, and compliant intelligent security monitoring systems. Therefore, this invention can improve the speed and accuracy of identifying key threats in security monitoring scenarios. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a security management method for security monitoring data based on the Internet of Things, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a module for implementing the Internet of Things-based security monitoring data security management method according to an embodiment of the present invention.

[0017] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] This application provides a method for managing security monitoring data based on the Internet of Things (IoT). The executing entity of this IoT-based security monitoring data management method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the IoT-based security monitoring data management method can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0020] Reference Figure 1 The diagram shown is a flowchart illustrating a security monitoring data security management method based on the Internet of Things (IoT) according to an embodiment of the present invention. In this embodiment, the security monitoring data security management method based on the IoT includes: S1. Collect multimodal security data from security monitoring scenarios, divide the multimodal security data into blocks to obtain block multimodal security data, and extract dynamic object keyframes and dimensionality-reduced audio features from the block multimodal security data.

[0021] It should be explained that the security monitoring scenario refers to a scenario that requires security monitoring, such as fixed monitoring scenarios (bank vaults, airport security checkpoints, smart buildings), mobile monitoring scenarios (patrol robots, vehicle-mounted monitoring, drone inspections), and special environment scenarios (underground utility tunnels, chemical plants, nuclear power plants). The multimodal security data includes video data, infrared thermal imaging, 3D point clouds, voiceprint data, and environmental sensors.

[0022] This invention divides the multimodal security data into blocks to obtain block multimodal security data. The acquisition and block processing of multimodal data enable the system to capture various information in security scenarios more comprehensively and meticulously, laying a solid foundation for subsequent intelligent analysis.

[0023] Specifically, the step of dividing the multimodal security data into blocks to obtain block multimodal security data includes: The multimodal security data is timestamped to obtain aligned multimodal security data; The aligned multimodal security data is stored in a buffer queue to perform pairing of the aligned multimodal security data, thereby obtaining paired multimodal security data; Define the segmentation boundaries of the paired multimodal security data, wherein the segmentation boundaries include fixed-duration segmentation boundaries and event-triggered segmentation boundaries; Based on the block boundaries, video data segments and audio data segments of the paired multimodal security data are generated; The video data segments and audio data segments are combined to obtain the segmented multimodal security data.

[0024] The aligned multimodal security data refers to the set of original video and audio data that has been synchronized using timestamps. The buffer queue is a first-in, first-out (FIFO) data structure in memory used to temporarily store the aligned multimodal security data, typically containing a video frame queue and an audio packet queue. The paired multimodal security data refers to the basic units of video frames and audio packets retrieved from the buffer queue and matched one-to-one according to their timestamps. The fixed-duration block boundary is a signal periodically generated by a timer within the system to segment the data stream; the time interval is preset and fixed. The event-triggered block boundary... A boundary refers to a signal generated by the real-time analysis module when it detects a preset anomaly or specific pattern in the data stream, used to segment the data stream. A video data segment refers to an independent video clip encoded from a set of paired video frames arranged in chronological order between two consecutive "segment boundaries". An audio data segment refers to an independent audio clip encoded from all audio data packets between the same two "segment boundaries" that are strictly time-aligned with the "video data segment". Segmented multimodal security data refers to a complete and independent data packet containing "video data segments", "audio data segments", and related metadata.

[0025] Optionally, the event-triggered block boundary can be continuously monitored by a lightweight real-time algorithm (such as inter-frame difference or audio energy detection). For example, once a preset event such as "violent movement" or "sudden increase in sound decibels" is detected, a boundary signal is immediately generated.

[0026] This invention extracts dynamic object keyframes and dimensionality-reduced audio features from the segmented multimodal security data to determine whether the data block contains abnormal behavior, thereby achieving efficient and accurate analysis.

[0027] Specifically, the extraction of dynamic object keyframes and dimensionality-reduced audio features from the segmented multimodal security data includes: The video data segments corresponding to the segmented multimodal security data are decoded to obtain decoded image frames; A background model of the decoded image frame is established to identify the foreground mask of the decoded image frame; Calculate the area of ​​the white pixels in the foreground mask; Based on the area of ​​the white pixels and a preset area threshold, candidate frames for the decoded image frame are determined; Determine the dynamic object keyframes of the candidate frames; The audio data segment in the segmented multimodal security data is decoded into PCM waveform data; Extract the MFCC feature vector from the PCM waveform data; The dimensionality of the MFCC feature vector is reduced to obtain the dimensionality-reduced audio features.

[0028] The decoded image frame refers to the uncompressed original bitmap image obtained by restoring compressed video data segments (such as H.264 format) using a corresponding decoding algorithm. The background model is a mathematical model obtained through statistical learning to characterize the static background portion of a surveillance scene. The foreground mask is a binary image used to identify whether a corresponding position in the original image frame belongs to the background or foreground. The white pixel area refers to the total number of pixels marked as foreground (white) in the binary foreground mask. The area threshold is a critical pixel area value used to determine whether motion is significant. The candidate frame refers to the original decoded image frame... Image frames whose corresponding foreground mask white pixel area exceeds a preset area threshold; dynamic object keyframes refer to the set of key image frames that best represent the core motion events in the entire video segment and are the fewest in number, obtained from the candidate frame set after redundancy removal and screening; PCM waveform data refers to the digital format that uses a series of discrete sampling point values ​​to losslessly represent continuous analog audio signals; MFCC feature vectors refer to features used to characterize the shape of the spectral envelope of short-time audio frames; and dimensionality-reduced audio features refer to a global, low-dimensional numerical vector obtained by statistically aggregating all MFCC feature vectors of a complete audio data segment.

[0029] Optionally, the establishment of the background model of the decoded image frame can be implemented using the KNN or MOG2 algorithm.

[0030] Optionally, the calculation of the area of ​​white pixels in the foreground mask can be performed by connected component analysis or direct pixel counting based on the foreground mask (binary image). Specifically, the entire image is traversed, and the total number of pixels with a value of 255 is counted. This sum represents the pixel area occupied by the moving object in the current frame.

[0031] Optionally, the dimensionality reduction of the MFCC feature vector to obtain the dimensionality-reduced audio features can be achieved by calculating the mean and standard deviation of all MFCC vectors in each dimension and then concatenating them.

[0032] Further, determining the dynamic object keyframes of the candidate frames includes: Define the sliding window of the candidate frames; Calculate the window foreground SSIM and the area of ​​the white pixels in the sliding window for the candidate frame; The weighted foreground SSIM of the candidate frame is calculated based on the window foreground SSIM and the area of ​​the white pixels in the window. Based on the weighted foreground SSIM, the dynamic object keyframes of the candidate frames are determined.

[0033] Furthermore, as another embodiment of the present invention, the weighted foreground SSIM is calculated using the following formula: ; in, Indicates the weighted foreground SSIM of the candidate frame. Represents the basis weights. This indicates the number of sliding windows corresponding to the candidate frames. Indicates candidate frames In the The area of ​​the white pixels in a sliding window. Indicates the first The area of ​​a sliding window. Indicates candidate frames and candidate frames In the The window foreground of a sliding window is SSIM. Indicates candidate frames In the Image blocks in a sliding window, Indicates candidate frames In the An image block in a sliding window.

[0034] The sliding window refers to a fixed-size rectangular frame that "slides" across the entire image from left to right and from top to bottom according to a preset step size. The window foreground SSIM refers to the... Within a local area defined by a sliding window, the structural similarity index between the current candidate frame and the previous candidate frame is calculated. The area of ​​the white pixels in the window refers to the area of ​​the first white pixel in the sliding window. The total number of white pixels in the foreground mask of the current candidate frame within a local area defined by a sliding window. The Weighted Foreground SSIM (SSIM) is the final output of the entire process and is a single comprehensive index. The basis weight is a preset constant greater than 0 and less than 1 (e.g., 0.1). It plays a "safety" role in the calculation formula of Weighted Foreground SSIM, preventing the weight of a pure background window from being zero, thus avoiding the problem of a zero denominator. The image patch of the sliding window refers to the rectangular area of ​​original image pixel data covered by the window when the sliding window is stationary at a certain position in the image. S2. Using the keyframes of the dynamic objects and the reduced-dimensional audio features, the segmented multimodal security data is divided into sensitive biometric data and regular monitoring metadata.

[0035] This invention uses keyframes of dynamic objects and reduced-dimensional audio features to divide the segmented multimodal security data into sensitive biometric data and regular monitoring metadata. In the early stages of data processing, sensitive information is strictly separated from ordinary information, ensuring that only non-sensitive metadata is subsequently uploaded to the public blockchain, thereby reducing the risk of privacy leakage at the source.

[0036] Specifically, the segmented multimodal security data is divided into sensitive biometric data and routine monitoring metadata using the dynamic object keyframes and reduced-dimensional audio features, including: Identify the target bounding box of the keyframe of the dynamic object; Based on the target bounding box, analyze the face integrity coefficient of the keyframe of the dynamic object; Based on the preset sensitive classification rules and the face integrity coefficient, the explicit sensitive data and potential sensitive data of the keyframe of the dynamic object are determined. Identify the sound type labels of the reduced-dimensional audio features; Based on the sound type label, identify audiovisual associated sensitive data in the potential sensitive data and audio-only sensitive data in the segmented multimodal security data; Based on the explicitly sensitive data, the audiovisual associated sensitive data, and the audio-only sensitive data, the segmented multimodal security data is divided into sensitive biometric data and regular monitoring metadata.

[0037] The target bounding box refers to a rectangular box that can uniquely identify the location and range of a specific object in an image, usually represented by a set of pixel coordinates. The face integrity coefficient is a quantitative indicator used to evaluate whether the face detected in the keyframe of a dynamic object is complete, clear, and usable for biometric identification. The sensitivity classification rule refers to a decision logic based on the face integrity coefficient and audio tags, used to automatically divide data blocks into different sensitivity levels. For example, rule 1: if a clear face (front or side) is detected in any keyframe, it is considered explicit sensitive data; rule 2: if a complete human shape is detected but a clear face is not detected (e.g., back view, distance, occlusion), it is considered potentially sensitive. Explicitly sensitive data refers to data blocks that are directly detected and confirmed to contain high-quality, identifiable facial biometric features in the keyframe of a dynamic object. Potentially sensitive data refers to data blocks that detect a human target in the keyframe of a dynamic object but cannot confirm the presence of a clear face. The data blocks identified by their identity (no face or poor face quality) are categorized as follows: the sound type label refers to a text identifier describing the semantics of the audio content obtained after classifying the dimensionality-reduced audio feature vector, such as human speech, screams, vehicle noise, wind sounds, silence, gunshots, and glass breaking sounds; the audiovisual associated sensitive data refers to sensitive data determined through multimodal fusion, such as seeing a back view (visual - potential sensitivity) and simultaneously hearing someone speaking (audio - human voice). After associating the two, it can be highly certain that the data block contains human activity, thus classifying it as a sensitive category; the audio-only sensitive data refers to data blocks that detect extremely threatening sounds (such as gunshots or explosions) in audio analysis; the sensitive biometric data refers to data containing any form of information that can identify a specific individual or extremely threatening event; and the regular monitoring metadata refers to data that does not contain any biometric information that can identify an individual or extremely threatening event in the final classification result.

[0038] Optionally, the identification of the target bounding box of the keyframe of the dynamic object can be achieved by a pre-trained deep convolutional neural network (CNN) object detection model (such as YOLOv7, RT-DETR or SSD).

[0039] Optionally, the sound type label for identifying the dimensionality-reduced audio features can be implemented using a lightweight audio classifier (e.g., based on SVM, Random Forest, or a small neural network).

[0040] S3. Store the sensitive biometric data in a private blockchain to obtain private blockchain sensitive data, store the regular monitoring metadata in a public blockchain to obtain public blockchain regular data, and establish a three-level threat detection network of the private blockchain sensitive data and the public blockchain regular data.

[0041] This invention stores sensitive biometric data on a private blockchain to obtain private blockchain sensitive data, and stores the regular monitoring metadata on a public blockchain to obtain public blockchain regular data. This not only ensures the immutability and high security of sensitive information, but also leverages the transparency and efficiency of the public blockchain to support the sharing and rapid retrieval of regular data. Specifically, the private blockchain sensitive data refers to processed sensitive biometric data stored on a private blockchain, where the private blockchain is a permissioned, centralized blockchain network; and the public blockchain regular data refers to regular monitoring metadata stored on a public blockchain, where the public blockchain is a permissionless, decentralized blockchain network.

[0042] This invention establishes a three-level threat detection network consisting of sensitive data from the private blockchain and regular data from the public blockchain, forming a highly efficient and organic whole that integrates real-time response, regional collaboration, and global intelligence, greatly enhancing the perception, analysis, and response capabilities of security systems.

[0043] Specifically, the establishment of the three-tier threat detection network for the private blockchain sensitive data and the public blockchain regular data includes: A three-tiered threat detection network is defined for the sensitive data of the private chain and the regular data of the public chain, wherein the three-tiered threat detection network includes an edge layer, a fog computing layer, and a global reinforcement layer; Configure the software environment, hierarchical algorithms, and communication links of the three-level threat detection network. Calculate the link effectiveness coefficient of the communication link; When the link validity coefficient meets the preset link validity threshold, a three-level threat detection network for the private chain sensitive data and the public chain regular data is established using the three-level threat detection network hierarchy, software environment, hierarchical algorithm and communication link.

[0044] The term "edge layer" refers to a new type of computing infrastructure deployed at the forefront and end of physical world data acquisition sources, serving as the "nerve endings" of the 3D threat detection network. The "fog computing layer" refers to an intermediate computing layer located between the edge layer and the cloud computing center, deployed at core nodes of the local area network (such as server rooms and aggregation switches), serving as the "central nervous system" of the 3D network. The "global reinforcement layer" refers to a central processing layer located in a remote cloud data center, possessing the strongest computing and storage capabilities, serving as the "core brain" of the 3D threat detection network. The "software environment" refers to the set of operating systems, containers, runtime libraries, frameworks, and dependencies pre-deployed to ensure the stable operation of algorithms at all levels. The "hierarchical algorithm" refers to [the specific algorithm / mechanism]. The "communication link" refers to [the specific communication link]. The "link effectiveness coefficient" refers to [the specific link effectiveness threshold]. The "three-level threat detection network" refers to [the specific level of the network]. Optionally, the edge layer consists of devices such as smart cameras, embedded AI chips, and IoT sensor gateways, directly embedded within or adjacent to the security equipment. Its main responsibility is to perform real-time preprocessing and preliminary analysis of the raw video and audio streams at the earliest possible time and location after data generation, greatly reducing the amount of data that needs to be uploaded and ensuring the fastest initial response speed of the system.

[0045] Optionally, the software environment configuring the three-level threat detection network includes the following: the edge layer software environment: a lightweight OS (OpenWrt, Yocto), an edge computing framework (EdgeX Foundry), and an AI inference engine (TFLite, ONNX Runtime); the fog computing layer software environment: a container platform (Docker), an orchestration tool (Kubernetes), a message broker (MQTT Broker, Kafka), and a database (Redis, PostgreSQL); and the global reinforcement layer software environment: a big data platform (Hadoop, Spark), a deep learning framework (TensorFlow, PyTorch), blockchain nodes, and a web application framework.

[0046] Further, calculating the link effectiveness coefficient of the communication link includes: Obtain the link test data of the communication link; Based on the link test data, the link bandwidth, average transmission delay, number of transmissions, and encryption strength score of the communication link are analyzed. The average transmission delay is normalized to obtain the normalized average transmission delay. The link effectiveness coefficient of the communication link is calculated by combining the link bandwidth, normalized average transmission delay, number of transmissions, and encryption strength score.

[0047] Furthermore, as another embodiment of the present invention, the link effectiveness coefficient is calculated using the following formula: ; in, The link effectiveness coefficient represents the link efficiency coefficient of the communication link. This indicates the link bandwidth of the communication link. This indicates the maximum bandwidth of the communication link. This represents the link bandwidth weight of the communication link. This represents the normalized average transmission delay of the communication link. Indicates the transmission delay weight of the communication link. Indicates the number of transmissions in the communication link. This represents the number of successful transmissions in the corresponding transmission count of the communication link. The weight representing the transmission success rate of the communication link. This indicates the encryption strength score of the communication link. This indicates the encryption strength weight of the communication link.

[0048] Wherein, the link bandwidth refers to the actual available data transmission rate in the current communication link; the maximum link bandwidth refers to the maximum transmission rate that the communication link can achieve under ideal conditions; the average transmission delay refers to the average transmission time of a data packet from the sender to the receiver; the normalized average transmission delay refers to the average transmission delay converted to dimensionless units; the number of transmissions refers to the total number of data packets attempted to be sent within the statistical period; the encryption strength score refers to the security strength score of the encryption algorithm used in the communication link, which is usually a normalized value between 0 and 1, with a higher value indicating stronger security; the link bandwidth weight refers to the weight used to adjust the importance of bandwidth in the link effectiveness coefficient; the transmission delay weight refers to the weight used to adjust the importance of delay in the link effectiveness coefficient; the transmission success rate weight refers to the weight used to adjust the importance of transmission success rate in the link effectiveness coefficient; and the encryption strength weight refers to the weight used to adjust the importance of encryption strength in the link effectiveness coefficient.

[0049] S4. Based on the public blockchain's regular data, use the edge layer in the three-level threat detection network to identify abnormal behavior patterns in the security monitoring scenario, and use the fog computing layer in the three-level threat detection network to analyze the global anomaly model parameters of the security monitoring scenario.

[0050] This invention uses the public blockchain's conventional data and the edge layer in the three-level threat detection network to identify abnormal behavior patterns in the security monitoring scenario for later use.

[0051] In detail, the step of identifying abnormal behavior patterns in the security monitoring scenario based on the public blockchain's regular data and utilizing the edge layer of the three-level threat detection network includes: The keyframes of dynamic objects and the reduced-dimensional audio features corresponding to the public blockchain's regular data are reconstructed to obtain the reconstructed features of the security monitoring scene. The reconstruction error of the reconstructed feature is calculated using the edge layer; Based on the reconstruction error, candidate anomalies for the security monitoring scenario are defined; Analyze the dynamic behavior of the keyframes of the dynamic object; By combining the candidate anomalies and the dynamic behaviors, the abnormal behavior patterns of the security monitoring scenario are determined.

[0052] The reconstructed features refer to an approximate representation generated by encoding and decoding the original input (such as keyframes of dynamic objects and dimensionality-reduced audio features). The reconstruction error refers to the difference between the original input features and the reconstructed features. The candidate anomaly refers to an event or data segment that is initially judged as "potentially abnormal" because the reconstruction error exceeds a preset threshold. The dynamic behavior refers to the behavioral characteristics of target objects (such as people and vehicles) in the security monitoring scenario, such as movement patterns, trajectory changes, speed changes, and interaction methods in the time dimension. The abnormal behavior pattern refers to the behavior pattern with clear semantics or potential threat that is finally confirmed after combining candidate anomalies and dynamic behavior analysis.

[0053] Optionally, the reconstruction of the dynamic object keyframes and dimensionality-reduced audio features corresponding to the public blockchain conventional data to obtain the reconstructed features of the security monitoring scene can be achieved by reconstructing the keyframes and audio features using an autoencoder, variational autoencoder (VAE), or generative adversarial network (GAN).

[0054] Optionally, the reconstruction error of the reconstructed features calculated using the edge layer is typically represented by metrics such as Euclidean distance, mean squared error (MSE), or cross-entropy.

[0055] Optionally, the dynamic behavior of the keyframes of the dynamic object can be extracted using methods such as optical flow, trajectory tracking, LSTM, or 3D CNN.

[0056] This invention utilizes the fog computing layer in the three-level threat detection network to analyze the global anomaly model parameters of the security monitoring scenario, thereby elevating the perception of abnormal behavior from a local, superficial level to a regional, pattern-based understanding level.

[0057] In detail, the step of analyzing the global anomaly model parameters of the security monitoring scenario using the fog computing layer in the three-level threat detection network includes: Identify the abnormal behavior targets corresponding to the abnormal behavior patterns in the security monitoring scenario; The scattered trajectory fragments of the abnormal behavior target are marked to generate the complete and continuous motion trajectory of the security monitoring scene; Analyze the audio-related anomaly events associated with the complete and continuous motion trajectory; The fog computing layer is used to calculate the abnormal event heatmap, behavior pattern parameters, and correlation parameters of the audio-related abnormal events; Based on the abnormal event heatmap, behavior pattern parameters, and correlation parameters, the global anomaly model parameters for the security monitoring scenario are determined.

[0058] The abnormal behavior target refers to a specific object instance identified as engaging in abnormal activity by the initial edge-level detection algorithm or rules. It is the core object for subsequent tracking and in-depth analysis by the fog computing layer, such as a person climbing over a wall or a vehicle lingering in a no-parking zone for an extended period. The scattered trajectory fragments refer to a set of location-time data points generated by a single edge monitoring device tracking the same abnormal behavior target for a short period within its limited field of view. The complete continuous motion trajectory refers to the global, continuous, and temporal motion path of the target within the entire monitoring area, reconstructed by the fog computing layer after fusing multiple scattered trajectory fragments from multiple different cameras regarding the same abnormal behavior target using algorithms. The audio-related abnormal events... This refers to an audio anomaly event that, in terms of time and geographical location, highly overlaps with the complete and continuous movement trajectory of the abnormal behavior target. The anomaly event heatmap is a two-dimensional image generated using statistical methods such as kernel density estimation, based on geographic space, to visualize the spatial distribution density and clustering of anomaly events. The behavior pattern parameter is a set of numerical indicators extracted from the complete and continuous movement trajectory of the abnormal behavior target to quantify its movement characteristics and behavior patterns. The correlation parameter is a mathematical indicator used to quantify whether there is spatiotemporal synergy or causal relationship between multiple abnormal behavior targets. The global anomaly model parameter is a set of quantitative feature vectors produced by the fog computing layer that can comprehensively describe the anomaly situation in the entire current monitoring area.

[0059] Optionally, the process of marking the scattered trajectory fragments of the abnormal behavior target to generate a complete and continuous motion trajectory of the security monitoring scene can be achieved through cross-camera re-identification and trajectory smoothing algorithms. These algorithms connect the motion fragments of the target under each camera like "connecting ropes," forming a complete spatiotemporal motion record from the appearance to the disappearance of the target, thereby fully revealing its behavioral route and intention.

[0060] Optionally, the calculation of the anomalous event heatmap of the audio-related anomalous events can be performed using methods such as kernel density estimation (KDE) to calculate the event occurrence density at each geographic coordinate.

[0061] S5. Based on the global anomaly model parameters and the private chain sensitive data, the global reinforcement layer in the three-level threat detection network is used to analyze the abnormal state of the security monitoring scenario, and a hierarchical response strategy for the security monitoring scenario is generated according to the abnormal state.

[0062] This invention, based on the global anomaly model parameters and the private chain sensitive data, utilizes the global reinforcement layer in the three-level threat detection network to analyze the abnormal states of the security monitoring scenario, ultimately achieving accurate identification of abnormal states in the security monitoring scenario. Specifically, the abnormal state refers to a state that deviates from normal behavior patterns and possesses potential security risks, identified after a comprehensive analysis of the security monitoring scenario through the global reinforcement layer, combining the global anomaly model parameters and the private chain sensitive data.

[0063] Finally, based on the abnormal states, this invention generates a tiered response strategy for the security monitoring scenario, achieving "on-demand response and precise handling." This strategy not only improves the system's response efficiency and accuracy but also optimizes resource allocation. The tiered response strategy, based on the abnormal states identified in the security monitoring scenario and considering factors such as risk level, impact scope, and urgency, formulates and executes a set of differentiated, hierarchical security response measures. For example, Level 1 (low risk): log recording, minor alarm notification; Level 2 (medium risk): enhanced monitoring, push alarms to management personnel, partial area locking; Level 3 (high risk): triggering audible and visual alarms, initiating area lockdown, linking access control systems; Level 4 (emergency): global alarm linkage, notifying security personnel, activating emergency plans, automatic video recording for evidence collection.

[0064] First, by segmenting multimodal security data and extracting keyframes of dynamic objects and dimensionality-reduced audio features, efficient compression and accurate representation of monitored content are achieved, significantly reducing data redundancy and computational overhead in subsequent processing. Second, by combining keyframes and audio features to intelligently classify data into sensitive biometric features and routine monitoring metadata, this effectively protects personal privacy and biometric security while improving the compliance and precision of data processing. Furthermore, an architecture is adopted that uses a private blockchain to store sensitive data and a public blockchain to store routine data. This not only ensures the immutability and high security of sensitive information but also leverages the transparency of the public blockchain. The system's high performance and efficiency support the sharing and rapid retrieval of routine data. Finally, a three-tiered threat detection network—edge-fog-global—is constructed, enabling multi-level, progressive intelligent analysis from real-time abnormal behavior recognition and global model parameter analysis to comprehensive abnormal state determination. This architecture fully leverages the real-time performance of edge computing, the synergistic nature of fog computing, and the decision-making capabilities of the global layer. It can accurately identify potential threats in complex scenarios and generate differentiated, tiered response strategies accordingly, significantly improving the intelligence level, response speed, and handling accuracy of security systems. This provides strong technical support for building a new generation of efficient, secure, and compliant intelligent security monitoring systems. Therefore, this invention can improve the speed and accuracy of identifying key threats in security monitoring scenarios.

[0065] like Figure 2The diagram shown is a functional module diagram of a security monitoring data security management system based on the Internet of Things according to the present invention.

[0066] The IoT-based security monitoring data security management system 200 described in this invention can be installed in an electronic device. Depending on the functions implemented, the IoT-based security monitoring data security management system may include a data feature extraction module 201, a sensitive data segmentation module 202, a detection network construction module 203, a scene anomaly analysis module 204, and a response strategy construction module 205. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, stored in the memory of the electronic device.

[0067] In this embodiment of the invention, the functions of each module / unit are as follows: The data feature extraction module 201 is used to collect multimodal security data from security monitoring scenarios, divide the multimodal security data into blocks to obtain block multimodal security data, and extract dynamic object keyframes and dimensionality-reduced audio features from the block multimodal security data. The sensitive data segmentation module 202 is used to segment the block multimodal security data into sensitive biometric data and regular monitoring metadata using the dynamic object keyframes and dimensionality-reduced audio features. The detection network construction module 203 is used to store the sensitive biometric data in a private chain to obtain private chain sensitive data, store the regular monitoring metadata in a public chain to obtain public chain regular data, and establish a three-level threat detection network of the private chain sensitive data and the public chain regular data. The scene anomaly analysis module 204 is used to identify abnormal behavior patterns of the security monitoring scene based on the public chain regular data and using the edge layer in the three-level threat detection network, so as to analyze the global anomaly model parameters of the security monitoring scene using the fog computing layer in the three-level threat detection network. The response strategy construction module 205 is used to analyze the abnormal state of the security monitoring scenario based on the global anomaly model parameters and the private chain sensitive data, using the global reinforcement layer in the three-level threat detection network, and generate a hierarchical response strategy for the security monitoring scenario according to the abnormal state.

[0068] In detail, the modules in the IoT-based security monitoring data security management system 200 described in this embodiment of the invention employ the same methods as described above. Figure 1 The method used is the same as the IoT-based security monitoring data security management method described above, and it can produce the same technical effect, so it will not be elaborated here.

[0069] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0070] Finally, it should be noted that in the above embodiments, each embodiment can be combined with each other or independent. Deleting any one of them will not affect the technical implementation of other embodiments. The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for security management of security monitoring data based on the Internet of Things, characterized in that, The method includes: Collect multimodal security data from security monitoring scenarios, divide the multimodal security data into blocks to obtain block multimodal security data, and extract dynamic object keyframes and dimensionality-reduced audio features from the block multimodal security data; Based on the dynamic object keyframes and reduced-dimensional audio features, the segmented multimodal security data is divided into sensitive biometric data and regular monitoring metadata. The sensitive biometric data is stored in a private blockchain to obtain private blockchain sensitive data, and the regular monitoring metadata is stored in a public blockchain to obtain public blockchain regular data. A three-level threat detection network of the private blockchain sensitive data and the public blockchain regular data is established. Based on the public blockchain's regular data, the edge layer in the three-level threat detection network is used to identify abnormal behavior patterns in the security monitoring scenario, and the fog computing layer in the three-level threat detection network is used to analyze the global anomaly model parameters of the security monitoring scenario. Based on the global anomaly model parameters and the private chain sensitive data, the global reinforcement layer in the three-level threat detection network is used to analyze the abnormal state of the security monitoring scenario, and a hierarchical response strategy for the security monitoring scenario is generated according to the abnormal state.

2. The IoT-based security monitoring data security management method as described in claim 1, characterized in that, The extraction of dynamic object keyframes and dimensionality-reduced audio features from the segmented multimodal security data includes: The video data segments corresponding to the segmented multimodal security data are decoded to obtain decoded image frames; A background model of the decoded image frame is established to identify the foreground mask of the decoded image frame; Calculate the area of ​​the white pixels in the foreground mask; Based on the area of ​​the white pixels and a preset area threshold, candidate frames for the decoded image frame are determined; Determine the dynamic object keyframes of the candidate frames; The audio data segment in the segmented multimodal security data is decoded into PCM waveform data; Extract the MFCC feature vector from the PCM waveform data; The dimensionality of the MFCC feature vector is reduced to obtain the dimensionality-reduced audio features.

3. The IoT-based security monitoring data security management method as described in claim 2, characterized in that, The process of determining the dynamic object keyframes of the candidate frames includes: Define the sliding window of the candidate frames; Calculate the window foreground SSIM and the area of ​​the white pixels in the sliding window for the candidate frame; Based on the window foreground SSIM and the area of ​​the white pixels in the window, the weighted foreground SSIM of the candidate frame is calculated using the following formula: ； in, Indicates the weighted foreground SSIM of the candidate frame. Represents the basis weights. This indicates the number of sliding windows corresponding to the candidate frames. Indicates candidate frames In the The area of ​​the white pixels in a sliding window. Indicates the first The area of ​​a sliding window. Indicates candidate frames and candidate frames In the The window foreground of a sliding window is SSIM. Indicates candidate frames In the Image blocks in a sliding window, Indicates candidate frames In the Image blocks in a sliding window; Based on the weighted foreground SSIM, the dynamic object keyframes of the candidate frames are determined.

4. The IoT-based security monitoring data security management method as described in claim 1, characterized in that, The process of dividing the multimodal security data into blocks to obtain block-based multimodal security data includes: The multimodal security data is timestamped to obtain aligned multimodal security data; The aligned multimodal security data is stored in a buffer queue to perform pairing of the aligned multimodal security data, thereby obtaining paired multimodal security data; Define the segmentation boundaries of the paired multimodal security data, wherein the segmentation boundaries include fixed-duration segmentation boundaries and event-triggered segmentation boundaries; Based on the block boundaries, video data segments and audio data segments of the paired multimodal security data are generated; The video data segments and audio data segments are combined to obtain the segmented multimodal security data.

5. The IoT-based security monitoring data security management method as described in claim 1, characterized in that, The method involves dividing the segmented multimodal security data into sensitive biometric data and routine monitoring metadata using the keyframes of the dynamic objects and the reduced-dimensional audio features, including: Identify the target bounding box of the keyframe of the dynamic object; Based on the target bounding box, analyze the face integrity coefficient of the keyframe of the dynamic object; Based on the preset sensitive classification rules and the face integrity coefficient, the explicit sensitive data and potential sensitive data of the keyframe of the dynamic object are determined. Identify the sound type labels of the reduced-dimensional audio features; Based on the sound type label, identify audiovisual associated sensitive data in the potential sensitive data and audio-only sensitive data in the segmented multimodal security data; Based on the explicitly sensitive data, the audiovisual associated sensitive data, and the audio-only sensitive data, the segmented multimodal security data is divided into sensitive biometric data and regular monitoring metadata.

6. The IoT-based security monitoring data security management method as described in claim 1, characterized in that, The establishment of the three-level threat detection network for the sensitive data of the private blockchain and the regular data of the public blockchain includes: A three-tiered threat detection network is defined for the sensitive data of the private chain and the regular data of the public chain, wherein the three-tiered threat detection network includes an edge layer, a fog computing layer, and a global reinforcement layer; Configure the software environment, hierarchical algorithms, and communication links of the three-level threat detection network. Calculate the link effectiveness coefficient of the communication link; When the link validity coefficient meets the preset link validity threshold, a three-level threat detection network for the private chain sensitive data and the public chain regular data is established using the three-level threat detection network hierarchy, software environment, hierarchical algorithm and communication link.

7. The IoT-based security monitoring data security management method as described in claim 6, characterized in that, The calculation of the link effectiveness coefficient of the communication link includes: Obtain the link test data of the communication link; Based on the link test data, the link bandwidth, average transmission delay, number of transmissions, and encryption strength score of the communication link are analyzed. The average transmission delay is normalized to obtain the normalized average transmission delay. Combining the link bandwidth, normalized average transmission delay, number of transmissions, and encryption strength score, the link effectiveness coefficient of the communication link is calculated using the following formula: ； in, The link effectiveness coefficient represents the link efficiency coefficient of the communication link. This indicates the link bandwidth of the communication link. This indicates the maximum bandwidth of the communication link. This represents the link bandwidth weight of the communication link. This represents the normalized average transmission delay of the communication link. Indicates the transmission delay weight of the communication link. Indicates the number of transmissions in the communication link. This represents the number of successful transmissions in the corresponding transmission count of the communication link. The weight representing the transmission success rate of the communication link. This indicates the encryption strength score of the communication link. This indicates the encryption strength weight of the communication link.

8. The security management method for security monitoring data based on the Internet of Things as described in claim 1, characterized in that, The step of identifying abnormal behavior patterns in the security monitoring scenario based on the public blockchain's regular data and utilizing the edge layer of the three-level threat detection network includes: The keyframes of dynamic objects and the reduced-dimensional audio features corresponding to the public blockchain's regular data are reconstructed to obtain the reconstructed features of the security monitoring scene. The reconstruction error of the reconstructed feature is calculated using the edge layer; Based on the reconstruction error, candidate anomalies for the security monitoring scenario are defined; Analyze the dynamic behavior of the keyframes of the dynamic object; By combining the candidate anomalies and the dynamic behaviors, the abnormal behavior patterns of the security monitoring scenario are determined.

9. The IoT-based security monitoring data security management method as described in claim 1, characterized in that, The analysis of global anomaly model parameters for the security monitoring scenario using the fog computing layer in the three-level threat detection network includes: Identify the abnormal behavior targets corresponding to the abnormal behavior patterns in the security monitoring scenario; The scattered trajectory fragments of the abnormal behavior target are marked to generate the complete and continuous motion trajectory of the security monitoring scene; Analyze the audio-related anomaly events associated with the complete and continuous motion trajectory; The fog computing layer is used to calculate the abnormal event heatmap, behavior pattern parameters, and correlation parameters of the audio-related abnormal events; Based on the abnormal event heatmap, behavior pattern parameters, and correlation parameters, the global anomaly model parameters for the security monitoring scenario are determined.

10. A security monitoring data security management system based on the Internet of Things, characterized in that, The system includes: The data feature extraction module is used to collect multimodal security data from security monitoring scenarios, divide the multimodal security data into blocks to obtain block multimodal security data, and extract dynamic object keyframes and dimensionality-reduced audio features from the block multimodal security data. The sensitive data segmentation module is used to segment the block multimodal security data into sensitive biometric data and regular monitoring metadata based on the keyframes of the dynamic objects and the reduced-dimensional audio features. The detection network construction module is used to store the sensitive biometric data in a private chain to obtain private chain sensitive data, store the regular monitoring metadata in a public chain to obtain public chain regular data, and establish a three-level threat detection network of the private chain sensitive data and the public chain regular data. The scene anomaly analysis module is used to identify abnormal behavior patterns of the security monitoring scene based on the public chain's regular data and using the edge layer in the three-level threat detection network, so as to analyze the global anomaly model parameters of the security monitoring scene using the fog computing layer in the three-level threat detection network. The response strategy construction module is used to analyze the abnormal state of the security monitoring scenario based on the global anomaly model parameters and the private chain sensitive data, using the global reinforcement layer in the three-level threat detection network, and generate a hierarchical response strategy for the security monitoring scenario according to the abnormal state.

Images