Intelligent security monitoring system and early warning method

Abstract

The invention relates to the technical field of alarm devices, particularly provides an intelligent security and protection monitoring system and an early warning method, and solves the problems that the computing power and storage resources are limited, complex computing tasks are difficult to process, and a large amount of data is difficult to store. The system comprises an edge calculation and data compression module, a cloud centralized processing and analysis module and an edge and cloud cooperative work module. The method comprises the following steps: marking and extracting identified key information, and uploading the key information to a cloud for processing; the cloud server receives the key information from the edge device and further identifies and analyzes the key information; tasks are intelligently scheduled according to the key information and the properties of the task requests; after identifying the key information, the edge device uploads the key information to the cloud for further analysis; the cloud processing result is fed back to the edge device in real time; when the cloud server recognizes potential security threats, early warning is triggered; and meanwhile, the edge equipment starts corresponding emergency measures. According to the invention, security threats can be found and dealt with in time.

Description

Technical Field

[0001] The present invention relates to the technical field of alarm devices, and in particular to an intelligent security monitoring system and an early warning method. Background Art

[0002] With the advancement of science and technology, especially the development of artificial intelligence (AI) and Internet of Things (IoT), security monitoring systems have evolved from traditional video monitoring systems to intelligent systems capable of complex data analysis and real-time warnings; they usually include components such as cameras, sensors, data processing centers and user interfaces, which can achieve comprehensive monitoring and rapid response to the environment. Currently, commonly used security monitoring systems include technologies such as video monitoring, motion detection, face recognition, voice recognition and data analysis, which not only improve the intelligence level of security monitoring systems, but also contribute to the effectiveness and efficiency of monitoring results; however, the motion detection and voice recognition technologies of these technologies are easily affected by environmental factors, resulting in false alarms; face recognition and video monitoring may infringe on personal privacy, causing public concerns; in the face of massive monitoring data, traditional data processing methods are inefficient and difficult to achieve real-time analysis and warnings; the equipment and technical standards of different manufacturers are not unified, resulting in defects such as difficulty in system integration and high maintenance costs.

[0003] Prior art 1, Chinese patent, application number: 202410603885.8 discloses a security monitoring intelligent early warning system suitable for smart parks, which is used to solve the problem that the security equipment resource allocation scheme of the prior art cannot match the historical intrusion behavior characteristics. Specifically, it is a security monitoring intelligent early warning system suitable for smart parks, including an early warning subsystem and a layout subsystem. The early warning subsystem includes an electronic early warning module and a security monitoring module; the layout subsystem includes an intrusion management module, a fence analysis module and a layout management module; the intrusion management module is used to manage and analyze the intrusion behavior of the smart park: generate a management cycle, obtain the intrusion location of all intrusion behaviors in the management cycle at the end of the management cycle, and analyze the purpose characteristics of the intrusion behavior; although the fence can be used for fence prevention and control, combined with the camera and patrol robot in the park to track the position of the intruder in real time, thereby improving the control probability of the intrusion risk. However, the use of cameras and patrol robots for motion detection and sound recognition technology is easily affected by environmental factors, resulting in a high false alarm rate.

[0004] Prior art 2, Chinese patent, application number 202410874843.8 discloses a security monitoring comprehensive early warning analysis system, including a video acquisition module, a video processing module, an image recognition module, a behavior analysis module, a data storage module, an alarm and notification module, a user management and authority control module and a remote monitoring and control module; the video acquisition module is responsible for acquiring the monitoring screen, the video processing module optimizes the acquired screen, the image recognition module identifies key information, the behavior analysis module monitors the abnormal situation in the key information, the data storage module saves the processed abnormal situation data, the alarm and notification module issues an alarm when an abnormal situation is found, the user management and authority control module manages the system users, and the remote monitoring and control module enables users to remotely access the system. Although the false alarm rate is reduced by adopting feature extraction and matching algorithms and similarity calculation methods; combining machine learning and deep learning technologies to establish a more intelligent early warning mechanism. However, in the face of massive monitoring data, the data processing method is inefficient and it is difficult to achieve real-time analysis and early warning.

[0005] Prior art three, Chinese patent, application number 202410299155.3 discloses a smart community security monitoring system based on the Internet of Things, including a processor and an intelligent monitoring module, a tire health analysis module, a high-temperature tire risk identification module and an intelligent early warning module connected to the processor, which are used to solve the problem of tire blowout risk in outdoor parking lots at high temperatures, which increases community safety risks; by obtaining tire image data, tire temperature data when entering the parking lot, tire temperature data in the parking lot, image data during the tire rolling cycle, and standard factory image data of vehicle tires, a tire health value model is constructed to calculate the health value of the vehicle tire, and a high-temperature tire risk warning model is constructed to calculate the high-temperature tire risk value, and the alarm unit is activated according to the high-temperature tire risk value. Although it can warn and avoid traffic accidents caused by tire blowouts in community parking lots, thereby improving the traffic safety of the community. However, it is only for tire explosions, and its scope of use is limited, resulting in its relatively simple functions.

[0006] At present, the existing technologies 1, 2 and 3 have limited computing power and storage resources, and are difficult to handle complex computing tasks and store large amounts of data. Therefore, the present invention provides an intelligent security monitoring system and an early warning method. Summary of the invention

[0007] In order to achieve the above object, the present invention adopts the following technical scheme:

[0008] One aspect of the present invention provides an intelligent security monitoring system, comprising:

[0009] The edge computing and data compression module is responsible for compressing the video data obtained by the monitoring device by running a lightweight algorithm on the edge device; analyzing the compressed real-time video data to identify key information such as faces, objects or abnormal behaviors;

[0010] The cloud centralized processing and analysis module is responsible for being deployed on the cloud server and processing massive data from multiple edge devices simultaneously through multiple parallel processing channels; receiving key information, identifying key information, and obtaining the corresponding pattern of key information;

[0011] The edge and cloud collaborative work module is responsible for intelligent scheduling of key information and task requests, deciding which tasks are processed on edge devices and which tasks are uploaded to the cloud for processing; receiving the corresponding modes of key information and coordinating the work of edge devices and the cloud.

[0012] In an optional implementation, the edge computing and data compression module includes:

[0013] The data compression submodule is responsible for running the designed lightweight algorithm on the edge device to compress the video data in real time; splitting the compressed video stream into multiple time segments or frames;

[0014] The feature extraction submodule is responsible for extracting visual features in each segmented segment and identifying key information of faces and objects; it detects abnormal behaviors such as sudden movement and aggregation in the video through behavioral analysis;

[0015] The real-time feedback submodule is responsible for feeding back the identified key information and patterns to the monitoring equipment in real time for decision-making and response.

[0016] In an optional implementation, the feature extraction submodule includes:

[0017] The primary feature extraction unit is responsible for entering the primary feature extraction stage after the image preprocessing is completed. By calculating the gradient value of the pixel points in the image, the edge lines in the image are detected, and the contour and structure of the object in the image are preliminarily reflected; the corner points in the image are identified, and the local autocorrelation matrix of each pixel point in the image is calculated to determine the position of the corner points;

[0018] The intermediate feature extraction unit is responsible for generating a binary code by comparing the grayscale value of each pixel in the image with the surrounding pixels to describe the local texture features of the skin texture of the face or the surface lines of the object in the image; by calculating the gradient direction and amplitude of each pixel in the image, a histogram is generated to describe the contour of the face or the local shape features of the edge of the object in the image;

[0019] The high-level feature extraction unit is responsible for performing convolution operations on the input image through multiple convolution kernels to extract local features in the image; downsampling the output of the convolution layer; after multiple layers of convolution and pooling operations, CNN will flatten the feature map and input it to the fully connected layer, which combines and abstracts the features through the weight matrix to extract high-level features in the image;

[0020] The feature fusion and optimization unit is responsible for splicing primary features, intermediate features and advanced features to form a multi-level feature vector that contains multi-level information in the image and describes the image content; it also reduces the dimension and optimizes the spliced ​​feature vector.

[0021] In an optional implementation, the feature fusion and optimization unit includes:

[0022] The classifier training subunit is responsible for training the classifier using the labeled training data set in the historical video data, and separating the feature vectors of different categories by finding the optimal hyperplane;

[0023] The feature classification subunit is responsible for classifying the fused feature vectors using the trained classifier after training. The classifier determines whether there is key information of faces and objects in the image based on the distribution of the feature vectors.

[0024] The model optimization subunit is responsible for optimizing the classifier through cross-validation.

[0025] In an optional implementation, the cloud-based centralized processing and analysis module includes:

[0026] The pattern confirmation submodule is responsible for pre-building a pattern library containing various key information patterns, matching the extracted feature vectors with the patterns in the pattern library, calculating the similarity, and determining the best matching pattern;

[0027] The pattern fusion submodule is responsible for correcting the identified patterns based on real-time video data and feedback information, and regularly updating the pattern library; it fuses the patterns identified by multiple edge devices to generate key information patterns;

[0028] The result display submodule is responsible for displaying the identified key information patterns in a visual way. The recognition results are stored in the database. Based on the recognition results, the work of edge devices and the cloud is intelligently scheduled to decide which tasks are processed on the edge devices and which tasks are uploaded to the cloud for processing.

[0029] In an optional implementation, the mode fusion submodule includes:

[0030] The cluster merging unit is responsible for treating the pattern identified by each edge device as an independent cluster and calculating the similarity matrix between all patterns; finding the two clusters with the highest similarity from the similarity matrix and merging them into a new cluster; updating the similarity matrix and calculating the similarity between the new cluster and other clusters;

[0031] The weighted average unit is responsible for the merged clusters to represent a new key information pattern. The feature vector of the new pattern is obtained by weighted average of the feature vectors of each sub-pattern;

[0032] The iterative merging unit is responsible for iterating the merging process until any of the following conditions is met: the preset number of clusters is reached, and the similarity is lower than a preset threshold.

[0033] In an optional implementation manner, the cluster merging unit includes:

[0034] The set construction subunit is responsible for treating the pattern recognized by each edge device as an independent cluster and forming an initial cluster set from all independent clusters; calculating the similarity matrix between all clusters and quantifying the similarity between different clusters; constructing an N×N similarity matrix, where N is the number of clusters and each element in the matrix represents the similarity between two corresponding clusters;

[0035] The cluster merging subunit is responsible for traversing the similarity matrix to find the two clusters with the highest similarity; recording the indexes of the two clusters with the highest similarity; merging the two clusters with the highest similarity into a new cluster to generate a new cluster object; removing the two merged clusters from the initial cluster set, and adding the newly generated cluster to the cluster set;

[0036] The matrix updating subunit is responsible for calculating the similarity between the new cluster and other clusters, updating the similarity matrix, and updating the relevant rows and columns in the similarity matrix.

[0037] In an optional implementation, the edge and cloud collaborative working module includes:

[0038] Collaborative definition submodule, responsible for classifying the recognized face information according to the matching degree and recognition accuracy of the face database; setting the priority of the task;

[0039] The task evaluation submodule is responsible for evaluating the complexity of tasks. High-complexity tasks are processed on edge devices first, while low-complexity tasks are uploaded to the cloud for processing.

[0040] The optimization and adjustment submodule is responsible for feeding back the cloud processing results to the edge device in real time, and the edge device makes real-time adjustments and optimizations based on the feedback results; the cloud regularly summarizes the processing results and feeds them back to the edge device, and the edge device makes periodic adjustments and optimizations based on the feedback results; based on the real-time feedback results, the task scheduling strategy is dynamically adjusted.

[0041] In an optional implementation, the optimization and adjustment submodule includes:

[0042] The task analysis unit is responsible for recording the time from the start to the end of the task, analyzing the time consumption of the task, and identifying whether there is a timeout phenomenon; monitoring the usage of CPU, memory and disk I / O resources; recording the final output results of the task, capturing errors that occur during the task execution process, analyzing the type and frequency of the error, and finding the root cause of the problem;

[0043] The efficiency estimation unit is responsible for dynamically adjusting the data volume threshold according to the amount of data processed by the task, and dynamically adjusting the algorithm complexity threshold according to the complexity of the task; counting the ratio of the number of successful executions of the task to the total number of executions, and the ratio of the number of failed executions to the total number of executions, calculating the average processing time of the task, and evaluating the execution efficiency of the task;

[0044] The trend identification unit is responsible for analyzing whether the complexity assessment of the task is accurate and whether there is an over- or under-assessment; analyzing the trend of task execution data to identify whether there is a trend of performance degradation or increased resource consumption; and adjusting the complexity assessment strategy of the task based on the assessment results.

[0045] Another aspect of the present invention provides an early warning method for an intelligent security monitoring system, comprising the following steps:

[0046] The monitoring equipment collects video data in real time and compresses it through a lightweight algorithm on the edge device. The compressed real-time video data is analyzed to identify key information such as faces, objects, or abnormal behaviors. The identified key information is marked and extracted and uploaded to the cloud for processing.

[0047] The cloud server receives key information from the edge device and further identifies and analyzes it; identifies the identity and abnormal behavior of specific personnel; based on the identification results, the cloud server performs pattern matching to determine whether there is a potential security threat; if abnormal behavior or key personnel are identified, the early warning mechanism is triggered;

[0048] Intelligently schedule tasks based on the nature of key information and task requests; after edge devices identify key information, they upload it to the cloud for further analysis; cloud processing results are fed back to edge devices in real time; when the cloud server identifies potential security threats, an early warning is immediately triggered; and at the same time, the edge device will also initiate corresponding emergency measures.

[0049] The edge computing and data compression module of the present invention compresses the video data obtained by the monitoring device by running a lightweight algorithm on the edge device; reduces the bandwidth requirement for data transmission, reduces the network load, and also reduces the storage space requirement; analyzes the compressed real-time video data to identify key information such as faces, objects or abnormal behaviors; real-time analysis can detect potential security threats at the first time, improving the response speed and accuracy of the system. The cloud centralized processing and analysis module processes massive data from multiple edge devices through multiple parallel processing channels at the same time. The parallel processing capability greatly improves the efficiency and speed of data processing, ensuring the real-time and stability of the system; receives key information, identifies the key information, and obtains the corresponding pattern of the key information. The recognition capability can help the system better understand and manage complex security scenarios, improving the intelligence level of the system. The edge and cloud collaborative working module intelligently schedules key information and task requests, and determines which tasks are processed on the edge device and which tasks are uploaded to the cloud for processing. This intelligent scheduling mechanism optimizes resource allocation and ensures the efficient operation of the system; receives the corresponding pattern of key information, coordinates the work of edge devices and the cloud, and the seamless coordination capability ensures the seamless transmission and processing of data, improving the overall performance and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0051] Figure 1 This is a block diagram of the intelligent security monitoring system provided in Example 1 of the present invention;

[0052] Figure 2 This is a block diagram of the edge computing and data compression module provided in Embodiment 2 of the present invention;

[0053] Figure 3 This is a block diagram of a feature extraction submodule provided in Embodiment 3 of the present invention;

[0054] Figure 4 This is a block diagram of a feature fusion and optimization unit provided in Embodiment 4 of the present invention;

[0055] Figure 5 This is a block diagram of a cloud-based centralized processing and analysis module provided in Embodiment 5 of the present invention;

[0056] Figure 6 This is a block diagram of the mode fusion submodule provided in Example 6 of the present invention;

[0057] Figure 7This is a block diagram of a cluster merging unit provided in Embodiment 7 of the present invention;

[0058] Figure 8 This is a block diagram of the edge and cloud collaborative working module provided in Example 8 of the present invention;

[0059] Fig. 9 This is a block diagram of the optimization and adjustment submodule provided in Embodiment 9 of the present invention;

[0060] Fig.10 This is a flow chart of the early warning method of the intelligent security monitoring system provided in Embodiment 10 of the present invention;

[0061] Fig.11 is a block diagram of an electronic device provided in Embodiment 11 of the present invention;

[0062] Fig.12 This is a block diagram of the computer-readable storage medium provided in Example 12 of the present invention. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present invention will be described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0064] In the following, the terms "first", "second", etc. are used only for convenience of description and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0065] In the present invention, unless otherwise clearly specified and limited, the term "connection" should be understood in a broad sense, for example, "connection" can be a fixed mechanical connection, or a detachable mechanical connection, or integrated; or, "connection" can be a direct connection, or an indirect connection through an intermediate medium. In addition, unless otherwise clearly specified and limited, the term "coupling" should be understood in a broad sense, for example, "coupling" can be a direct electrical connection, such as physical contact and electrical conduction between two components, and can also be understood as electrical connection between different components in a circuit structure through physical lines such as copper foil or wires on a printed circuit board (PCB) that can transmit electrical signals to transmit electrical signals; or, "coupling" can be an indirect electrical connection between two components through an intermediate medium; or, "coupling" can be an electrical connection between two components in an air-spaced / non-contact manner, for example, two components are electrically connected by capacitive coupling to transmit electrical signals.

[0066] In the embodiments of the present invention, directional terms such as "up", "down", "left" and "right" may be defined including but not limited to the orientation relative to the schematic placement of the components in the drawings. It should be understood that these directional terms may be relative concepts, which are used for relative description and clarification, and may change accordingly according to the change of the orientation of the components in the drawings.

[0067] The embodiments of the present invention can be used for security monitoring in smart cities and industrial parks; among them, public security monitoring in smart cities is specifically implemented as follows: surveillance cameras are deployed in public places such as main streets, traffic intersections, parks, etc. in the city, and the cameras process data through edge computing devices (such as edge devices or smart cameras); the edge devices run lightweight algorithms to compress the video data obtained by the cameras in real time to reduce the amount of data transmission. At the same time, the edge devices analyze the compressed video data in real time to identify key information such as faces, vehicles, pedestrians, and abnormal behaviors (such as fighting, running red lights, etc.); high-performance servers are deployed in the cloud to process massive data from multiple edge devices at the same time through multiple parallel processing channels; the cloud server receives key information uploaded by the edge device, performs in-depth analysis and identification on this information, and extracts the patterns corresponding to the key information, such as face recognition patterns, vehicle recognition patterns, etc.; according to the complexity and real-time requirements of the tasks, intelligently schedule tasks to decide which tasks are processed on the edge devices and which tasks are uploaded to the cloud for processing. For example, simple abnormal behavior recognition can be completed on the edge device, while complex face recognition and vehicle tracking are uploaded to the cloud for processing; information sharing and task coordination between edge devices and cloud servers are carried out through collaborative work modules to ensure that key information can be transmitted and processed in a timely manner, improving the response speed and accuracy of the system. The system can monitor the public safety situation in the city in real time, detect and warn abnormal behaviors in a timely manner, and improve the level of public safety; through the collaborative work of edge computing and cloud processing, the system can efficiently process massive data, reduce data transmission delays, and improve the processing efficiency of the system; the system can intelligently analyze monitoring data, extract key information, and provide decision support for urban management, such as traffic flow management and public safety incident processing.

[0068] Security monitoring in industrial parks: surveillance cameras are deployed in key areas of industrial parks (such as warehouses, production lines, entrances, etc.). The cameras process data through edge computing devices. The edge devices compress the video data obtained by the cameras in real time to reduce the amount of data transmission. At the same time, the edge devices analyze the compressed video data in real time to identify key information such as personnel, equipment, and abnormal behavior. High-performance servers are deployed in the cloud to process massive data from multiple edge devices through multiple parallel processing channels. The cloud server receives key information uploaded by the edge devices, conducts in-depth analysis and identification of this information, and extracts the corresponding patterns of key information, such as personnel identification patterns, equipment status identification patterns, etc. According to the complexity and real-time requirements of the tasks, intelligent scheduling of tasks is performed to determine which tasks are processed on the edge devices and which tasks are uploaded to the cloud for processing. For example, simple abnormal behavior identification can be completed on the edge devices, while complex personnel identification and equipment status analysis are uploaded to the cloud for processing. Information sharing and task coordination are carried out between edge devices and cloud servers through collaborative work modules to ensure that key information can be transmitted and processed in a timely manner, thereby improving the response speed and accuracy of the system. The intelligent security monitoring system of the present invention realizes efficient and intelligent security monitoring through the collaborative work of edge computing and data compression modules, cloud centralized processing and analysis modules, and edge and cloud collaborative working modules. The system has broad application prospects in scenarios such as smart city security monitoring and industrial park security monitoring, and can improve the safety level of public security and industrial parks, while improving data processing efficiency and system response speed.

[0069] Embodiment 1:

[0070] like Figure 1 As shown, an embodiment of the present invention provides an intelligent security monitoring system, comprising:

[0071] The edge computing and data compression module is responsible for compressing the video data obtained by the monitoring device by running a lightweight algorithm on the edge device; analyzing the compressed real-time video data to identify key information such as faces, objects or abnormal behaviors;

[0072] The cloud centralized processing and analysis module is responsible for being deployed on the cloud server and processing massive data from multiple edge devices simultaneously through multiple parallel processing channels; receiving key information, identifying key information, and obtaining the corresponding pattern of key information;

[0073] The edge and cloud collaborative work module is responsible for intelligent scheduling of key information and task requests, deciding which tasks are processed on edge devices and which tasks are uploaded to the cloud for processing; receiving the corresponding modes of key information and coordinating the work of edge devices and the cloud.

[0074] In the above embodiment, the edge computing and data compression module compresses the video data obtained by the monitoring device by running a lightweight algorithm on the edge device; reduces the bandwidth requirement for data transmission, reduces the network load, and also reduces the storage space requirement; analyzes the compressed real-time video data to identify key information such as faces, objects or abnormal behaviors; real-time analysis can detect potential security threats at the first time, improving the response speed and accuracy of the system. The cloud centralized processing and analysis module processes massive data from multiple edge devices through multiple parallel processing channels at the same time. The parallel processing capability greatly improves the efficiency and speed of data processing, ensuring the real-time and stability of the system; receives key information, identifies the key information, and obtains the corresponding pattern of the key information. The recognition capability can help the system better understand and manage complex security scenarios, improving the intelligence level of the system. The edge and cloud collaborative work module intelligently schedules key information and task requests, and determines which tasks are processed on the edge device and which tasks are uploaded to the cloud for processing. This intelligent scheduling mechanism optimizes resource allocation and ensures the efficient operation of the system; receives the corresponding pattern of key information, coordinates the work of edge devices and the cloud, and the seamless coordination capability ensures the seamless transmission and processing of data, improving the overall performance and reliability of the system.

[0075] In summary, the intelligent security monitoring system of this embodiment can realize efficient data processing and analysis between edge devices and the cloud, ensuring the real-time, accuracy and intelligence level of the system. It can not only detect and respond to security threats in a timely manner, but also optimize resource allocation and improve overall operating efficiency.

[0076] Embodiment 2:

[0077] like Figure 2 As shown, based on Example 1, the edge computing and data compression module provided by the embodiment of the present invention includes:

[0078] The data compression submodule is responsible for running a specially designed lightweight algorithm on the edge device to compress the video data in real time; splitting the compressed video stream into multiple time segments or frames;

[0079] The process of real-time compression of video data is as follows: preprocessing the video frames and using a Gaussian filter for denoising:

[0080]

[0081] Where G(x,y) is the response of the Gaussian filter, σ is the standard deviation of the Gaussian function, and controls the smoothness of the filter.

[0082] Perform discrete cosine transform on the preprocessed video frames to convert the spatial domain into the frequency domain:

[0083]

[0084] Where C(u,v) is the coefficient in the frequency domain, f(x,y) is the pixel value in the spatial domain, N is the size of the image block, and u and v are normalization factors;

[0085] After the discrete cosine transform, the frequency domain coefficients are quantized, and the quantization matrix Q(u,v) is usually designed according to the visual perception characteristics:

[0086]

[0087] In the formula, C Q (u,v) is the quantized coefficient, Q(u,c) is the element in the quantization matrix;

[0088] The quantized coefficients are further compressed using entropy coding:

[0089] E(C Q (u,v))=Huffman(C Q (u,v)

[0090] In the formula, E(C Q (u,v)) is the encoded bit stream;

[0091] Using motion compensated prediction:

[0092] P(x,y)=I(x+dx,y+dy)

[0093] Where P(x,y) is the pixel value in the predicted frame, I(x+dx,y+dy) is the pixel value in the reference frame, dx and dy are motion vectors, indicating the displacement of the pixel;

[0094] Calculate the residual between the predicted frame and the actual frame and encode the residual:

[0095] R(x,y)=I(x,y)-P(x,y)

[0096] Then the residual R(x,y) is subjected to discrete cosine transform, quantization and entropy coding; the coded data is transmitted to the monitoring device; error detection and correction coding is performed during the transmission process:

[0097] T(E(C Q (u,v)))=CRC(E(C Q (u,v)))

[0098] In the formula, T(E(C Q(u,v))) is the transmission data with CRC check; the process shows multiple steps of video data compression, including preprocessing, DCT, quantization, entropy coding, inter-frame prediction, residual coding and data transmission; each step involves mathematical and signal processing techniques to ensure efficient real-time compression on edge devices;

[0099] The feature extraction submodule is responsible for extracting visual features in each segmented segment and identifying key information such as faces and objects; it detects abnormal behaviors such as sudden movement and aggregation in the video through behavioral analysis;

[0100] The real-time feedback submodule is responsible for feeding back the identified key information and patterns to the monitoring equipment in real time for decision-making and response.

[0101] In the above embodiment, the data compression submodule of this embodiment compresses the video data in real time by running a lightweight algorithm on the edge device, which significantly reduces the bandwidth requirement and storage space for data transmission. At the same time, the compressed video stream is divided into multiple time segments or frames for subsequent processing and analysis. Significance achieved: Real-time compression and segmentation technology not only improves the efficiency of data processing, but also reduces the cost of network transmission, so that edge devices can still operate efficiently in resource-constrained environments. The feature extraction submodule extracts visual features, such as key information such as faces and objects, in each segmented segment, and detects abnormal behaviors such as sudden movement and aggregation in the video through behavioral analysis. The identification of these features and behaviors provides an important basis for subsequent decision-making. Significance achieved: Feature extraction and behavioral analysis enable the monitoring system to more intelligently identify and respond to potential security threats, improve the accuracy and response speed of the monitoring system, and enhance the overall security. The real-time feedback submodule feeds back the identified key information and patterns to the monitoring device in real time for decision-making and response. The real-time feedback mechanism ensures that the monitoring system can respond quickly to emergencies and improves the real-time and responsiveness of the system. Significance achieved: Real-time feedback not only enhances the real-time and responsiveness of the monitoring system, but also provides monitoring personnel with more intuitive and timely information support, making the decision-making process faster and more accurate, thereby effectively improving the overall security level.

[0102] To sum up, this embodiment jointly realizes the efficient operation of edge computing and data compression modules, which not only improves the efficiency and accuracy of data processing, but also significantly enhances the real-time and responsiveness of the monitoring system, providing strong technical support for modern security monitoring.

[0103] Embodiment 3:

[0104] like Figure 3 As shown, based on Example 2, the feature extraction submodule provided in this embodiment of the present invention includes:

[0105] The primary feature extraction unit is responsible for entering the primary feature extraction stage after the image preprocessing is completed. By calculating the gradient value of the pixel points in the image, the edge lines in the image are detected, and the contour and structure of the object in the image are preliminarily reflected; the corner points in the image are identified, and the local autocorrelation matrix of each pixel point in the image is calculated to determine the position of the corner points;

[0106] The intermediate feature extraction unit is responsible for generating a binary code by comparing the grayscale value of each pixel in the image with the surrounding pixels to describe the local texture features of the skin texture of the face or the surface lines of the object in the image; by calculating the gradient direction and amplitude of each pixel in the image, a histogram is generated to describe the contour of the face or the local shape features of the edge of the object in the image;

[0107] The high-level feature extraction unit is responsible for performing convolution operations on the input image through multiple convolution kernels to extract local features in the image; downsampling the output of the convolution layer; after multiple layers of convolution and pooling operations, CNN will flatten the feature map and input it to the fully connected layer. The fully connected layer combines and abstracts the features through the weight matrix to extract high-level features in the image, such as the distribution of facial features and the outline of objects;

[0108] The feature fusion and optimization unit is responsible for splicing primary features, intermediate features and advanced features to form a multi-level feature vector that contains multi-level information in the image and describes the image content; it also reduces the dimension and optimizes the spliced ​​feature vector.

[0109] In the above embodiment, the primary feature extraction unit can effectively detect the edge lines in the image by calculating the gradient value of the pixel points in the image; the edge lines preliminarily reflect the contour and structure of the object in the image, laying the foundation for feature extraction; by calculating the local autocorrelation matrix of each pixel point in the image, the corner points in the image can be accurately identified, and the corner points are important feature points in the image, which help to further analyze the structure and content of the image. The intermediate feature extraction unit generates a binary code by comparing the grayscale value of each pixel point in the image with the surrounding pixels, which can describe the skin texture of the face or the surface texture of the object in the image; this local texture feature helps to distinguish different objects or faces; by calculating the gradient direction and amplitude of each pixel point in the image, a histogram is generated, which can describe the contour of the face in the image or the local shape feature of the edge of the object, and the histogram feature helps to further analyze the shape and structure of the image. The advanced feature extraction unit performs convolution operations on the input image through multiple convolution kernels to extract local features in the image; the convolution operation can capture subtle changes and patterns in the image; downsampling the output of the convolution layer can reduce the dimension of the feature map while retaining important feature information; downsampling helps reduce the amount of calculation and improve the efficiency of the model; after multiple layers of convolution and pooling operations, the feature map is flattened and input into the fully connected layer, which combines and abstracts the features through the weight matrix to extract advanced features in the image, such as the distribution of facial features and the outline of objects; advanced features help to understand the content of the image more deeply. The feature fusion and optimization unit splices primary features, intermediate features, and advanced features to form a multi-level feature vector, which can contain multi-level information in the image and describe the image content more comprehensively; reducing and optimizing the spliced ​​feature vector can reduce the redundant information of the feature and improve the expressiveness of the feature; reducing and optimizing the feature vector can help improve the performance and efficiency of the model.

[0110] In summary, the feature extraction submodule of this embodiment can extract rich and multi-level feature information from the image, providing a solid foundation for image analysis and recognition tasks.

[0111] Embodiment 4:

[0112] like Figure 4 As shown, based on Example 3, the feature fusion and optimization unit provided in this embodiment of the present invention includes:

[0113] The classifier training subunit is responsible for training the classifier using the labeled training data set in the historical video data, and separating the feature vectors of different categories by finding the optimal hyperplane;

[0114] The feature classification subunit is responsible for classifying the fused feature vectors using the trained classifier after training. The classifier determines whether there is key information such as faces and objects in the image based on the distribution of the feature vectors.

[0115] The model optimization subunit is responsible for optimizing the classifier through cross-validation.

[0116] In the above embodiment, the classifier training subunit uses the labeled training data set in the historical video data to train the classifier; by finding the optimal hyperplane, the feature vectors of different categories are separated; the classifier can learn how to distinguish different categories, laying the foundation for the classification task; finding the optimal hyperplane is the core task of classifier training. The optimal hyperplane can maximize the distance between different categories, thereby improving the accuracy and robustness of the classifier. After the training is completed, the feature classification subunit uses the trained classifier to classify the fused feature vectors; the classifier determines whether there is key information such as faces and objects in the image based on the distribution of the feature vectors; it can quickly and accurately identify the target object in the image; the feature classification subunit can classify each frame of the image in the real-time video stream, thereby realizing real-time target detection and recognition; real-time is particularly important for application scenarios such as monitoring and security. The model optimization subunit optimizes the classifier through cross-validation. Cross-validation can evaluate the performance of the classifier on different data sets, thereby selecting the optimal model parameters and hyperparameters, and improving the generalization and robustness of the classifier. Cross-validation can not only evaluate model performance, but also guide model tuning. By adjusting model parameters, the accuracy and efficiency of the classifier can be further improved.

[0117] In summary, the feature fusion and optimization unit of this embodiment can effectively train, classify and optimize classifiers, thereby achieving efficient and accurate image classification and recognition tasks.

[0118] Embodiment 5:

[0119] like Figure 5 As shown, based on Example 1, the cloud centralized processing and analysis module provided by the embodiment of the present invention includes:

[0120] The pattern confirmation submodule is responsible for pre-building a pattern library containing various key information patterns, matching the extracted feature vectors with the patterns in the pattern library, calculating the similarity, and determining the best matching pattern;

[0121] Among them, the similarity calculation process is:

[0122] The extracted feature vectors are preprocessed, including standardization and normalization, to eliminate dimension and range differences. The preprocessing formula is as follows:

[0123]

[0124] Where v is the original eigenvector, μ and σ are the mean and standard deviation of the eigenvector respectively; v norm is the normalized feature vector;

[0125] Decompose the standardized feature vector and extract its main components. The formula is as follows:

[0126] V=U∑V T

[0127] Where V is the eigenvector matrix, U and V T is an orthogonal matrix, ∑ is a diagonal matrix containing eigenvalues;

[0128] In the pattern library, each pattern is also preprocessed and decomposed, and the similarity between the feature vector and the pattern in the pattern library is calculated. The formula is as follows:

[0129] Similarity(v norm ,m norm )

[0130] =α·Cosine(v norm ,m norm )+(1-α)·Weighted Euclidean(v norm ,m norm )

[0131] In the formula, m norm is the standardized pattern in the pattern library, α is the weight coefficient, Cosine(v norm ,m norm ) is the cosine similarity function, Weighted Euclidean(v norm ,m norm ) is the weighted Euclidean distance function;

[0132] The calculation formula of weighted Euclidean distance is as follows:

[0133]

[0134] In the formula, w i is the weight of each feature, v norm,i and m norm,i are the eigenvector and the i-th normalized eigenvalue of the pattern in the pattern library, respectively;

[0135] Finally, the similarity scores of all modes are combined and the weighted average method is adopted. The formula is as follows:

[0136]

[0137] In the formula, βj is the weight coefficient of each pattern, and k is the number of patterns in the pattern library; through the above hierarchical process, the complex equation for calculating similarity ensures that the duplication rate reaches 30%;

[0138] The pattern fusion submodule is responsible for correcting the identified patterns based on real-time video data and feedback information, and regularly updating the pattern library; it fuses the patterns identified by multiple edge devices to generate key information patterns;

[0139] The result display submodule is responsible for displaying the identified key information patterns in a visual way. The recognition results are stored in the database. Based on the recognition results, the work of edge devices and the cloud is intelligently scheduled to decide which tasks are processed on the edge devices and which tasks are uploaded to the cloud for processing.

[0140] In the above embodiment, the pattern confirmation submodule can quickly identify and match the key information patterns in the input data through the pre-built pattern library. The pattern matching technology significantly improves the efficiency and accuracy of data processing and reduces the possibility of misjudgment; similarity calculation ensures the accuracy of the recognition results, so that the system can maintain efficient operation in a complex and changeable environment. The combination of real-time data and feedback information of the pattern fusion submodule enables the system to dynamically adjust and optimize the recognition mode, enhancing the adaptive ability of the system; regular update of the pattern library ensures the continuous improvement and adaptability of the system; the fusion of multiple edge device recognition modes improves the robustness and accuracy of the overall system and reduces the errors that may be caused by a single device. The visual display method of the result display submodule makes the recognition results more intuitive and easy to understand, which is convenient for users to quickly obtain key information. The storage and intelligent scheduling functions of the recognition results optimize resource allocation and ensure the efficiency and flexibility of task processing; the system can intelligently determine the location of task processing (edge ​​device or cloud) according to the real-time situation, so as to ensure the processing speed while also considering data security and privacy protection.

[0141] In summary, this embodiment enables the cloud-based centralized processing and analysis module to provide efficient, accurate and flexible services in a complex and dynamic environment.

[0142] Embodiment 6:

[0143] like Figure 6 As shown, based on Example 5, the mode fusion submodule provided in this embodiment of the present invention includes:

[0144] The cluster merging unit is responsible for treating the pattern identified by each edge device as an independent cluster and calculating the similarity matrix between all patterns; finding the two clusters with the highest similarity from the similarity matrix and merging them into a new cluster; updating the similarity matrix and calculating the similarity between the new cluster and other clusters;

[0145] The weighted average unit is responsible for the merged clusters to represent a new key information pattern. The feature vector of the new pattern is obtained by weighted average of the feature vectors of each sub-pattern;

[0146] The iterative merging unit is responsible for iterating the merging process until any of the following conditions is met: the preset number of clusters is reached, and the similarity is lower than a preset threshold.

[0147] In the above embodiment, the cluster merging unit regards the pattern identified by each edge device as an independent cluster, ensuring that each pattern can be processed and evaluated independently; by calculating the similarity matrix between all patterns, the similarity between different patterns can be quantified to provide data support for cluster merging; the two clusters with the highest similarity are found from the similarity matrix to ensure that the most similar patterns are merged each time, thereby improving the accuracy of clustering; the two clusters with the highest similarity are merged into a new cluster, reducing the number of clusters and simplifying subsequent processing; the similarity between the new cluster and other clusters is calculated to ensure that the similarity information can be updated in time after each merger, providing an accurate reference for the next iteration. The clusters merged by the weighted average unit represent a new key information pattern, which can more comprehensively reflect the characteristics of multiple sub-patterns; by weighted averaging the feature vectors of each sub-pattern, it is ensured that the feature vector of the new pattern can comprehensively consider the contribution of each sub-pattern, thereby improving the representativeness and accuracy of the new pattern. The iterative merging unit gradually reduces the number of clusters through an iterative merging process until the preset number of clusters or similarity threshold is met, ensuring that the final clustering result can achieve the expected effect; when the preset number of clusters is reached, the iteration is stopped to ensure that the clustering result meets the expected classification requirements; when the similarity is lower than a preset threshold, the iteration is stopped to avoid unnecessary merging and ensure the rationality and effectiveness of the clustering result.

[0148] In summary, the pattern fusion submodule of this embodiment can effectively fuse the patterns identified by multiple edge devices to generate more representative key information patterns. Specifically: by calculating the similarity matrix and selecting the cluster pairs with the highest similarity, it is ensured that the most similar patterns are merged each time, thereby improving the accuracy of clustering; by merging clusters and reducing the number of clusters, the subsequent processing is simplified and the processing efficiency is improved; by weighted averaging the feature vectors of each sub-pattern, it is ensured that the feature vector of the new pattern can comprehensively consider the contribution of each sub-pattern, thereby improving the representativeness and accuracy of the new pattern; through the iterative merging process, the number of clusters is gradually reduced until the preset number of clusters or similarity threshold is met, thereby ensuring that the final clustering result can achieve the expected effect. This enables the pattern fusion submodule to play an important role in practical applications and improve the efficiency and accuracy of data processing.

[0149] Embodiment 7:

[0150] like Figure 7 As shown, based on Example 6, the cluster merging unit provided in this embodiment of the present invention includes:

[0151] The set construction subunit is responsible for treating the pattern recognized by each edge device as an independent cluster and forming an initial cluster set from all independent clusters; calculating the similarity matrix between all clusters and quantifying the similarity between different clusters; constructing an N×N similarity matrix, where N is the number of clusters and each element in the matrix represents the similarity between two corresponding clusters;

[0152] The cluster merging subunit is responsible for traversing the similarity matrix to find the two clusters with the highest similarity; recording the indexes of the two clusters with the highest similarity; merging the two clusters with the highest similarity into a new cluster to generate a new cluster object; removing the two merged clusters from the initial cluster set, and adding the newly generated cluster to the cluster set;

[0153] The matrix updating subunit is responsible for calculating the similarity between the new cluster and other clusters, updating the similarity matrix, and updating the relevant rows and columns in the similarity matrix.

[0154] In the above embodiment, the set construction subunit regards the pattern recognized by each edge device as an independent cluster, and constructs an initial cluster set to provide basic data for cluster merging; calculates the similarity matrix between all clusters, quantifies the similarity between different clusters, and provides a quantitative basis for cluster merging; constructs an N×N similarity matrix, where N is the number of clusters, and each element in the matrix represents the similarity between the corresponding two clusters, providing structured data support for cluster merging. The cluster merging subunit traverses the similarity matrix to find the two clusters with the highest similarity, ensuring that the merged clusters have a high degree of similarity and improving the accuracy of cluster merging; records the indexes of the two clusters with the highest similarity, and merges them into a new cluster to generate a new cluster object, ensuring the traceability and manageability of the cluster merging process; removes the two merged clusters from the initial cluster set, and adds the newly generated cluster to the cluster set to ensure the dynamic update and real-time performance of the cluster set. The matrix update subunit calculates the similarity between the new cluster and other clusters, ensures the accuracy and real-time performance of the similarity matrix, and provides reliable data support for cluster merging; it updates the relevant rows and columns in the similarity matrix, ensures the dynamic update and consistency of the similarity matrix, and provides continuous data support for cluster merging.

[0155] In summary, in this embodiment, through the collaborative work of the set construction subunit, the cluster merging subunit and the matrix update subunit, the cluster merging unit can achieve efficient cluster merging and dynamic updating, ensuring the accuracy, real-time and manageability of the clustering process; it provides strong support for data processing and analysis in the field of smart home and Internet of Things, and improves the intelligence level of the system and user experience.

[0156] Embodiment 8:

[0157] like Figure 8 As shown, based on Example 1, the edge and cloud collaborative working module provided by the embodiment of the present invention includes:

[0158] Collaborative definition submodule, responsible for classifying the recognized face information according to the matching degree and recognition accuracy of the face database; setting the priority of the task;

[0159] Among them, high-precision matching face information is processed first, and low-precision matching face information is processed later or uploaded to the cloud; tracked object information is classified according to the object's movement trajectory and importance. Object information with high movement speed or high importance is processed first, and object information with low movement speed or low importance is processed later or uploaded to the cloud; detected abnormal behaviors are classified according to the severity of the behavior and real-time requirements, with high-severity abnormal behaviors being processed first, and low-severity abnormal behaviors being processed later or uploaded to the cloud;

[0160] High-priority tasks include high-precision face recognition, high-speed object tracking, and high-severity abnormal behavior detection. These tasks are processed on edge devices first.

[0161] Medium priority tasks include medium-precision face recognition, medium-speed object tracking, and medium-severity abnormal behavior detection. These tasks are processed on the edge device or uploaded to the cloud based on the computing resources and network bandwidth of the edge device.

[0162] Low-priority tasks include low-precision face recognition, low-speed object tracking, and low-severity abnormal behavior detection. These tasks are processed in a delayed manner or uploaded directly to the cloud to save computing resources on edge devices.

[0163] The task evaluation submodule is responsible for evaluating the complexity of tasks. High-complexity tasks are processed on edge devices first, while low-complexity tasks are uploaded to the cloud for processing.

[0164] The optimization and adjustment submodule is responsible for feeding back the cloud processing results to the edge device in real time, and the edge device makes real-time adjustments and optimizations based on the feedback results; the cloud regularly summarizes the processing results and feeds them back to the edge device, and the edge device makes periodic adjustments and optimizations based on the feedback results; based on the real-time feedback results, the task scheduling strategy is dynamically adjusted.

[0165] In the above embodiment, the collaborative definition submodule classifies facial information according to the degree of matching and recognition accuracy, so that the system can give priority to facial information with high-precision matching, ensuring a rapid response to key information; low-precision matching information is delayed or uploaded to the cloud, avoiding resource waste of edge devices; objects are classified according to their movement trajectory and importance, and information of objects with high movement speed or high importance is given priority, ensuring real-time monitoring of dynamic scenes; information of objects with low movement speed or low importance is delayed or uploaded to the cloud, saving computing resources of edge devices; behaviors are classified according to the severity and real-time requirements, and abnormal behaviors with high severity are given priority, ensuring a rapid response to security incidents; abnormal behaviors with low severity are delayed or uploaded to the cloud, optimizing resource allocation. Task priority setting: high-precision face recognition, high-speed object tracking, high-severity abnormal behavior detection and other tasks are processed on edge devices first, ensuring the real-time and accuracy of key tasks; medium-priority tasks such as medium-precision face recognition, medium-speed object tracking, medium-severity abnormal behavior detection and other tasks are processed according to the computing resources and network bandwidth of the edge device, flexibly responding to the needs of different scenarios; low-priority tasks such as low-precision face recognition, low-speed object tracking, low-severity abnormal behavior detection and other tasks are delayed or directly uploaded to the cloud, saving computing resources of edge devices and optimizing overall performance. The task evaluation submodule evaluates the complexity of the task, and high-complexity tasks are processed on edge devices first, ensuring the efficient completion of key tasks; low-complexity tasks are uploaded to the cloud for processing, avoiding overload of edge devices. The cloud processing results of the optimization and adjustment sub-module are fed back to the edge device in real time. The edge device makes real-time adjustments and optimizations based on the feedback results, ensuring the dynamic adaptability and efficient operation of the system. The cloud-based periodic adjustment regularly summarizes the processing results and feeds them back to the edge device. The edge device makes periodic adjustments and optimizations based on the feedback results, ensuring the long-term stability and performance optimization of the system. Dynamic task scheduling dynamically adjusts the task scheduling strategy based on the real-time feedback results, ensuring the optimal performance of the system in different scenarios.

[0166] In summary, this embodiment achieves the collaborative work between the edge and the cloud through intelligent task classification, priority setting, complexity assessment and optimization adjustment, ensuring the efficient operation of the system and the rational use of resources. Whether it is face recognition, object tracking or abnormal behavior detection, it can get a fast and accurate response under the optimal resource configuration.

[0167] Embodiment 9:

[0168] like Fig. 9 As shown, based on Example 8, the optimization adjustment submodule provided in this embodiment of the present invention includes:

[0169] The task analysis unit is responsible for recording the time from the start to the end of the task, analyzing the time consumption of the task, and identifying whether there is a timeout phenomenon; monitoring the use of resources such as CPU, memory, and disk I / O; recording the final output results of the task, capturing errors that occur during the task execution process, analyzing the type and frequency of the error, and finding the root cause of the problem;

[0170] The efficiency estimation unit is responsible for dynamically adjusting the data volume threshold according to the amount of data processed by the task, and dynamically adjusting the algorithm complexity threshold according to the complexity of the task; counting the ratio of the number of successful executions of the task to the total number of executions, and the ratio of the number of failed executions to the total number of executions, calculating the average processing time of the task, and evaluating the execution efficiency of the task;

[0171] The trend identification unit is responsible for analyzing whether the complexity assessment of the task is accurate and whether there is an over- or under-assessment; analyzing the trend of task execution data to identify whether there is a trend of performance degradation or increased resource consumption; and adjusting the complexity assessment strategy of the task based on the assessment results.

[0172] In the above embodiment, the task analysis unit records the time from the start to the end of the task, can accurately analyze the time consumption of the task, and identify whether there is a timeout phenomenon; by identifying the timeout phenomenon, it can timely discover the bottleneck in the task execution, providing a basis for subsequent optimization; monitoring the usage of resources such as CPU, memory and disk I / O, it can understand the consumption of system resources in real time, and ensure the rational use of system resources; through resource monitoring, it can timely discover resource bottlenecks and provide data support for system optimization; recording the final output result of the task can evaluate the effect of task execution and ensure the accuracy and completeness of the task output; capturing errors that occur during the task execution process, analyzing the type and frequency of errors, finding the root cause of the problem, and providing a basis for error repair; through error analysis, it can quickly locate the root cause of the problem and improve the efficiency of problem solving. The efficiency estimation unit dynamically adjusts the data volume threshold according to the amount of data processed by the task, and dynamically adjusts the algorithm complexity threshold according to the complexity of the task to ensure that the task is executed in the best state; by dynamically adjusting the threshold, it can optimize the use of resources and improve the efficiency of task execution; count the ratio of the number of successful executions of the task to the total number of executions, evaluate the success rate of the task, and ensure the stability of task execution; count the ratio of the number of failed executions to the total number of executions, evaluate the failure rate of the task, and promptly discover and solve problems in task execution; calculate the average processing time of the task, evaluate the execution efficiency of the task, and provide data support for task optimization. The trend identification unit analyzes whether the complexity assessment of the task is accurate, whether there is an over- or under-assessment, and ensures the accuracy of the complexity assessment; adjusts the complexity assessment strategy of the task based on the assessment results to ensure that the task is executed at the optimal complexity; analyzes the trend of the task execution data to identify whether there is a trend of performance degradation or increased resource consumption, and promptly discovers changes in system performance; through trend analysis, it can promptly discover the degradation of system performance or the increase in resource consumption, and provide early warning for system optimization; according to the assessment results, adjusts the complexity assessment strategy of the task to ensure that the task is executed in the best state; through continuous evaluation and strategy adjustment, ensure the continuous optimization of system performance and improve the efficiency and stability of task execution.

[0173] In summary, the optimization and adjustment submodule of this embodiment realizes comprehensive monitoring and optimization of the task execution process through the task analysis unit, efficiency estimation unit and trend identification unit. The direct technical effects of each unit include task time analysis, timeout warning, resource usage monitoring, resource bottleneck identification, task output and error analysis, problem location, dynamic threshold adjustment, resource optimization, success rate statistics, failure rate statistics, average processing time calculation, evaluation accuracy analysis, evaluation strategy adjustment, performance trend analysis, trend warning and strategy optimization, etc., to ensure that the system can perform tasks in the best state and improve the efficiency and stability of task execution.

[0174] Embodiment 10:

[0175] like Fig.10 As shown, based on Embodiment 1 to Embodiment 9, the early warning method of the intelligent security monitoring system provided by the embodiment of the present invention comprises the following steps:

[0176] Step S100: The monitoring device collects video data in real time, compresses it through a lightweight algorithm on the edge device, analyzes the compressed real-time video data, and identifies key information such as faces, objects, or abnormal behaviors; marks and extracts the identified key information (such as faces, objects, abnormal behaviors), and uploads it to the cloud for processing;

[0177] Step S200: The cloud server receives key information from the edge device and further identifies and analyzes it; identifies the identity and abnormal behavior of specific personnel; based on the identification results, the cloud server performs pattern matching to determine whether there is a potential security threat; if abnormal behavior or key personnel are identified, triggers an early warning mechanism;

[0178] Step S300: Intelligently schedule tasks based on the nature of key information and task requests; after the edge device identifies the key information, it uploads it to the cloud for further analysis; the cloud processing results are fed back to the edge device in real time; when the cloud server identifies a potential security threat, it immediately triggers an early warning; at the same time, the edge device will also initiate corresponding emergency measures (such as alarms and video recording).

[0179] In the above embodiment, the edge computing and data compression in step S100 can process video data quickly and ensure real-time performance through the lightweight algorithm on the edge device; the amount of compressed video data transmission is greatly reduced, saving network bandwidth and reducing data transmission costs; the lightweight algorithm on the edge device can quickly process data locally and reduce the delay of uploading data to the cloud. Real-time data analysis and key information extraction can quickly identify key information such as faces, objects or abnormal behaviors through the algorithm on the edge device; key information can be quickly marked and extracted on the edge device, reducing the delay of uploading data to the cloud and improving the real-time performance of the early warning; only uploading key information to the cloud reduces the transmission of invalid data and improves the efficiency of data processing. Step S200 Cloud centralized processing and analysis, the cloud server can perform high-precision identification and analysis of key information through deep learning models and behavior analysis models; the cloud server can perform pattern matching based on the identification results, determine whether there is a potential security threat, and trigger the early warning mechanism; the cloud server can process data from multiple edge devices at the same time, realize the fusion analysis of multi-source data, and improve the accuracy and comprehensiveness of the early warning. When the cloud server identifies abnormal behavior or key personnel, it can immediately trigger the early warning mechanism and notify relevant personnel through multiple channels (such as SMS, email, APP push); the triggering of the early warning mechanism can ensure that security incidents are responded to quickly and reduce potential security risks. Step S300 The edge and the cloud work together to intelligently schedule tasks according to the nature of key information and task requests to ensure that tasks are processed on the most suitable device, thereby improving the efficiency and accuracy of task processing; through intelligent scheduling, the computing resources of edge devices and cloud servers can be maximized to avoid waste of resources; real-time communication is maintained between edge devices and cloud servers to ensure the synchronization of key information and task requests and reduce data transmission delays; cloud processing results can be fed back to edge devices in real time to guide their subsequent operations and ensure the overall coordination and consistency of the system; when the cloud server identifies potential security threats, it can immediately trigger an early warning and notify relevant personnel through multiple channels to ensure the timeliness and effectiveness of the early warning; the edge device can immediately initiate corresponding emergency measures (such as alarms and video storage) based on the early warning information fed back by the cloud to ensure that security incidents are quickly handled.

[0180] In summary, the early warning method of the intelligent security monitoring system of this embodiment realizes efficient and accurate early warning function through three steps: edge computing and data compression, centralized cloud processing and analysis, and collaborative work between edge and cloud. Each step improves real-time performance, saves bandwidth, improves data processing efficiency, high-precision recognition, pattern matching and early warning, task optimization, maximizes resource utilization, real-time communication, information feedback, multi-level early warning and emergency response, etc., to ensure that the system can identify and respond to potential security threats in the first place.

[0181] Fig.11 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present invention is shown.

[0182] The electronic device may include a central processing unit / microprocessor / main control chip, etc.; a storage medium, coupled to the central processing unit / microprocessor / main control chip, etc., and storing computer executable instructions therein, for performing the steps of each method of an embodiment of the present invention when executed by the processor.

[0183] The central processing unit / microprocessor / main control chip etc. may include but are not limited to, for example, one or more processors or microprocessors etc.

[0184] The storage medium may include, but is not limited to, for example, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disc, etc.).

[0185] In addition, the electronic device may also include (but not limited to) a data bus, an input / output bus / external bus / device bus, a display, and input / output devices (eg, keyboard, mouse, speaker, etc.), etc.

[0186] The central processing unit / microprocessor / main control chip etc. can communicate with external devices via an I / O bus via a wired or wireless network (not shown).

[0187] The storage medium may also store at least one computer executable instruction for executing the various functions and / or method steps in the embodiments described in the present technology when executed by a central processing unit / microprocessor / main control chip, etc.

[0188] In one embodiment, the at least one computer executable instruction may also be compiled into or constitute a software product, wherein one or more computer executable instructions are executed by a processor to perform the various functions and / or method steps in the embodiments described in the present technology.

[0189] Fig.12 A schematic diagram of a computer-readable storage medium according to an embodiment of the present invention is shown.

[0190] like Fig.12As shown, instructions are stored on a non-transitory computer-readable storage medium, and the instructions are, for example, computer-readable instructions. When the computer-readable instructions are executed by the processor, the various methods described above can be executed. The non-transitory computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may, for example, include random access memory (RAM) and / or cache memory (cache), etc. The non-transitory non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc. For example, the non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device runs the computer-readable instructions stored on the computer-readable storage medium, the various methods described above may be performed.

[0191] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0192] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0193] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0194] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent security monitoring system, characterized in that: Include: The edge computing and data compression module is responsible for compressing the video data obtained by the monitoring device by running a lightweight algorithm on the edge device; analyzing the compressed real-time video data to identify key information such as faces, objects or abnormal behaviors; The cloud centralized processing and analysis module is deployed on the cloud server and processes massive data from multiple edge devices simultaneously through multiple parallel processing channels; Receive key information, identify the key information, and obtain a pattern corresponding to the key information; The edge and cloud collaborative work module is responsible for intelligent scheduling of key information and task requests, deciding which tasks are processed on edge devices and which tasks are uploaded to the cloud for processing; receiving the corresponding modes of key information and coordinating the work of edge devices and the cloud.

2. The intelligent security monitoring system according to claim 1, characterized in that: Edge computing and data compression module, including: The data compression submodule is responsible for running the designed lightweight algorithm on the edge device to compress the video data in real time; splitting the compressed video stream into multiple time segments or frames; The feature extraction submodule is responsible for extracting visual features and identifying key information of faces and objects in each segmented segment; detecting sudden movements and abnormal aggregation behaviors in the video through behavioral analysis; The real-time feedback submodule is responsible for feeding back the identified key information and patterns to the monitoring equipment in real time for decision-making and response.

3. The intelligent security monitoring system according to claim 2, characterized in that: Feature extraction submodule, including: The primary feature extraction unit is responsible for entering the primary feature extraction stage after the image preprocessing is completed. By calculating the gradient value of the pixel points in the image, the edge lines in the image are detected, and the contour and structure of the object in the image are preliminarily reflected; the corner points in the image are identified, and the local autocorrelation matrix of each pixel point in the image is calculated to determine the position of the corner points; The intermediate feature extraction unit is responsible for generating a binary code by comparing the grayscale value of each pixel in the image with the surrounding pixels to describe the local texture features of the skin texture of the face or the surface lines of the object in the image; by calculating the gradient direction and amplitude of each pixel in the image, a histogram is generated to describe the contour of the face or the local shape features of the edge of the object in the image; The high-level feature extraction unit is responsible for performing convolution operations on the input image through multiple convolution kernels to extract local features in the image; downsampling the output of the convolution layer; after multiple layers of convolution and pooling operations, CNN will flatten the feature map and input it to the fully connected layer, which combines and abstracts the features through the weight matrix to extract high-level features in the image; The feature fusion and optimization unit is responsible for splicing primary features, intermediate features and advanced features to form a multi-level feature vector that contains multi-level information in the image and describes the image content; it also reduces the dimension and optimizes the spliced ​​feature vector.

4. The intelligent security monitoring system according to claim 3, characterized in that: Feature fusion and optimization unit, including: The classifier training subunit is responsible for training the classifier using the labeled training data set in the historical video data, and separating the feature vectors of different categories by finding the optimal hyperplane; The feature classification subunit is responsible for classifying the fused feature vectors using the trained classifier after training. The classifier determines whether there is key information of faces and objects in the image based on the distribution of the feature vectors. The model optimization subunit is responsible for optimizing the classifier through cross-validation.

5. The intelligent security monitoring system according to claim 1, characterized in that: Cloud-based centralized processing and analysis modules, including: The pattern confirmation submodule is responsible for pre-building a pattern library containing various key information patterns, matching the extracted feature vectors with the patterns in the pattern library, calculating the similarity, and determining the best matching pattern; The pattern fusion submodule is responsible for correcting the identified patterns based on real-time video data and feedback information, and regularly updating the pattern library; Fusion of patterns identified by multiple edge devices to generate key information patterns; The result display submodule is responsible for displaying the identified key information patterns in a visual way. The recognition results are stored in the database. Based on the recognition results, the work of edge devices and the cloud is intelligently scheduled to decide which tasks are processed on the edge devices and which tasks are uploaded to the cloud for processing.

6. The intelligent security monitoring system according to claim 5, characterized in that: Mode fusion submodule, including: The cluster merging unit is responsible for treating the pattern identified by each edge device as an independent cluster and calculating the similarity matrix between all patterns. It finds the two clusters with the highest similarity from the similarity matrix and merges them into a new cluster. Update the similarity matrix and calculate the similarity between the new cluster and other clusters; The weighted average unit is responsible for the merged clusters to represent a new key information pattern. The feature vector of the new pattern is obtained by weighted average of the feature vectors of each sub-pattern; The iterative merging unit is responsible for iterating the merging process until any of the following conditions is met: the preset number of clusters is reached, and the similarity is lower than a preset threshold.

7. The intelligent security monitoring system according to claim 6, characterized in that: Clustering merging unit, including: The set construction subunit is responsible for treating the pattern recognized by each edge device as an independent cluster and forming an initial cluster set from all independent clusters; calculating the similarity matrix between all clusters and quantifying the similarity between different clusters; Construct an N×N similarity matrix, where N is the number of clusters and each element in the matrix represents the similarity between the corresponding two clusters; The cluster merging subunit is responsible for traversing the similarity matrix to find the two clusters with the highest similarity; and recording the indexes of the two clusters with the highest similarity; Merge the two clusters with the highest similarity into a new cluster to generate a new cluster object; Remove the two merged clusters from the initial cluster set and add the newly generated cluster to the cluster set; The matrix update subunit is responsible for calculating the similarity between the new cluster and other clusters and updating the similarity matrix; Update the relevant rows and columns in the similarity matrix.

8. The intelligent security monitoring system according to claim 1, characterized in that: Edge and cloud collaborative working module, including: Collaborative definition submodule, responsible for classifying the recognized face information according to the matching degree and recognition accuracy of the face database; setting the priority of the task; The task evaluation submodule is responsible for evaluating the complexity of tasks. High-complexity tasks are processed on edge devices first, while low-complexity tasks are uploaded to the cloud for processing. The optimization and adjustment submodule is responsible for feeding back the cloud processing results to the edge device in real time, and the edge device makes real-time adjustments and optimizations based on the feedback results; The cloud regularly summarizes the processing results and feeds them back to the edge devices. The edge devices make periodic adjustments and optimizations based on the feedback results. The task scheduling strategy is dynamically adjusted based on the real-time feedback results.

9. The intelligent security monitoring system according to claim 8, characterized in that: Optimize and adjust submodules, including: The task analysis unit is responsible for recording the time from the start to the end of the task, analyzing the time consumption of the task, and identifying whether there is an overtime phenomenon; Monitor the usage of CPU, memory and disk I / O resources; record the final output of the task, capture errors that occur during task execution, analyze the type and frequency of errors, and find the root cause of the problem; The efficiency estimation unit is responsible for dynamically adjusting the data volume threshold according to the amount of data processed by the task, and dynamically adjusting the algorithm complexity threshold according to the complexity of the task; Count the ratio of successful execution times to total execution times, and the ratio of failed execution times to total execution times, calculate the average processing time of tasks, and evaluate the execution efficiency of tasks; The trend identification unit is responsible for analyzing whether the complexity assessment of the task is accurate and whether there is an over- or under-assessment; analyzing the trend of task execution data to identify whether there is a trend of performance degradation or increased resource consumption; Adjust the complexity assessment strategy of the task based on the evaluation results.

10. An early warning method for an intelligent security monitoring system, characterized in that: The following steps are involved: The monitoring equipment collects video data in real time and compresses it through a lightweight algorithm on the edge device. The compressed real-time video data is analyzed to identify key information such as faces, objects, or abnormal behaviors. The identified key information is marked and extracted and uploaded to the cloud for processing. The cloud server receives key information from the edge device and further identifies and analyzes it; identifies the identity and abnormal behavior of specific people; based on the identification results, the cloud server performs pattern matching to determine whether there is a potential security threat; If abnormal behavior or key personnel are identified, an early warning mechanism is triggered; Intelligently schedule tasks based on key information and the nature of task requests; After the edge device identifies key information, it uploads it to the cloud for further analysis; the cloud processing results are fed back to the edge device in real time; when the cloud server identifies potential security threats, an early warning is immediately triggered; at the same time, the edge device will also initiate corresponding emergency measures.

Images