Intelligent monitoring method and system based on Internet of Things, medium and program product

Abstract

The invention discloses an intelligent monitoring method and system based on the Internet of Things, a medium and a program product, and the method comprises the steps: abstracting a monitoring region into a dynamic graph structure, and obtaining a self-adaptive monitoring scene graph; fusing at least one piece of video data and at least one kind of data of other sensors at each node to form a multi-mode sensing network; carrying out distributed feature extraction on the multi-modal sensing data to obtain data features, and compressing the data features into low-dimensional vectors; constructing a space-time knowledge graph; executing an anomaly detection algorithm on a preset number of spatial scales; matching a corresponding adjustment mode in a preset adjustment scheme according to the type and the degree of the abnormal event, and adjusting the numerical weights of the nodes and the edges to obtain an adjusted adaptive monitoring scene graph; and according to the adjusted adaptive monitoring scene graph and the numerical value weight corresponding to the node, synchronously adjusting the working parameters of all the cameras of the Internet of Things and other sensors. According to the invention, the integrity and comprehensiveness of the monitoring data acquired by the Internet of Things camera equipment are improved.

Description

Technical Field

[0001] The present application belongs to the field of image communication, and in particular, relates to an intelligent monitoring method, system, medium and program product based on the Internet of Things. Background Art

[0002] With the rapid development of IoT technology, IoT-based intelligent monitoring methods have been widely used in urban management, public safety, and traffic control. This method mainly relies on a fixed IoT camera network to collect video data and analyze it to achieve monitoring goals. However, traditional intelligent monitoring methods have problems such as limited coverage, insufficient adaptability, and low data processing efficiency. They are difficult to meet the increasingly complex needs of urban environment monitoring, especially in emergencies or emergencies, and cannot be quickly adjusted to adapt to new monitoring requirements.

[0003] To solve the above problems, the existing technology proposes an intelligent monitoring system based on edge computing. The system integrates an edge computing module in each IoT camera, enabling the camera to have local data processing and analysis capabilities. By distributing some computing tasks from the central server to each camera, this method significantly improves data processing efficiency, reduces network transmission pressure, and enhances the system's real-time response capabilities. At the same time, edge computing also enables the camera to autonomously adjust parameters according to local environmental changes, improving the adaptability of the system.

[0004] However, the existing edge computing-based intelligent monitoring system still faces a key technical problem: the lack of an effective cross-device collaboration mechanism. Although each camera has a certain degree of intelligent processing capabilities, they still operate relatively independently and cannot fully utilize the collective intelligence of the entire network. Especially in complex monitoring scenarios, the perspective and information of a single camera are often limited, making it difficult to fully capture and understand the development process of the entire event. Summary of the invention

[0005] The present application provides an intelligent monitoring method, system, medium and program product based on the Internet of Things, which are used to improve the integrity and comprehensiveness of monitoring data acquired by Internet of Things camera equipment.

[0006] In the first aspect, the present application provides an intelligent monitoring method based on the Internet of Things, which abstracts the monitoring area into a dynamic graph structure to obtain an adaptive monitoring scene graph, wherein the nodes in the adaptive monitoring scene graph represent predetermined key monitoring points, and the edges represent the logical association relationship between each node; At each node, a multimodal perception network is formed by fusing video data from at least one IoT camera and data from at least one other sensor, including a sound sensor, a thermal sensor, and a motion sensor; Using a preset feature extraction algorithm to perform distributed feature extraction on the multimodal sensor data of each node, obtain data features, and compress the data features into a low-dimensional vector of a preset dimension. The multimodal sensor data is data collected by the multimodal perception network; Based on the adaptive monitoring scene graph and low-dimensional vector, a spatiotemporal knowledge graph reflecting the current status of the monitoring area is constructed; Execute anomaly detection algorithms at a preset number of spatial scales based on the spatiotemporal knowledge graph; When it is determined that an abnormal event exists according to the abnormality detection algorithm, a corresponding adjustment method is matched in the preset adjustment scheme according to the type and degree of the abnormal event, and the numerical weights of the corresponding nodes and edges in the adaptive monitoring scene graph are adjusted according to the adjustment method to obtain an adjusted adaptive monitoring scene graph; According to the adjusted adaptive monitoring scene graph and the numerical weights corresponding to the nodes, the working parameters of all IoT cameras and other sensors are adjusted synchronously. The working parameters of IoT cameras include angle, focal length and sampling rate, and the working parameters of other sensors include sampling frequency.

[0007] By adopting the above technical solution, the monitoring area is abstracted into a dynamic graph structure, and the construction of an adaptive monitoring scene graph is realized. This graph structure enables the system to flexibly represent and manage complex monitoring environments and effectively capture the logical associations between key monitoring points. By fusing multiple sensor data at each node to form a multimodal perception network, the system obtains more comprehensive and richer environmental information, improving the accuracy and reliability of monitoring. Distributed feature extraction and low-dimensional vector compression technology enable the system to efficiently process a large amount of multimodal data, improving the efficiency of calculation and storage. The spatiotemporal knowledge graph constructed based on the adaptive monitoring scene graph and low-dimensional vector provides rich contextual information for anomaly detection and enhances the system's ability to understand complex events. The application of multi-scale anomaly detection algorithms improves the system's ability to identify abnormal events of different types and scales. The mechanism of dynamically adjusting the monitoring scene graph and sensor parameters according to abnormal events enables the system to adapt to environmental changes in real time, optimize resource allocation, and improve monitoring efficiency. This technical solution improves the integrity and comprehensiveness of monitoring data acquired by IoT camera devices.

[0008] In combination with some embodiments of the first aspect, in some embodiments, video data of at least one IoT camera and data of at least one other sensor are fused at each node to form a multimodal perception network, specifically including: Preprocess the IoT camera video data of each node to obtain preprocessed video data, where the preprocessing includes denoising, image stabilization and resolution standardization; Convert the data from other sensors into time series data that is time synchronized with the preprocessed video data; A multimodal fusion algorithm is used to integrate the preprocessed video data and time series data into a unified multimodal feature representation to form a multimodal perception network.

[0009] By adopting the above technical solutions, the preprocessing of IoT camera video data, the time synchronization conversion of other sensor data and the application of multimodal fusion algorithms have achieved high-quality multimodal data integration. The denoising, image stabilization and resolution standardization in the video data preprocessing step improve the quality and consistency of video data and reduce the interference of environmental factors on subsequent analysis. The conversion of other sensor data into a time series synchronized with the video data improves the consistency of different types of data in the time dimension. The implementation of time synchronization enables the system to accurately align the data of different sensors, so as to better understand and explain the events occurring in the monitoring area. The application of multimodal fusion algorithms integrates different types of data into a unified feature representation, fully utilizes the complementarity of various sensor data, and improves the system's perception of the environment. This fusion method can not only capture complex events that are difficult to identify with a single sensor, but also reduce the possible misjudgment caused by a single sensor, thereby improving the accuracy and reliability of the monitoring system.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, distributed feature extraction is performed on multimodal sensing data of each node using a preset feature extraction algorithm, specifically including: Extracting spatial feature information of video data from multimodal sensor data; Extracting temporal feature information of time series data from multimodal sensor data; Integrate spatial feature information and temporal feature information into data features.

[0011] By adopting the above technical solutions, the distributed feature extraction method takes advantage of the characteristics of different types of data and improves the accuracy and efficiency of feature extraction. By integrating spatial features and temporal features, the system obtains a comprehensive, multi-dimensional data feature representation that contains both spatial and temporal information, which can better reflect the complexity and dynamics of the monitoring environment. It improves processing efficiency, enhances the scalability of the system, and can better cope with large-scale monitoring scenarios.

[0012] In combination with some embodiments of the first aspect, in some embodiments, after synchronously adjusting the working parameters of all IoT cameras and other sensors according to the adjusted adaptive monitoring scene graph and the numerical weights corresponding to the nodes, the method further includes: Combining spatiotemporal knowledge graphs and abnormal events, a structured situation-aware event description is generated that includes event semantic description, impact scope definition, and potential evolution trend analysis. Based on the description of context-aware events, the preset graph representation learning model and time series prediction model are used to predict the development path of abnormal events within a predetermined time window and generate corresponding forward-looking response strategies; Using online learning algorithms, we continuously learn newly generated abnormal events and corresponding forward-looking response strategies, and update the structure of the adaptive monitoring scene graph and the parameters of the anomaly detection algorithm.

[0013] By adopting the above technical solutions, the development path of abnormal events is predicted by using the preset graph representation learning model and the time series prediction model, so that the system has the prediction function and can predict the development of the situation in advance, so as to formulate a more effective response strategy. Generate a forward-looking response strategy so that the system can actively respond to potential risks instead of passively react, which improves the prevention and disposal capabilities of the system. Through the online learning algorithm, the system can continuously update and optimize its own knowledge base and decision-making model by continuously learning new abnormal events and corresponding forward-looking response strategies, so that the system can become more intelligent and efficient over time and better respond to new abnormal situations. Update the structure of the adaptive monitoring scene graph and the parameters of the anomaly detection algorithm to improve the flexibility and accuracy of the system. This dynamic update mechanism enables the system to adapt to changes in the monitoring environment and maintain efficient monitoring performance.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, a structured context-aware event description including event semantic description, impact scope definition, and potential evolution trend analysis is generated by combining a spatiotemporal knowledge graph and abnormal events, specifically including: Parse the entities and relationships in the spatiotemporal knowledge graph to obtain semantic descriptions of events; Based on the results of the anomaly detection algorithm, determine the impact scope and severity level of the abnormal event and obtain the impact scope definition; Use the preset causal reasoning model to analyze the potential causes and evolution trends of abnormal events and obtain potential evolution trend analysis; The semantic description of events, the definition of impact scope and the analysis of potential evolution trends are integrated to obtain a structured situation-aware event description.

[0015] By adopting the above technical solutions, the system can more accurately assess the actual impact of abnormal events based on the impact scope and severity level determined by the results of the anomaly detection algorithm. The system's prediction ability is enhanced by analyzing the potential causes and evolution trends of abnormal events using a preset causal reasoning model. The semantic description of events, the definition of the impact scope, and the analysis of potential evolution trends are integrated into a structured context-aware event description, which includes both the static characteristics of the event and the dynamic development information, improving the pertinence and effectiveness of event processing.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, after continuously learning newly generated abnormal events and corresponding forward-looking response strategies using an online learning algorithm and updating the structure of the adaptive monitoring scene graph and the parameters of the abnormality detection algorithm, the method further includes: Build an abnormal event knowledge base and store historical abnormal events and corresponding historical forward-looking response strategies in the abnormal event knowledge base; Use a clustering algorithm to classify historical abnormal events to obtain common abnormal event types, where the common abnormal event types are types corresponding to historical abnormal events whose number is greater than a preset threshold; For each common abnormal event type, statistically analyze the success rate of the corresponding historical forward-looking response strategy; Determine the historical forward-looking response strategy with the highest success rate as the best response strategy, and establish a mapping relationship between the best response strategy and the corresponding common abnormal event types; Based on the mapping relationship, abnormal event handling suggestions are automatically generated.

[0017] By adopting the above technical solutions, constructing an abnormal event knowledge base, clustering analysis, statistical analysis and establishing mapping relationships, we can achieve continuous optimization and automatic suggestion generation for abnormal event handling strategies. Using clustering algorithms to classify historical abnormal events and obtain common abnormal event types helps the system identify and summarize common problems, and improves the efficiency and accuracy of abnormal event handling. For each common abnormal event type, the success rate of the corresponding historical forward-looking response strategy is statistically analyzed, so that the system can quantitatively evaluate the effects of different strategies and provide an objective basis for selecting the best strategy. The historical forward-looking response strategy with the highest success rate is determined as the best response strategy, and a mapping relationship with common abnormal event types is established, which achieves optimization and automation of strategy selection, improves the efficiency of abnormal event handling, and can continuously optimize the processing strategy, making the system more intelligent and efficient as the usage time increases.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, based on the mapping relationship, an abnormal event handling suggestion is automatically generated, specifically including: Establish a template for abnormal event handling suggestions, which includes event description, possible causes, recommended response strategies and expected effects; When a new abnormal event is detected, the type with the highest similarity to the new abnormal event is searched among the common abnormal event types to obtain the corresponding mapping relationship; According to the mapping relationship, determine the corresponding optimal response strategy; The specific information of the new abnormal event and the obtained best response strategy are filled into the abnormal event handling suggestion template to obtain the abnormal event handling suggestion.

[0019] By adopting the above technical solutions, establishing a processing suggestion template, similarity matching and information filling, the rapid generation and personalized customization of abnormal event processing suggestions are realized. The abnormal event processing suggestion template is established, which contains key information such as event description, possible causes, recommended response strategies and expected effects, providing a structured framework for processing suggestions, ensuring that the generated suggestions are comprehensive and well-organized, and improving the speed, quality and consistency of abnormal event response.

[0020] In a second aspect, an embodiment of the present application provides an intelligent monitoring system based on the Internet of Things, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and one or more processors call the computer instructions to enable the system to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a system, causes the system to execute the method described in the first aspect and any possible implementation of the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer program product, characterized in that when the computer program product runs on a system, the system executes the method described in any possible implementation manner in the first aspect.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The present application provides an intelligent monitoring method based on the Internet of Things, which abstracts the monitoring area into a dynamic graph structure and realizes the construction of an adaptive monitoring scene graph. This graph structure enables the system to flexibly represent and manage complex monitoring environments and effectively capture the logical associations between key monitoring points. By fusing multiple sensor data at each node to form a multimodal perception network, the system obtains more comprehensive and richer environmental information and improves the accuracy and reliability of monitoring. Distributed feature extraction and low-dimensional vector compression technology enable the system to efficiently process a large amount of multimodal data and improve the efficiency of calculation and storage. The spatiotemporal knowledge graph constructed based on the adaptive monitoring scene graph and low-dimensional vector provides rich contextual information for anomaly detection and enhances the system's ability to understand complex events. The application of multi-scale anomaly detection algorithms improves the system's ability to identify abnormal events of different types and scales. The mechanism of dynamically adjusting the monitoring scene graph and sensor parameters according to abnormal events enables the system to adapt to environmental changes in real time, optimize resource allocation, and improve monitoring efficiency. This technical solution improves the integrity and comprehensiveness of monitoring data acquired by IoT camera equipment.

[0024] 2. The present application provides an intelligent monitoring method based on the Internet of Things, which uses a preset graph representation learning model and a time series prediction model to predict the development path of abnormal events, so that the system has a prediction function and can predict the development of events in advance, so as to formulate a more effective response strategy. Generate a forward-looking response strategy so that the system can actively respond to potential risks instead of passively react, thereby improving the prevention and disposal capabilities of the system. Through the online learning algorithm, the system can continuously learn new abnormal events and corresponding forward-looking response strategies, and the system can continuously update and optimize its own knowledge base and decision-making model, so that the system can become more intelligent and efficient over time, and can better respond to new abnormal situations. Updating the structure of the adaptive monitoring scene graph and the parameters of the anomaly detection algorithm improves the flexibility and accuracy of the system. This dynamic update mechanism enables the system to adapt to changes in the monitoring environment and maintain efficient monitoring performance.

[0025] 3. This application provides an intelligent monitoring method based on the Internet of Things. Based on the impact scope and severity level determined by the results of the anomaly detection algorithm, the system can more accurately evaluate the actual impact of abnormal events. The potential causes and evolution trends of abnormal events are analyzed using a preset causal reasoning model to enhance the system's predictive ability. The semantic description of events, the definition of the impact scope, and the analysis of potential evolution trends are integrated into a structured context-aware event description, which includes the static characteristics of the event and the dynamic development information, thereby improving the pertinence and effectiveness of event processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of an intelligent monitoring method based on the Internet of Things in an embodiment of the present application.

[0027] Figure 2 It is a flow chart of an abnormal event prediction method based on intelligent monitoring of the Internet of Things in an embodiment of the present application.

[0028] Figure 3 It is a schematic diagram of the physical device structure of an intelligent monitoring system based on the Internet of Things provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more listed items.

[0030] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0031] Combine the following Figure 1 , describes an intelligent monitoring method based on the Internet of Things in an embodiment of the present application: please refer to Figure 1 , which is a flow chart of an intelligent monitoring method based on the Internet of Things in an embodiment of the present application.

[0032] S101, abstracting the monitoring area into a dynamic graph structure to obtain an adaptive monitoring scene graph; The system abstracts the monitoring area into a dynamic graph structure to obtain an adaptive monitoring scene graph. The nodes in the adaptive monitoring scene graph represent pre-determined key monitoring points, and the edges represent the logical associations between each node. In this process, the system first conducts a comprehensive analysis of the monitoring area and identifies key monitoring points, which may include entrances and exits, locations of important equipment, and areas with dense personnel. Each key monitoring point is abstracted as a node in the graph structure. Next, the system analyzes the logical associations between these nodes, such as physical distance, functional relevance, data flow paths, etc., and represents these relationships as edges between nodes.

[0033] When constructing an adaptive monitoring scene graph, the system can adopt a multi-level graph structure design. For example, the entire monitoring area can be divided into multiple sub-areas, each of which forms a sub-graph, and then these sub-graphs are connected to form the overall monitoring scene graph. This hierarchical design helps improve the scalability and management efficiency of the system. At the same time, the system can assign initial weights to each node and edge to reflect its importance in the entire monitoring network. These weights can be derived based on pre-set rules or historical data analysis.

[0034] In order to achieve dynamic adaptability, the system can design an automatic update mechanism. When the monitoring area changes, such as adding or removing certain monitoring points, the system can automatically adjust the graph structure and add or delete the corresponding nodes and edges. In addition, the system can periodically evaluate the importance of each node and edge and dynamically adjust its weight according to the actual monitoring effect. This adaptive mechanism enables the monitoring scene graph to be continuously optimized over time to better reflect the actual monitoring needs.

[0035] The system can also embed time dimension information in the adaptive monitoring scene graph. For example, a timestamp attribute can be added to each node to record the most recent data update time or the time when an abnormal event occurred. This time-space fusion graph structure design facilitates subsequent time series analysis and event tracking.

[0036] S102, fusing video data of at least one IoT camera and data of at least one other sensor at each node to form a multimodal perception network; At each node, the video data of at least one IoT camera and the data of at least one other sensor are integrated to form a multimodal perception network. The other sensors include sound sensors, thermal sensors, and motion sensors. Specifically: the video data of the IoT camera of each node is preprocessed to obtain preprocessed video data. The preprocessing includes denoising, image stabilization, and resolution standardization. Convert the data from other sensors into time series data that is time synchronized with the preprocessed video data; A multimodal fusion algorithm is used to integrate the preprocessed video data and time series data into a unified multimodal feature representation to form a multimodal perception network.

[0037] The system first preprocesses the video data of the IoT camera of each node. The preprocessing process includes denoising, image stabilization and resolution standardization. Denoising can use algorithms such as Gaussian filtering and median filtering to effectively reduce random noise in the video and improve image quality. Image stabilization technology can compensate for camera jitter or vibration to ensure the stability of the video. Resolution standardization is to unify the videos collected by different cameras into a standard resolution for subsequent processing.

[0038] For other sensor data, the system converts it into time series data that is synchronized with the preprocessed video data. This step involves data synchronization and timestamp alignment technology. The system can use interpolation or resampling methods to ensure that all sensor data are aligned in the time dimension, laying the foundation for subsequent multimodal fusion.

[0039] After data preprocessing and synchronization are completed, the system uses a multimodal fusion algorithm to integrate the preprocessed video data and time series data into a unified multimodal feature representation. Early fusion, mid-term fusion, or late fusion strategies can be used here. Early fusion is performed directly at the raw data level, mid-term fusion is performed after feature extraction, and late fusion is performed at the decision level. The system can select an appropriate fusion strategy based on the specific application scenario and data characteristics.

[0040] S103, using a preset feature extraction algorithm to perform distributed feature extraction on the multimodal sensor data of each node to obtain data features, and compressing the data features into a low-dimensional vector of a preset dimension; The system uses a preset feature extraction algorithm to perform distributed feature extraction on the multimodal sensor data of each node, obtains data features, and compresses the data features into a low-dimensional vector of a preset dimension. The multimodal sensor data is the data collected by the multimodal perception network. Specifically: extract the spatial feature information of the video data in the multimodal sensor data; Extracting temporal feature information of time series data from multimodal sensor data; Integrate spatial feature information and temporal feature information into data features.

[0041] The system extracts spatial feature information from video data in multimodal sensor data. Here, deep learning models such as convolutional neural networks (CNNs), such as ResNet or Inception networks, can be used to extract rich spatial features from video frames. The system can focus on key information such as object detection, person recognition, and action recognition, and extract features that can reflect scene content and changes.

[0042] For time series data, the system extracts its temporal feature information. This can be achieved through models such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs), or temporal convolutional networks (TCNs). These models can capture the temporal dependency and long-term trends of data and extract features that reflect the data change pattern.

[0043] After extracting spatial features and temporal features, the system integrates these two types of feature information into a unified data feature. The integration process can use an attention mechanism or a gated fusion network to adaptively learn the importance of different modal features and generate a comprehensive feature representation.

[0044] In order to improve the efficiency and adaptability of feature extraction, the system can adopt a distributed computing framework. Each node can independently perform preliminary feature extraction, and then aggregate the extracted features to the central server for further integration and optimization. This distributed architecture can not only improve computing efficiency, but also reduce network bandwidth requirements.

[0045] After feature extraction is completed, the system compresses the data features into a low-dimensional vector of a preset dimension. This step can be achieved through dimensionality reduction techniques such as principal component analysis (PCA), t-SNE, or autoencoders. The dimension of the low-dimensional vector can be dynamically adjusted according to actual needs and computing resources to achieve a balance between information retention and computing efficiency.

[0046] S104, constructing a spatiotemporal knowledge graph reflecting the current state of the monitoring area based on the adaptive monitoring scene graph and the low-dimensional vector; The process of building a spatiotemporal knowledge graph first involves mapping the nodes and edges in the adaptive monitoring scene graph to entities and relationships in the knowledge graph. Each monitoring node can be transformed into an entity in the knowledge graph, whose attributes include location information, device type, current status, etc. The edges between nodes can be mapped to the relationship between entities, representing physical connections, logical associations, or data flows, etc.

[0047] The system uses the low-dimensional feature vector obtained in the previous step as the dynamic attribute of the entity, and combines it with static attributes (such as device ID, installation location, etc.) to form a comprehensive entity description. These feature vectors can be updated periodically to reflect the real-time status changes of the monitored object.

[0048] In order to build spatiotemporal associations, the system can introduce the time dimension into the knowledge graph. Each entity and relationship can be attached with a timestamp to indicate the time when the data was collected or when an event occurred. The system can design a time series index structure to support efficient time series query and analysis.

[0049] S105, executing anomaly detection algorithms at a preset number of spatial scales according to the spatiotemporal knowledge graph; The system defines a preset number of spatial scales. These spatial scales can include different levels such as single device level, local area level, and overall monitoring area level. Each spatial scale corresponds to a different level of abstraction in the knowledge graph, thus supporting multi-scale anomaly detection.

[0050] For each spatial scale, the system uses a specific anomaly detection algorithm. At the device level, the system can use statistical-based methods, such as control charts or time series analysis, to detect abnormal states of individual devices. At the local area level, the system can use clustering algorithms or graph pattern matching techniques to identify abnormal patterns within the area. At the overall monitoring area level, the system can apply global anomaly detection algorithms, such as graph neural network-based methods, to capture abnormal states of the entire monitoring network.

[0051] When performing anomaly detection, the system makes full use of the semantic information and associations in the spatiotemporal knowledge graph. For example, when analyzing the abnormal state of a certain area, it not only considers the current data of the area, but also refers to the state and historical data of adjacent areas, thereby improving the accuracy and robustness of detection.

[0052] In order to adapt to different types of anomalies, the system can implement a multi-model fusion strategy. For example, it can run rule-based detection, statistics-based detection, and machine learning-based detection at the same time, and then combine the results of each model through weighted voting or other integration methods to obtain the final anomaly judgment.

[0053] The system can also introduce context-aware mechanisms to dynamically adjust the thresholds and strategies for anomaly detection according to different times, locations, and scenarios. For example, during crowded hours, the system may increase its sensitivity to abnormal crowd behavior; during equipment maintenance, it may relax the criteria for judging abnormal equipment status.

[0054] In order to improve the detection efficiency, the system can adopt a hierarchical detection strategy. First, a quick scan is performed at a coarse-grained scale to find potential abnormal areas, and then a more fine-grained in-depth analysis is performed in these areas, thereby improving the processing speed while ensuring the detection quality.

[0055] S106. When it is determined that an abnormal event exists according to the abnormality detection algorithm, a corresponding adjustment method is matched in the preset adjustment scheme according to the type and degree of the abnormal event, and the numerical weights of the corresponding nodes and edges in the adaptive monitoring scene graph are adjusted according to the adjustment method to obtain an adjusted adaptive monitoring scene graph; The system determines the specific type and degree of abnormal events based on the results of the anomaly detection algorithm. The type of abnormality may include equipment failure, environmental abnormalities, abnormal human behavior, etc., while the degree can be expressed by quantitative or qualitative indicators, such as minor, moderate, severe, etc.

[0056] The system maintains a preset adjustment solution library, which contains response strategies for abnormal events of different types and degrees. These strategies involve how to adjust monitoring resource allocation, change data collection frequency, adjust abnormal detection sensitivity, etc. The system matches the most suitable adjustment method in the adjustment solution library according to the currently detected abnormal event.

[0057] The matching process can be implemented using a rule engine or a machine learning model. The rule engine can quickly select the appropriate adjustment plan based on predefined if-then rules, while the machine learning model can automatically recommend the optimal adjustment strategy by learning historical data.

[0058] After determining the adjustment method, the system begins to adjust the numerical weights of the relevant nodes and edges in the adaptive monitoring scene graph. For example, if an anomaly is detected in a certain area, the system may increase the weights of the nodes in that area and its adjacent nodes to increase the monitoring intensity of these areas. At the same time, the weights of some low-risk area nodes may be reduced to optimize resource allocation.

[0059] The adjustment process involves not only changes in the weight of a single node, but also weight adjustments of the relationships (edges) between nodes. For example, if an abnormal correlation is found between two regions, the system may increase the weight of the edge connecting the nodes of the two regions to strengthen the monitoring of this correlation.

[0060] S107. According to the adjusted adaptive monitoring scene graph and the numerical weights corresponding to the nodes, the working parameters of all IoT cameras and other sensors are synchronously adjusted.

[0061] According to the adjusted adaptive monitoring scene graph and the numerical weights corresponding to the nodes, the working parameters of all IoT cameras and other sensors are adjusted synchronously. The working parameters of IoT cameras include angle, focal length and sampling rate, and the working parameters of other sensors include sampling frequency. The system parses the adjusted adaptive monitoring scene graph and extracts the new weight value of each node. These weight values ​​reflect the importance and degree of attention of each monitoring point in the current situation. The system maps these weight values ​​to the specific hardware parameter adjustment range to determine the specific parameter values ​​that need to be adjusted for each device.

[0062] For IoT cameras, the system can adjust the working parameters including angle, focal length and sampling rate. Angle adjustment can be achieved by controlling the pan / tilt to focus the camera on the area that needs the most attention. Focal length adjustment can change the camera's field of view. Areas with higher weights may require a narrower focal length to obtain a clearer image. Sampling rate adjustment can change the frequency of video data acquisition. For high-weight areas, the system may increase the sampling rate to obtain more detailed monitoring data.

[0063] For other types of sensors, such as sound sensors, thermal sensors, and motion sensors, the system mainly adjusts their sampling frequency. Sensors in high-weight areas may be set to a higher sampling frequency to provide more real-time and denser data.

[0064] When adjusting parameters, the system needs to consider the physical limitations and performance boundaries of the hardware device. For example, the angle adjustment range of the camera, the maximum and minimum focal length, the maximum sampling rate of the sensor, etc. The system can build a device capability model to ensure that all adjustments are within the acceptable range of the device.

[0065] In the above embodiment, the monitoring area is abstracted into a dynamic graph structure, and the construction of an adaptive monitoring scene graph is realized. This graph structure enables the system to flexibly represent and manage complex monitoring environments and effectively capture the logical associations between key monitoring points. By fusing multiple sensor data at each node to form a multimodal perception network, the system obtains more comprehensive and richer environmental information, and improves the accuracy and reliability of monitoring. Distributed feature extraction and low-dimensional vector compression technology enable the system to efficiently process a large amount of multimodal data, improving the efficiency of calculation and storage. The spatiotemporal knowledge graph constructed based on the adaptive monitoring scene graph and low-dimensional vector provides rich contextual information for anomaly detection and enhances the system's ability to understand complex events. The application of multi-scale anomaly detection algorithms improves the system's ability to identify abnormal events of different types and scales. The mechanism of dynamically adjusting the monitoring scene graph and sensor parameters according to abnormal events enables the system to adapt to environmental changes in real time, optimize resource allocation, and improve monitoring efficiency. This technical solution improves the integrity and comprehensiveness of monitoring data acquired by IoT camera devices.

[0066] After the above embodiments are executed, the system can continue to build a prediction model. When the IoT camera detects an abnormal event, the system can predict the future trend of the abnormal event based on the prediction model, so as to make a response strategy in advance and reduce the adverse effects of the abnormal event. Figure 2 , a method for predicting abnormal events based on intelligent monitoring of the Internet of Things in an embodiment of the present application is described: See also Figure 2 , which is a flow chart of an abnormal event prediction method based on intelligent monitoring of the Internet of Things in an embodiment of the present application.

[0067] S201. Combine the spatiotemporal knowledge graph and abnormal events to generate a structured context-aware event description including event semantic description, impact scope definition, and potential evolution trend analysis; The system combines spatiotemporal knowledge graphs and abnormal events to generate structured context-aware event descriptions that include event semantic descriptions, impact scope definitions, and potential evolution trend analysis, including: Parse the entities and relationships in the spatiotemporal knowledge graph to obtain semantic descriptions of events; Based on the results of the anomaly detection algorithm, determine the impact scope and severity level of the abnormal event and obtain the impact scope definition; Use the preset causal reasoning model to analyze the potential causes and evolution trends of abnormal events and obtain potential evolution trend analysis; The semantic description of events, the definition of impact scope and the analysis of potential evolution trends are integrated to obtain a structured situation-aware event description.

[0068] In this step, the system first uses the spatiotemporal knowledge graph as the basic data source. The spatiotemporal knowledge graph is a structured knowledge representation method that organizes information such as entities, relationships, and attributes into a graph structure, and contains information in the time and space dimensions. By parsing this knowledge graph, the system can extract semantic information related to the current abnormal event. Specifically, the system can retrieve entities related to the abnormal event from the knowledge graph, such as the location where the event occurred, the people or objects involved, and the relationship between these entities. This information constitutes the semantic description of the event and provides context for subsequent analysis.

[0069] At the same time, the system also uses the results of the anomaly detection algorithm to determine the scope and severity level of the incident. Anomaly detection algorithms can be implemented based on a variety of methods, such as statistical methods, machine learning methods, or deep learning methods. These algorithms identify behaviors or states that deviate from normal patterns by analyzing sensor data, video streams, or other relevant data sources. Based on the results of anomaly detection, the system can determine the spatial scope of the abnormal event, such as the size of the affected area, the number of devices involved, etc. In addition, the system can also grade the severity of the incident according to preset standards, such as minor, moderate, severe, etc. This information together constitutes the definition of the scope of impact.

[0070] In order to analyze the potential causes and possible evolution trends of abnormal events, the system uses a preset causal reasoning model. This model can be built based on Bayesian networks, structural equation models, or other algorithms suitable for causal reasoning. The system inputs current event information and related historical data into the causal reasoning model, and the model outputs possible cause chains and future development paths. These outputs constitute the analysis results of potential evolution trends.

[0071] Finally, the system integrates the semantic description of the event, the definition of the impact scope, and the analysis of potential evolution trends into a structured, context-aware event description. This description can be in a structured data format such as JSON or XML to facilitate subsequent processing and analysis. The structured description includes various aspects of the event, such as event type, time of occurrence, location, involved entities, impact scope, severity, possible causes, predicted trends, etc.

[0072] S202, based on the context-aware event description, using a preset graph representation learning model and a time series prediction model, predict the development path of the abnormal event within a predetermined time window, and generate a corresponding forward-looking response strategy; In this step, the system first uses the preset graph representation learning model to process the structured context-aware event description generated in the previous step. The graph representation learning model is a deep learning model that can effectively process graph structured data, such as graph convolutional neural network (GCN), graph attention network (GAT) or graph neural network (GNN). The system converts the structured event description into a graph structure, where nodes represent entities (such as events, places, people, etc.) and edges represent the relationships between entities. The graph representation learning model learns low-dimensional vector representations of each node through multiple layers of nonlinear transformations. These vectors capture the semantic and structural information of the nodes.

[0073] Next, the system uses the learned graph representation as input and feeds it into the preset time series prediction model. The time series prediction model can be a deep learning model suitable for processing sequence data, such as a long short-term memory network (LSTM), a gated recurrent unit (GRU) or a Transformer. These models can capture the dynamic characteristics of events changing over time. The system sets a predetermined time window (such as the next 24 hours, 48 ​​hours or a week) and allows the time series prediction model to predict the development of events within this window. The prediction results include possible changes in the state of the event, the expansion or reduction of the scope of influence, changes in the entities involved, etc.

[0074] Based on the predicted development path, the system further generates corresponding forward-looking response strategies. This process can be achieved through decision trees, rule engines, or reinforcement learning. The system considers factors such as the predicted event development path, available resources, and response time to generate a series of possible response strategies. Each strategy contains specific action steps, required resources, expected effects, and other information.

[0075] S203, using an online learning algorithm to continuously learn newly generated abnormal events and corresponding forward-looking response strategies, and update the structure of the adaptive monitoring scene graph and the parameters of the abnormality detection algorithm; In this step, the system uses online learning algorithms to continuously learn about new abnormal events and corresponding response strategies. Online learning algorithms allow the system to update the model in real time as new data is received, without having to retrain the entire model. This approach is particularly suitable for dealing with dynamically changing environments and continuously generated new data. The system can use various online learning algorithms, such as online gradient descent, online random forest, or online support vector machine.

[0076] When a new abnormal event occurs, the system first compares it with previous events to identify the unique features of the new event. These features may include new event types, new impact patterns, or new spatiotemporal distributions. The system integrates these new features into the existing knowledge structure to expand the system's cognitive scope of abnormal events. At the same time, the system also records the forward-looking response strategies and their effects for these new events.

[0077] Based on the new learning results, the system updates the structure of the adaptive monitoring scene graph. The adaptive monitoring scene graph is a dynamic graph structure used to represent various entities (such as cameras, sensors, buildings, etc.) in the monitoring environment and their relationships. The system may add new nodes to the graph to represent newly discovered key locations or entities, or adjust the connection weights between nodes to reflect changes in the relationship between entities. This update ensures that the monitoring scene graph can accurately reflect the current state of the monitoring environment.

[0078] At the same time, the system will also update the parameters of the anomaly detection algorithm based on the new learning results. This may involve adjusting thresholds, updating feature weights, or modifying model structures. For example, if the system finds that some new abnormal events cannot be effectively detected by the existing model, it may add new features or adjust the model architecture to improve detection capabilities. This parameter update ensures that the anomaly detection algorithm can adapt to the changing environment and new types of anomalies.

[0079] S204, constructing an abnormal event knowledge base, and storing historical abnormal events and corresponding historical forward-looking response strategies in the abnormal event knowledge base; In this step, the system builds a comprehensive abnormal event knowledge base to store and manage historical abnormal events and their corresponding forward-looking response strategies. This knowledge base is not just a simple data storage system, but a structured knowledge management platform that can support complex query, analysis, and reasoning operations.

[0080] The system first designs the data model of the knowledge base. This model may include multiple entity types, such as events, strategies, locations, times, etc., as well as various relationships between these entities. Each abnormal event record may contain a detailed description of the event, the time and location of occurrence, the scope of impact, the severity, cause analysis, etc. The corresponding forward-looking response strategy record may contain information such as the specific steps of the strategy, the required resources, the execution time, and the expected effect.

[0081] The system uses appropriate database technology to implement this knowledge base. Considering the complexity of the data and the diversity of relationships, the system may choose a graph database (such as Neo4j) or a document database (such as MongoDB) to store this data. These database technologies can effectively handle complex data structures and relationships and support efficient query and retrieval operations.

[0082] During storage, the system preprocesses and normalizes the data. This may include data cleaning (removing noise and outliers), data transformation (converting data into a unified format), and data enhancement (adding additional metadata or tags). The system also creates various indexes to improve the efficiency of subsequent queries and retrieval.

[0083] S205, using a clustering algorithm to classify historical abnormal events to obtain common abnormal event types; In this step, the system uses clustering algorithms to classify historical abnormal events stored in the abnormal event knowledge base to identify and define common abnormal event types. Clustering is an unsupervised learning method that can group similar data points together based on the intrinsic characteristics of the data without pre-defining category labels.

[0084] The system first extracts the features of historical abnormal events from the knowledge base. These features may include the type of event, location, scope of impact, duration, severity, etc. The system converts these features into numerical vectors or other formats suitable for processing by machine learning algorithms. During the feature extraction process, the system may apply some feature engineering techniques, such as feature selection (selecting the most relevant and informative features) and feature scaling (standardizing features of different scales), to improve the clustering effect.

[0085] Next, the system selects an appropriate clustering algorithm to process the data. Common clustering algorithms include K-means, hierarchical clustering, DBSCAN (density-based clustering), etc. Each algorithm has its own characteristics and applicable scenarios. The system may select the most appropriate algorithm based on the characteristics and distribution of the data. For example, if the number of event types is roughly known, K-means can be used; if the data distribution is irregular or noisy, DBSCAN can be considered.

[0086] The system executes the selected clustering algorithm to cluster similar abnormal events together. During the clustering process, the system may need to adjust the algorithm parameters (such as the K value in K-means, or the neighborhood radius in DBSCAN) multiple times to obtain the best clustering results. The system can use some evaluation indicators, such as the Silhouette Coefficient or the Calinski-Harabasz Index, to evaluate the quality of clustering and guide parameter adjustment.

[0087] After clustering is completed, the system analyzes and interprets each cluster. The system can extract the center point or representative sample of each cluster, as well as the distribution characteristics of the data points within the cluster, to define and describe the type of abnormal event represented by this cluster. The system can also assign a semantic label to each cluster for easy subsequent use and understanding.

[0088] S206. For each common abnormal event type, statistically analyze the success rate of the corresponding historical forward-looking response strategy; First, the system needs to define and quantify the metric of "success rate". The success rate may be based on multiple factors, such as the degree of resolution of the incident after the strategy is executed, the degree of reduction in the scope of impact, the time required to return to normal, etc. The system can design a comprehensive scoring mechanism to integrate these factors into a unified success rate metric. This scoring mechanism may use a weighted average or other more complex mathematical models.

[0089] Next, the system extracts the historical records of each common abnormal event type from the abnormal event knowledge base, as well as the corresponding forward-looking response strategies and their execution results. The system needs to deal with possible incomplete or inconsistent data, such as some historical records may be missing execution results, or the result descriptions may not be standardized. The system can apply data cleaning and data interpolation techniques to deal with these problems and ensure that the analysis is based on reliable data.

[0090] The system then performs a statistical analysis of the execution results of each strategy. This includes calculating the average success rate of each strategy, the distribution of the success rate (such as variance, quartiles, etc.), the trend of change over time, etc. The system can apply various statistical methods, such as descriptive statistics, inferential statistics, time series analysis, etc., to fully understand the effect of the strategy.

[0091] S207, determining the historical forward-looking response strategy with the highest success rate as the best response strategy, and establishing a mapping relationship between the best response strategy and the corresponding common abnormal event type; First, the system needs to define the criteria for "best". Although the success rate is the main consideration, the system may also need to consider other factors, such as the implementation cost of the strategy, the time required, resource consumption, etc. The system can design a multi-objective optimization model to take these factors into consideration. For example, you can use the weighted sum method to assign weights to each factor and then calculate the comprehensive score; or use Pareto optimization to find a strategy that performs well on multiple objectives.

[0092] Next, the system applies this optimization model to select the best response strategy for each common abnormal event type. This process may involve complex calculations and comparisons. If there are multiple strategies that perform very similarly, the system may need further decision logic to break the tie, such as considering the stability of the strategy (variance of success rate) or scalability.

[0093] After determining the best strategy, the system needs to establish a mapping relationship between the best response strategy and the corresponding common abnormal event types. This mapping relationship can be implemented as a data structure, such as a hash table or a table in a relational database. Each mapping entry contains the identifier and description of the event type, as well as detailed information about the corresponding best response strategy.

[0094] To improve the flexibility and adaptability of the mapping, the system can save multiple candidate strategies for each event type, rather than just a single optimal strategy. These candidate strategies can be sorted according to the comprehensive score, and in actual applications, the most suitable strategy can be selected according to the specific situation.

[0095] S208. Automatically generate abnormal event handling suggestions based on the mapping relationship.

[0096] Based on the mapping relationship, the system automatically generates suggestions for handling abnormal events, including: Establish a template for abnormal event handling suggestions, which includes event description, possible causes, recommended response strategies and expected effects; When a new abnormal event is detected, the type with the highest similarity to the new abnormal event is searched among the common abnormal event types to obtain the corresponding mapping relationship; According to the mapping relationship, determine the corresponding optimal response strategy; The specific information of the new abnormal event and the obtained best response strategy are filled into the abnormal event handling suggestion template to obtain the abnormal event handling suggestion.

[0097] First, when the system detects a new abnormal event, it needs to find the most similar type to the new event among the common abnormal event types. This matching process can use various similarity calculation methods, such as Euclidean distance, cosine similarity, or more complex machine learning models (such as support vector machines or neural networks). The system may need to consider multiple features of the event, such as event type, location, scope of impact, severity, etc., to conduct a comprehensive similarity assessment.

[0098] After finding the most similar event type, the system obtains the corresponding best response strategy through the mapping relationship. If multiple candidate strategies are stored in the mapping, the system may need further decision logic to select the strategy that best suits the current situation. This may involve considering factors such as current resource conditions and time constraints.

[0099] Next, the system needs to convert the selected response strategy into specific processing suggestions. This process uses a pre-defined abnormal event processing suggestion template. The template contains parts such as event description, possible causes, recommended response strategy, and expected effects. The system fills the template with the specific information of the new abnormal event and the best response strategy obtained to generate a complete processing suggestion.

[0100] When populating the template, the system may need to make some adaptive adjustments. For example, adjusting certain details of the strategy based on the specific characteristics of the new event, or modifying certain recommendations based on the current environmental conditions. This may involve using natural language generation (NLG) technology to ensure that the generated recommendations are both accurate and easy to understand.

[0101] The generated treatment suggestions should not be simple blocks of text, but structured information. The system can organize the suggestions into multiple levels or modules, such as emergency measures, mid-term strategies, long-term prevention, etc., to facilitate operators to quickly understand and implement them.

[0102] In the above embodiment, the development path of abnormal events is predicted by using a preset graph representation learning model and a time series prediction model, so that the system has a prediction function and can predict the development of events in advance, so as to formulate a more effective response strategy. Generate a forward-looking response strategy so that the system can actively respond to potential risks instead of passively react, thereby improving the prevention and disposal capabilities of the system. Through the online learning algorithm, the system can continuously update and optimize its own knowledge base and decision-making model by continuously learning newly generated abnormal events and corresponding forward-looking response strategies, so that the system can become more intelligent and efficient over time, and can better respond to new abnormal situations. Updating the structure of the adaptive monitoring scene graph and the parameters of the anomaly detection algorithm improves the flexibility and accuracy of the system. This dynamic update mechanism enables the system to adapt to changes in the monitoring environment and maintain efficient monitoring performance.

[0103] The following describes the system in the embodiment of the present invention from the perspective of hardware processing. Figure 3 , is a schematic diagram of the physical device structure of an intelligent monitoring system based on the Internet of Things provided in an embodiment of the present application.

[0104] It should be noted that Figure 3 The structure of the system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0105] like Figure 3As shown, the system includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 308 to the random access memory (RAM) 303, such as executing the method in the above embodiment. In RAM 303, various programs and data required for system operation are also stored. CPU 301, ROM 302 and RAM 303 are connected to each other through a bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0106] The following components are connected to the I / O interface 305: an input section 306 including a camera, an infrared sensor, etc.; an output section 307 including a liquid crystal display (LCD) and a speaker, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed so that a computer program read therefrom is installed into the storage section 308 as needed.

[0107] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 309, and / or installed from a removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, various functions defined in the present invention are performed.

[0108] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.

[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Among them, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the system described in the above embodiment; or may exist independently without being assembled into the system. The above storage medium carries one or more computer programs, and when the above one or more computer programs are executed by a processor of a system, the system implements the method provided in the above embodiment.

[0111] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0112] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0113] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0114] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. An intelligent monitoring method based on the Internet of Things, characterized in that: include: Abstracting the monitoring area into a dynamic graph structure to obtain an adaptive monitoring scene graph, wherein the nodes in the adaptive monitoring scene graph represent predetermined key monitoring points, and the edges represent the logical association relationship between each of the nodes; At each of the nodes, video data from at least one IoT camera and data from at least one other sensor are integrated to form a multimodal perception network, wherein the other sensor includes a sound sensor, a thermal sensor, and a motion sensor; Performing distributed feature extraction on the multimodal sensor data of each of the nodes using a preset feature extraction algorithm to obtain data features, and compressing the data features into a low-dimensional vector of a preset dimension, wherein the multimodal sensor data is data collected by the multimodal perception network; Based on the adaptive monitoring scene graph and the low-dimensional vector, construct a spatiotemporal knowledge graph reflecting the current state of the monitoring area; Executing an anomaly detection algorithm at a preset number of spatial scales according to the spatiotemporal knowledge graph; In the case where it is determined according to the anomaly detection algorithm that an abnormal event exists, a corresponding adjustment method is matched in a preset adjustment scheme according to the type and degree of the abnormal event, and the numerical weights of the corresponding nodes and edges in the adaptive monitoring scene graph are adjusted according to the adjustment method to obtain an adjusted adaptive monitoring scene graph; According to the adjusted adaptive monitoring scene graph and the numerical weights corresponding to the nodes, the working parameters of all the IoT cameras and the other sensors are synchronously adjusted, the working parameters of the IoT cameras include angle, focal length and sampling rate, and the working parameters of the other sensors include sampling frequency.

2. The method according to claim 1, characterized in that The fusing of video data of at least one IoT camera and data of at least one other sensor at each node to form a multimodal perception network specifically includes: Preprocessing the video data of the IoT camera of each node to obtain preprocessed video data, wherein the preprocessing includes denoising, image stabilization and resolution standardization; Converting the data of the other sensors into time series data synchronized with the pre-processed video data; A multimodal fusion algorithm is used to integrate the preprocessed video data and the time series data into a unified multimodal feature representation to form a multimodal perception network.

3. The method according to claim 1 or 2, characterized in that: The method of performing distributed feature extraction on the multimodal sensor data of each node by using a preset feature extraction algorithm specifically includes: Extracting spatial feature information of video data in the multimodal sensing data; Extracting time series feature information of time series data in the multimodal sensing data; The spatial feature information and the temporal feature information are integrated into data features.

4. The method according to claim 1, characterized in that: After synchronously adjusting the working parameters of all the IoT cameras and the other sensors according to the adjusted adaptive monitoring scene graph and the numerical weights corresponding to the nodes, the method further includes: Combining the spatiotemporal knowledge graph and the abnormal event, generating a structured situation-aware event description including event semantic description, impact scope definition, and potential evolution trend analysis; Based on the context-aware event description, a preset graph representation learning model and a time series prediction model are used to predict the development path of the abnormal event within a predetermined time window, and generate a corresponding forward-looking response strategy; By using an online learning algorithm, newly generated abnormal events and corresponding forward-looking response strategies are continuously learned, and the structure of the adaptive monitoring scene graph and the parameters of the abnormality detection algorithm are updated.

5. The method according to claim 4, characterized in that The combining of the spatiotemporal knowledge graph and the abnormal event to generate a structured situation-aware event description including event semantic description, impact scope definition and potential evolution trend analysis specifically includes: Parsing entities and relationships in the spatiotemporal knowledge graph to obtain semantic descriptions of events; Based on the results of the anomaly detection algorithm, determine the impact scope and severity level of the abnormal event to obtain an impact scope definition; Analyze the potential causes and evolution trends of the abnormal events using a preset causal reasoning model to obtain a potential evolution trend analysis; The event semantic description, the impact scope definition and the potential evolution trend analysis are integrated to obtain a structured situation-aware event description.

6. The method according to claim 4 or 5, characterized in that: After continuously learning newly generated abnormal events and corresponding forward-looking response strategies using the online learning algorithm and updating the structure of the adaptive monitoring scene graph and the parameters of the abnormality detection algorithm, the method further includes: Building an abnormal event knowledge base, storing historical abnormal events and corresponding historical forward-looking response strategies in the abnormal event knowledge base; Using a clustering algorithm to classify the historical abnormal events, to obtain common abnormal event types, where the common abnormal event types are types corresponding to historical abnormal events whose number is greater than a preset threshold; For each of the common abnormal event types, statistically analyze the success rate of the corresponding historical forward-looking response strategies; Determine the historical forward-looking response strategy with the highest success rate as the best response strategy, and establish a mapping relationship between the best response strategy and the corresponding common abnormal event type; Based on the mapping relationship, abnormal event handling suggestions are automatically generated.

7. The method according to claim 6, characterized in that The automatically generating abnormal event handling suggestions based on the mapping relationship specifically includes: Establishing an abnormal event handling suggestion template, wherein the abnormal event handling suggestion template includes an event description, possible causes, a recommended response strategy, and expected effects; When a new abnormal event is detected, the type with the highest similarity to the new abnormal event is searched among the common abnormal event types to obtain a corresponding mapping relationship; Determine the corresponding optimal response strategy according to the mapping relationship; The specific information of the new abnormal event and the acquired optimal response strategy are filled into the abnormal event processing suggestion template to obtain the abnormal event processing suggestion.

8. An intelligent monitoring system based on the Internet of Things, characterized in that: The system comprises: One or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the system to execute the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a system, the system is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on a system, the system is caused to execute the method according to any one of claims 1 to 7.

Images