Security and protection Internet of Things platform

Abstract

Provided is a security and protection Internet of Things platform, comprising: an edge layer, the edge layer comprising a plurality of edge computing nodes forming a distributed system, the plurality of edge computing nodes being used for jointly completing a training task of a multi-modal security early warning model based on multi-class modal historical data and historical alarm data in an early warning area, a multi-mode safety early warning model is obtained and stored; the plurality of edge computing nodes are also used for collecting at least two types of modal data in the early warning area, wherein the at least two types of modal data comprise static image data and dynamic video data; the at least two types of modal data are input into a multi-modal safety early warning model stored in at least one edge computing node, safety early warning information in an early warning area is obtained, and the safety early warning information comprises a target suspected object, at least one occurrence position of the target suspected object, and at least one occurrence position of the target suspected object. And early warning event information corresponding to the target suspected object.

Description

Technical Field

[0001] This application relates to the field of Internet of Things (IoT) technology, and more particularly to a security IoT platform. Background Technology

[0002] With the acceleration of urbanization and the increasing demand for social security, traditional security systems are gradually revealing their limitations in addressing the complex and ever-changing challenges to public safety. For example, single-modal monitoring devices (such as video cameras and sensors) can only collect specific types of data, making it difficult to comprehensively perceive multi-dimensional risk factors in the environment. Centralized data processing architectures, relying on cloud servers for model training, suffer from high data transmission latency, high bandwidth consumption, and privacy risks, and cannot meet real-time requirements. Furthermore, traditional security systems typically employ a post-event analysis model, lacking the ability to proactively detect and intervene in potential threats, resulting in low efficiency in handling security incidents. Summary of the Invention

[0003] To address the aforementioned issues, this application provides a security IoT platform.

[0004] On one hand, a security IoT platform is provided, comprising: an edge layer, wherein the edge layer includes multiple edge computing nodes constituting a distributed system, the multiple edge computing nodes being used to jointly complete the training task of a multimodal security early warning model based on multimodal historical data and historical alarm data within the early warning area, and to obtain and store the multimodal security early warning model; the multiple edge computing nodes are also used to collect at least two types of modal data within the early warning area, the at least two types of modal data including static image data and dynamic video data; inputting the at least two types of modal data into the multimodal security early warning model stored by at least one edge computing node to obtain security early warning information within the early warning area, the security early warning information including a target suspect object, at least one location of the target suspect object, and early warning event information corresponding to the target suspect object.

[0005] The security IoT platform provided in this application has the following beneficial effects: By training the multimodal security early warning model locally through multiple edge computing nodes at the edge layer, decentralized training can be achieved, shortening the training time and improving training efficiency. Furthermore, using historical data and historical incident data within the early warning area for model training allows the trained multimodal early warning model to better adapt to the environmental characteristics and security needs of the warning area. Simultaneously, by acquiring different modal data from the early warning area, different dimensions of the warning area can be described, and through comprehensive analysis of the multimodal data, security early warning information for the warning area can be obtained. This overcomes the limitations of early warning from a single data source and significantly reduces the false alarm rate. Moreover, by fusing high-resolution detail features of static images with the spatiotemporal continuity information of dynamic videos, the reliability of early warnings in complex scenarios is improved, and continuous tracking and evaluation of suspected targets can be achieved. Attached Figure Description

[0006] To more clearly illustrate the specific embodiments of this application or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0007] Figure 1 This illustration shows a structural diagram of a security IoT platform according to an embodiment of this application; Figure 2 This diagram illustrates the structure of a multimodal security early warning model according to an embodiment of this application. Figure 3 This diagram illustrates the structure of a security data center layer according to an embodiment of this application. Detailed Implementation

[0008] To make the above and other features and advantages of this application clearer, the application is further described below with reference to the accompanying drawings. It should be understood that the specific embodiments given herein are for the purpose of explanation to those skilled in the art, and are exemplary only, not restrictive.

[0009] In the following description, numerous specific details are set forth to provide a thorough understanding of this application. However, it will be apparent to those skilled in the art that the specific details are not required to practice this application. In other instances, well-known steps or operations have not been described in detail to avoid obscuring this application.

[0010] This application provides a security IoT platform, which is a comprehensive IoT platform for realizing intelligent and efficient security monitoring and management. It integrates various modal data acquisition devices or security subsystems, and through data acquisition, transmission, analysis, and processing, provides comprehensive and multi-layered security protection, situational awareness, and management solutions for different locations. In this application, "security IoT platform" can also be referred to as a security IoT system.

[0011] This security IoT platform is built on a PaaS architecture, employing technologies such as containers and microservice frameworks. Leveraging low-latency and high-real-time edge computing, the platform pushes computing, storage, and decision-making capabilities down to edge computing nodes closer to the data source, forming a distributed architecture that collaborates with the edge and cloud.

[0012] Figure 1 This application provides a schematic diagram of the structure of a security IoT platform according to an embodiment of the present application. Figure 1 As shown, the security IoT platform 10 includes at least an edge layer 11. The edge layer 11 may include multiple edge computing nodes 111 constituting a distributed system. These edge computing nodes can communicate with each other via wired or wireless means.

[0013] In one embodiment of this application, multiple edge computing nodes 111 are used to jointly complete the training task of a multimodal security early warning model based on multimodal historical data and historical alarm data within the early warning area, and obtain and store the multimodal security early warning model.

[0014] One embodiment of this application involves multiple edge computing nodes 111 that include components and services for building a distributed system. That is, the components and services of the distributed system are deployed on multiple edge computing nodes 111. The distributed system decomposes complex tasks into multiple subtasks and distributes these subtasks to various edge computing nodes, thereby achieving the execution of complex tasks through task-distributed processing.

[0015] In some embodiments of this application, multiple edge computing nodes 111 are used to decompose the training task of the multimodal security early warning model into multiple sub-tasks, and at least two edge computing nodes perform multiple sub-tasks based on multimodal historical data and historical alarm data within the early warning area to obtain and store the trained multimodal security early warning model.

[0016] In one embodiment of this application, the distributed system decomposes the training task of the multimodal security early warning model into multiple sub-tasks, and determines at least two edge computing nodes with sufficient computing resources among the multiple edge computing nodes. The multiple sub-tasks are sent to the at least two edge computing nodes with sufficient computing resources respectively, and the edge computing nodes with sufficient computing resources execute the multiple sub-tasks to obtain the multimodal security early warning model.

[0017] In another embodiment of this application, the distributed system decomposes the training task of the multimodal security early warning model into multiple sub-tasks, and divides multiple edge computing nodes into two training clusters according to the regional range. Multiple sub-tasks are sent to each training cluster, and at least two edge computing nodes with sufficient computing resources in each training cluster execute multiple sub-tasks.

[0018] Furthermore, the multi-modal historical data and historical alarm data are divided into a first training set and a second training set. The first training set includes a first portion of the multi-modal historical data and historical alarm data, and the second training set includes a second portion of the multi-modal historical data and historical alarm data. The first training set is used to train the multi-modal security early warning model in the first training cluster, and the second training set is used to train the multi-modal security early warning model in the second training cluster.

[0019] Finally, the distributed system aggregates and merges the first multimodal initial security warning model trained by the first training cluster and the second multimodal initial security warning model trained by the second training cluster to obtain a multimodal security warning model, which is then placed into a resource-sharing pool. Each edge computing node obtains a multimodal security warning model from the resource-sharing pool.

[0020] One embodiment of this application involves a warning area that can be a large area, which may include at least one monitoring area with a length not exceeding 20 kilometers. The length of the large area is not less than 100 kilometers.

[0021] One embodiment of this application involves multi-modal historical data, which is multi-modal data over a historical time period. Modal data can refer to data existing in different types. The types of modal data can include, but are not limited to, image modality, video modality, speech modality, environmental modality, and text modality.

[0022] In one embodiment of this application, each edge computing node 111 includes multiple modal data acquisition devices, and each modal data acquisition device can collect one type of modal data. Multiple edge computing nodes 111 constitute a security Internet of Things (IoT) to achieve comprehensive and effective monitoring and early warning of the warning area.

[0023] In one embodiment of this application, each modal data acquisition device can acquire modal data within a monitoring area of ​​the early warning area.

[0024] One embodiment of this application relates to a distributed system including a resource sharing pool. Each edge computing node can obtain the data required for its own tasks from the resource sharing pool.

[0025] In one embodiment of this application, multi-modal historical data and historical alarm data are stored in a resource-sharing pool of a distributed system. When performing training tasks, each edge computing node 111 can obtain the multi-modal historical data and historical alarm data it needs from the resource-sharing pool.

[0026] One embodiment of this application relates to a multimodal security early warning model that can be a multi-task output early warning model constructed based on a deep learning network model.

[0027] In one embodiment of this application, after multiple edge computing nodes 111 jointly complete the training task of a multimodal security early warning system, the trained multimodal security early warning model is deployed on each edge computing node 111. Each edge computing node stores the multimodal security early warning model.

[0028] In some embodiments of this application, multiple edge computing nodes 111 are also used to collect at least two types of modal data within the warning area; input the at least two types of modal data into a multimodal security warning model stored in at least one edge computing node to obtain security warning information within the warning area.

[0029] One embodiment of this application involves at least two types of modal data whose modal types belong to at least two of the multiple modal types corresponding to multiple types of modal historical data.

[0030] In one embodiment of this application, at least two types of modal data may include multiple types of modal data collected within a preset duration in at least one monitoring area of ​​the warning zone. The preset duration can be set according to user needs, for example, a preset duration of 5 minutes.

[0031] In one embodiment of this application, each edge computing node 111 acquires at least two types of modal data from a monitoring area collected by its corresponding multiple modal data acquisition devices.

[0032] It should be noted that different modalities of data can be acquired in different ways. For example, image modal data can be acquired every minute, while video modal data can be acquired continuously.

[0033] In one embodiment of this application, at least two types of modal data may include static image data and dynamic video data. The static image data includes at least one of the following: radar image data, drone image data, and camera image data. The dynamic video data includes at least one of the following: camera video data and agent video data. The agent video data may be video data collected by an agent. The agent may include, but is not limited to, drones, robots, and robot dogs.

[0034] In one embodiment of this application, when the edge computing node 111 has sufficient computing resources, at least two types of modal data are input into a pre-stored multimodal security early warning model to obtain security early warning information within the monitoring area.

[0035] In one embodiment of this application, when the computing resources of the edge computing node 111 are insufficient, at least two types of modal data are sent to the adjacent edge computing nodes with idle computing resources. The adjacent edge computing nodes input the at least two types of modal data into the pre-stored multimodal security early warning model to obtain security early warning information within the monitoring area.

[0036] In one embodiment of this application, the plurality of edge computing nodes 111 store the security warning information of at least one monitored area into a resource sharing pool, thereby obtaining the security warning information within the warning area.

[0037] One embodiment of this application involves security warning information including a target suspect, at least one location of the target suspect, and warning event information corresponding to the target suspect. The warning event information may include, but is not limited to, warning events, the probability of occurrence of warning events, and warning levels. The warning event information may include more than one warning event.

[0038] It should be noted that the safety warning information within the warning area includes safety warning information from at least one monitoring area.

[0039] One embodiment of this application involves at least one suspected object. A suspected object can refer to an object whose characteristics satisfy suspicious features. These objects may include, but are not limited to, communication terminals, crowds, vehicles, intelligent agents, and people. Suspicious features can be obtained through open-source intelligence data and historical crime data. Open-source intelligence data may include, but is not limited to, the distribution of armed forces, the clothing characteristics of armed forces, and social media data. Historical crime data may include, but is not limited to, the areas where historical crime events frequently occur, the source of the events' subjects, their routes of action, and clothing characteristics. Suspicious features may include, but are not limited to, action characteristics, route characteristics, physical characteristics, clothing characteristics, communication terminal number characteristics, and location characteristics.

[0040] One embodiment of this application relates to the location of the suspected target within the warning area. At least one location may include the initial location of appearance, and may also include subsequent locations.

[0041] In one embodiment of this application, when a suspected target appears in the monitoring area corresponding to an edge computing node according to a security warning information, the edge computing node adjacent to this edge computing node will control the modal acquisition device to continuously collect data on the suspected target in order to continuously track the suspected target.

[0042] In some embodiments of this application, at least two types of modal data further include at least one of the following: terminal communication data, electromagnetic wave detection data, and optical fiber monitoring data.

[0043] The terminal communication data is used to extract the user terminal's phone number information, so that the characteristics of the phone number information can be used to determine whether there is a target suspect. The fiber optic monitoring data is used to monitor whether the fiber optic cable is vibrating, so that the vibration state of the fiber optic cable can be used to determine whether there is a target suspect nearby.

[0044] It should be noted that fiber optic monitoring data is applicable to pipeline transportation scenarios where fiber optic cables are present near the pipeline (such as oil pipeline transportation scenarios).

[0045] In the above embodiments, by comprehensively analyzing and processing other modal data, image modal data, and video modal data, more comprehensive and accurate security warning information can be obtained, thereby improving the anti-interference capability of the warning.

[0046] Figure 2 This illustration shows a structural diagram of a multimodal security early warning model provided in an embodiment of this application, as shown below. Figure 2 As shown, the multimodal security early warning model 20 includes a feature extraction layer 21, a feature fusion layer 22, a target detection layer 23, and a target fusion judgment layer 24.

[0047] The feature extraction layer 21 is used to extract features from each type of modality data to obtain feature extraction data for each type.

[0048] In one embodiment of this application, each type of feature extraction data may refer to the feature extraction data corresponding to each type of modality data.

[0049] In one embodiment of this application, the feature extraction layer 21 may include multiple feature extractors. Each feature extractor corresponds to the feature extraction of a type of modality data.

[0050] In one embodiment of this application, the feature extraction layer can select the corresponding feature extractor to extract features according to the type of each modality data, thereby converting each type of modality data from the original data into a high-dimensional feature representation.

[0051] The feature fusion layer 22 is used to perform feature fusion on the feature extraction data of each class to obtain feature fused data of each class.

[0052] One embodiment of this application involves a feature fusion layer comprising multiple feature fusion networks. Each feature fusion network corresponds to the feature fusion of a type of modality data. Feature fusion refers to the fusion of multiple features of the same modality data.

[0053] In one embodiment of this application, the feature fusion layer can select the corresponding feature fusion network for feature fusion based on the modality type of each type of feature fusion data, thereby obtaining a more comprehensive feature representation for each type of modality data.

[0054] The target detection layer 23 is used to perform target detection on at least the suspicious objects in each type of modal data and the location of each suspicious object based on the feature fusion data of each type, so as to obtain the target detection results of each type of modal data.

[0055] In one embodiment of this application, the target detection result includes at least a suspicious object and at least one location where it appears. The target detection layer 23 can independently perform target detection on each type of modal data based on each type of fused feature, obtain suspicious objects for each type of modal data, and determine at least one location where the suspicious object appears by detecting the collection location of the modal data of the suspicious object.

[0056] In one embodiment of this application, the target detection layer 23 can also perform target detection on suspicious communication terminals and their locations in the terminal communication data based on the feature fusion data corresponding to the terminal communication data, and obtain the target detection result corresponding to the terminal communication data.

[0057] The target fusion judgment layer 24 is used to fuse and judge the target detection results of at least two types of modal data to obtain security warning information.

[0058] In one embodiment of this application, the target fusion judgment layer 24 can integrate target detection results from multiple modalities and generate security warning information through spatiotemporal correlation and logical reasoning. Specifically, the target fusion judgment layer 24 correlates target detection results from at least two modalities based on at least two types of data. For example, it correlates target detection results from static image data and dynamic video data of the same monitored area at 9:00 AM.

[0059] Furthermore, the target fusion judgment layer 24 can dynamically adjust the fusion weights of the target detection results based on the reliability of task detection for each type of modality data. For example, for a person detection task, the video modality is more reliable than the image modality, and the fusion weight of the target detection results in the video modality is greater than that in the image modality. For static object detection, the image modality is more reliable than the video modality, and the fusion weight of the target detection results in the video modality is less than that in the image modality.

[0060] The target fusion judgment layer 24 can also use fusion weights to fuse the detection results of each target to obtain the suspected target and at least one location where it appears.

[0061] One embodiment of this application involves weighted average fusion, which means performing weighted average fusion on parameters of the same type in the detection results of various targets.

[0062] Furthermore, the target fusion judgment layer 24 can determine the early warning event information corresponding to the suspected target based on the rule engine or a decision network built based on machine learning. The rule engine consists of logical judgment rules set based on historical police data and expert experience.

[0063] For example, the final target detection result is "There is a suspected target, and the location is a key protected area in the adjacent warning area". According to the rule engine, the warning event is determined to be that the suspected target may damage the key protected area, and the warning level is a high warning level.

[0064] It should be noted that the number of suspected targets involved in the entire text may be more than one.

[0065] In some of the above embodiments, feature extraction layers extract feature data from various modalities, providing a foundation for subsequent feature fusion. Feature fusion layers fuse multiple features within the same modality, yielding a more comprehensive feature representation. Object detection layers perform independent object detection on various modalities, and object fusion judgment layers fuse multiple object detection results to infer security warning information. This model overcomes the limitations of single-modal data sources through layered decoupling and cross-modal fusion. Furthermore, by comprehensively analyzing and judging multimodal data, it can obtain highly reliable security warning information, thereby improving the accuracy and reliability of warnings.

[0066] In some embodiments of this application, the warning event information includes warning routes and potentially dangerous areas associated with suspicious objects.

[0067] The target fusion judgment layer 24 is specifically used to fuse and judge the detection results of at least two types of targets to obtain the suspected target objects and at least two locations where each suspected target object appears; and, for each suspected target object, combined with the geographical information of its at least two locations, to obtain its corresponding warning route and potential danger area.

[0068] One embodiment of this application relates to a warning route that can refer to the movement route of a suspected target from its current location to a potentially dangerous area. The potentially dangerous area can refer to the final target location inferred from the direction of the suspected target's movement.

[0069] In one embodiment of this application, the target fusion judgment layer 24 can match each occurrence location with a geographic information system to obtain the geographic information of the occurrence location, and determine the current movement direction and current movement route of the suspicious person based on the geographic information of multiple occurrence locations. Furthermore, it judges the suspiciousness of the target suspect's actions based on the current movement route. Simultaneously, it filters potential danger areas corresponding to the target suspect based on the target suspect's current movement direction, and obtains a warning route through a path planning algorithm.

[0070] Potentially dangerous areas can include restricted areas, areas with frequent police incidents, and key protected areas marked on a geographic information system.

[0071] In the above embodiments, by predicting the warning routes and potential danger areas related to the target suspect, resources can be deployed in advance to avoid blind responses, focus on potential danger areas and warning routes, reduce operation and maintenance costs, thereby reducing the probability of warning events and significantly improving the reliability of regional security protection.

[0072] Furthermore, the target fusion judgment layer 24 can also determine whether the target suspect's movement route is a normal road or an abnormal road based on the target suspect's initial location. A normal road refers to a road that can be found through road network data. An abnormal road refers to a road that cannot be found through road network data.

[0073] If the target suspect's movement route is a normal road, the source area of ​​the target suspect is traced based on road network data. The suspicion level of the target suspect's actions is then judged based on the regional risk level of the source area, and the probability of the warning event occurring is determined based on the suspicion level of the actions.

[0074] If the target suspect's movement route is an abnormal road, the movement route and the potential danger area it will eventually reach are predicted according to the movement rules for abnormal roads. The suspicion level of the target suspect's actions is then determined based on the regional risk level of the potential danger area reached, and the probability of a warning event is determined based on the suspicion level. The movement rule for abnormal roads is to avoid normal roads.

[0075] In some embodiments of this application, multiple edge computing nodes 111 are used to update a multimodal security early warning model based on at least two types of modal data.

[0076] In one embodiment of this application, at least two types of modal data are added to multi-modal historical data, and multiple edge computing nodes 111 use the added multi-modal historical dataset to perform training tasks.

[0077] In another embodiment of this application, each edge computing node uses at least two types of modal data collected by itself to perform local updates, then aggregates the updated model parameters of each node on an edge computing node, and distributes the aggregated model parameters to each edge computing node to obtain the updated multimodal security early warning model.

[0078] In this way, the matching degree between the model and the early warning area environment can be improved, and by continuously updating the model, edge computing nodes can learn the characteristics and behavioral patterns of new threats, adjust early warning strategies in a timely manner, and improve the platform's security and defense capabilities.

[0079] In some embodiments of this application, the plurality of edge computing nodes are used to acquire at least two types of modal data within the warning area in response to the occurrence of an abnormal event.

[0080] One embodiment of this application relates to an abnormal event triggered by the detection of anomalies based on multi-sensor data set within the boundary of a warning area. The multi-sensor data may include sound sensor data and infrared sensor data. Abnormal situations may include, but are not limited to, the presence of a crowd exceeding a preset number, the presence of a suspicious object exhibiting suspicious voice characteristics, or a group of vehicles exceeding a preset number.

[0081] In one embodiment of this application, a multi-sensor is set in the monitoring area corresponding to at least one edge computing node. When the edge computing node detects an abnormal situation based on the multi-sensor data, it activates multiple modal acquisition devices to collect data.

[0082] In the above example, the acquisition of modal data is triggered by abnormal events, thereby dynamically adjusting the sampling frequency of multimodal data and improving the response speed and decision-making accuracy of security warnings.

[0083] In some embodiments, the multiple edge computing nodes 111 can also be used to decide on corresponding processing measures based on the warning event information in the warning area.

[0084] In one embodiment of this application, the processing measures include an agent scheduling strategy and / or an on-site handling strategy. The agent scheduling strategy includes the number, type, movement route, and actions to be performed by the agents. The on-site handling strategy includes on-site personnel and handling methods.

[0085] In this way, the intelligent agent scheduling strategy and / or on-site handling strategy can be dynamically adjusted based on the early warning event information, thereby improving the resource optimization rate, enabling more accurate handling of early warning events, and reducing resource waste.

[0086] In some embodiments of this application, such as Figure 1 As shown, the security IoT platform also includes a security data center layer 12, which is located above the edge layer.

[0087] In one embodiment of this application, the security data center layer 12 includes an asset management database. This asset management database is used to configure node information and alarm configuration information for the edge layer.

[0088] An embodiment of this application involves alarm configuration information that includes at least alarm conditions and alarm rules. The alarm rules define the alarm conditions corresponding to different alarm types. Alarm information may include, but is not limited to, alarm ID, alarm type, alarm time, alarm location, alarm level (e.g., emergency, important, general), relevant data indicators, and processing suggestions.

[0089] One embodiment of this application involves node information used to configure edge computing nodes enabled in the edge layer 11. Node information may include, but is not limited to, the number of nodes, unique node identifiers, and node locations.

[0090] Thus, by setting up an asset management database at the security data center layer, the node information and alarm configuration information of the edge layer are centrally configured and managed, avoiding the cumbersome operations of configuring in multiple scattered locations and greatly improving the management efficiency of the security IoT platform. Furthermore, this allows the security IoT platform to flexibly configure the edge computing nodes enabled at the edge layer 11 according to actual security needs. Whether adding new nodes to expand security coverage or adjusting the location of existing nodes to optimize the monitoring layout, this can be achieved by simply modifying the node information in the asset management database, providing strong support for the flexible deployment and dynamic adjustment of the security IoT platform.

[0091] In some embodiments of this application, the security data center layer 12 is used to obtain security warning information from the edge layer 11; when the security warning information meets the alarm conditions, alarm information and alarm list are generated according to alarm rules.

[0092] In one embodiment of this application, the edge layer 11 sends the generated security warning information to the security data center layer 12 according to a preset time interval or a time-triggered mechanism. The security data center layer 12 reads the pre-configured alarm rules from the asset management database, matches and judges the security warning information with the alarm rules, and determines whether the security warning information meets the alarm conditions.

[0093] If the safety warning information meets the alarm conditions, generate alarm information according to the alarm rules, and populate the alarm information into the preset alarm list to generate an alarm list corresponding to this safety warning information, or add the alarm information to an existing alarm list.

[0094] In this way, we can filter out the truly urgent warning events from the massive amount of security warning information, thereby improving the accuracy of security warnings.

[0095] In some embodiments of this application, the asset management database is also used to configure data visualization information. This data visualization information may include data display rules for different data formats and corresponding visualization components. Visualization components are the basic elements for building visualization pages, such as bar charts, line charts, pie charts, maps, and tables.

[0096] In one embodiment of this application, the security data center layer 12 is used to determine visualization components based on data visualization information, and to construct a visualization page about the warning area based on at least two types of modal data and the visualization components.

[0097] In one embodiment of this application, the secure data center layer determines the corresponding data display rules and matching visualization components from the data visualization information based on the type of modal data.

[0098] For example, when displaying temperature change data at different times within a warning area, the data visualization information can display temperature data as a line chart. The security data center layer will choose the line chart as a visualization component to clearly present the temperature change trend over time.

[0099] One embodiment of this application relates to a visualization page for warning areas that presents data of different modalities in an intuitive and easy-to-understand manner, enabling users to quickly understand the security situation within the warning area.

[0100] In some embodiments of this application, the security data center layer 12 is used to obtain at least two types of modal data within the warning area from the edge layer, and to perform statistical analysis on the at least two types of modal data to obtain statistical analysis information for each time period within the warning area.

[0101] One embodiment of this application involves statistical analysis information used to provide detailed data support for security management personnel, helping them understand the changing trends and patterns of security conditions within the warning area, so that security management personnel can adjust security strategies.

[0102] For example, if a certain type of security incident is found to occur frequently during a certain period, the monitoring intensity during that period can be strengthened or the alarm threshold can be adjusted to improve the effectiveness and adaptability of the security IoT platform.

[0103] This application also provides a security data center layer. Figure 3 This application provides a schematic diagram of the structure of a security data center layer according to an embodiment of the present application. Figure 3 As shown, the security data center layer 12 includes an IaaS layer 121, a platform layer 122, an application layer 123, and a presentation layer 124.

[0104] Among them, the presentation layer 124 supports the rapid development and deployment of microservices, enabling the display of visual pages about the warning areas on multiple terminals, including a large visualization screen.

[0105] Application layer 123 provides software services. These services primarily offer research and design users access to the software and related functionalities. The application layer includes an asset management database.

[0106] Platform layer 122 enables access to, control and management of software and hardware resources and development tools, and provides necessary interfaces and support for storage, computing, and tool resources.

[0107] The IaaS layer 121 provides computing and storage capabilities as well as network infrastructure, and interacts with the edge layer for data exchange.

[0108] Thus, this security data center layer can support new API technologies, big data technologies, machine learning, secondary development, and other technologies, and has good robustness, ease of use, maintainability, and scalability.

[0109] It should be understood that the specific features, operations, and details described herein with respect to the methods of this application can also be similarly applied to the apparatus and system of this application, or vice versa. Furthermore, each step of the methods of this application described above can be performed by a corresponding component or unit of the apparatus or system of this application.

[0110] It should be understood that the technical features described above can be combined arbitrarily. Although not all possible combinations of these technical features are described, any combination of these technical features should be considered to be covered by this specification, provided that such combination does not contain contradictions.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A security IoT platform, characterized in that, include: Edge layer, in which, The edge layer includes multiple edge computing nodes that constitute a distributed system. These multiple edge computing nodes are used to jointly complete the training task of a multimodal security early warning model based on multimodal historical data and historical alarm data within the early warning area, and obtain and store the multimodal security early warning model. The multiple edge computing nodes are also used to collect at least two types of modal data within the warning area, including static image data and dynamic video data; The at least two types of modal data are input into a multimodal security early warning model stored in at least one edge computing node to obtain security early warning information within the early warning area. The security early warning information includes a target suspect, at least one location of the target suspect, and early warning event information corresponding to the target suspect.

2. The security IoT platform according to claim 1, characterized in that, The multiple edge computing nodes are used to decompose the training task of the multimodal security early warning model into multiple sub-tasks, and at least two edge computing nodes execute the multiple sub-tasks based on multimodal historical data and historical alarm data within the early warning area to obtain and store the trained multimodal security early warning model.

3. The security IoT platform according to claim 1 or 2, characterized in that, The plurality of edge computing nodes are used to update the multimodal security early warning model based on the at least two types of modal data.

4. The security IoT platform according to claim 1, characterized in that, The multimodal security early warning model includes a feature extraction layer, a feature fusion layer, a target detection layer, and a target fusion judgment layer; the feature extraction layer is used to extract features from each type of modal data to obtain feature extraction data for each type. The feature fusion layer is used to perform feature fusion on each type of feature extracted data to obtain feature fused data for each type. The target detection layer is used to detect suspicious objects and the location of each suspicious object in each type of modal data based on the feature fusion data of each type, so as to obtain the target detection result of each type of modal data; The target fusion judgment layer is used to fuse and judge the target detection results of the at least two types of modal data to obtain security warning information, wherein the target suspect includes at least one of the suspicious objects.

5. The security IoT platform according to claim 1, characterized in that, The at least two types of modal data also include at least one of the following: terminal communication data, electromagnetic wave detection data, and optical fiber monitoring data.

6. The secure Internet of Things platform according to claim 1, characterized in that, The multiple edge computing nodes are used to acquire at least two types of modal data within the warning area in response to the occurrence of abnormal events.

7. The security IoT platform according to any one of claims 1-6, characterized in that, It also includes a security data center layer, which includes an asset management database. The asset management database is used to configure the node information and alarm configuration information of the edge layer. The alarm configuration information includes at least alarm conditions and alarm rules. The node information is used to configure the edge computing nodes enabled by the edge layer.

8. The security IoT platform according to claim 7, characterized in that, The security data center layer is used to obtain the security warning information from the edge layer; when the security warning information meets the alarm conditions, alarm information and alarm list are generated according to the alarm rules.

9. The security IoT platform according to claim 7, characterized in that, The security data center layer is used to acquire at least two types of modal data within the warning area from the edge layer, and to perform statistical analysis on the at least two types of modal data to obtain statistical analysis information for each time period within the warning area.

10. The security IoT platform according to claim 7, characterized in that, The asset management database is also used to configure data visualization information, and the security data center layer is used to determine visualization components based on the data visualization information, and to construct a visualization page about the warning area based on the at least two types of modal data and the visualization components.

Images