Multi-modal data aggregation auxiliary warning system and method based on power gap system

Abstract

The invention relates to the technical field of power system automation and edge computing, and discloses a multi-modal data aggregation auxiliary warning system and method based on a power gap system. Comprising a distributed virtualized networking and resource pooling module, a multi-modal data sharing and subscribing module, a distributed collaborative AI analysis and decision module and a collaborative event aggregation and reporting module. The distributed virtualization networking and resource pooling module abstracts hardware resources into resource descriptors and constructs a distributed global resource pool; the multi-mode data sharing and subscribing module realizes cross-node data sharing based on resource identifiers, the warning accuracy is improved through multi-view fusion, the system robustness and the resource utilization efficiency are ensured through AI task dynamic scheduling and distributed decision making, and network and cloud load is reduced through edge side semantic aggregation. According to the method, the problems of data islands, high false alarm rate, non-uniform computing power, decision conflicts and the like in a distributed edge computing scene are effectively solved.

Description

Technical Field

[0001] This invention relates to the field of power system automation and edge computing technology, specifically a multimodal data aggregation-assisted warning system based on the HarmonyOS power system. Background Technology

[0002] With the development of smart grid and Internet of Things (IoT) technologies, a large number of intelligent electronic devices and various types of sensors have been deployed in power systems. These devices generate massive amounts of multimodal data at the network edge, providing a data foundation for system status monitoring and fault early warning.

[0003] In existing technologies, a common approach is to transmit the data collected by these edge devices to a centralized cloud platform or main station for unified analysis and decision-making. However, this centralized architecture is highly dependent on network communication, which not only results in significant data transmission latency and consumes a large amount of backbone network bandwidth, but also carries the risk of a single point of failure in the central node causing the entire system to crash.

[0004] To address these issues, edge computing architecture has been introduced into power systems. Its core idea is to push data processing and analysis capabilities down to edge nodes closer to the data source. However, existing edge computing solutions still have shortcomings in practical applications. Edge nodes typically operate independently, forming data and resource silos. The heterogeneous hardware resources and data formats of each node make cross-node data sharing and collaboration difficult, hindering comprehensive analysis based on multi-dimensional data. Therefore, decisions relying solely on data from a single node or a single modality are prone to misjudgments, leading to a high false alarm rate. Furthermore, the uneven distribution of computing power among edge nodes means that complex AI analysis tasks may create performance bottlenecks on resource-constrained nodes. The lack of effective collaboration mechanisms can also lead to multiple nodes responding simultaneously to the same event, resulting in duplicate alarms or conflicting control commands. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a multimodal data aggregation-assisted alarm system based on the HarmonyOS power system, which solves the problems of difficult resource integration, unreliable data fusion analysis, uneven computing load, and easy conflict in collaborative decision-making in distributed edge environments.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a multimodal data aggregation-assisted alarm system based on the HarmonyOS power system, comprising:

[0007] The distributed virtualization networking and resource pooling module is used to abstract the hardware resources of the multiple nodes, such as image sensors, environmental sensors, and AI acceleration units, into resource descriptors containing globally unique resource identifiers. Based on this, the module constructs a global resource pool shared by all nodes and existing in the form of distributed data replicas, realizing a unified view of heterogeneous resources across the entire network.

[0008] The multimodal data sharing and subscription module encapsulates data collected from hardware resources by each node into standardized data messages containing source resource identifiers. Using these source resource identifiers as the subject of message publishing, this module constructs a transparent cross-node data center through a publish / subscribe mechanism, enabling on-demand and efficient data sharing.

[0009] The distributed collaborative AI analysis and decision-making module utilizes data provided by the cross-node data bus to perform collaborative intelligent analysis. One function of this module is to perform multi-view data fusion analysis by collecting multiple initial detection results under the same physical location label from one or more fusion nodes and comprehensively evaluating these results. The evaluation process includes weighted fusion of the confidence levels of each detection result to calculate a combined confidence level. :

[0010] ;

[0011] in, For the first The confidence level of each test result The weights are set based on the parameters in the resource descriptor. Simultaneously, this module utilizes pre-stored camera calibration parameters in the resource descriptor to perform spatial consistency checks on multiple detection results to confirm that they point to the same physical target. This check result is quantified as a spatial consistency factor. The final fusion confidence level Determined by the following formula:

[0012] ;

[0013] Another function of this module is to dynamically schedule AI computing tasks. When the AI ​​computing resource load rate of a source node exceeds a preset migration trigger threshold, this module calculates a migration cost function based on the resource load rate of the candidate target node and the network latency between the target node and the source node. :

[0014] ;

[0015] in, It is the normalized load rate of the candidate nodes. It is the normalized network latency. and The weighting factor is used. This module selects the node with the lowest cost as the target node for task migration. In addition, this module runs a distributed rule engine that, based on preset rules, performs collaborative decision-making for cross-node events that meet the triggering conditions.

[0016] The collaborative event aggregation and reporting module ensures the uniqueness and completeness of event processing after the collaborative decision is triggered. This module first uses a distributed election mechanism to determine a unique event-responsible node from among multiple nodes that may trigger the decision simultaneously, based on an election criterion that integrates node role priority, real-time resource status, and a fixed identifier. This responsible node is responsible for aggregating the complete event context related to the collaborative decision, including snapshots of the conditions that triggered the decision, result reports of all executed actions, and related raw sensing data, and finally generating a structured event report for reporting.

[0017] A second aspect of this invention provides a multimodal data aggregation-assisted early warning method based on the HarmonyOS power system, the method comprising the following steps:

[0018] S1: Abstract the hardware resources of multiple nodes in a distributed network into resource descriptors containing resource identifiers, and build a distributed global resource pool shared by all nodes.

[0019] S2: Encapsulate the data collected by the hardware resources into standardized data messages, and use the resource identifier as the topic to achieve cross-node data sharing through a publish / subscribe mechanism;

[0020] S3: Based on the cross-node data sharing, perform multi-perspective data fusion analysis, dynamically schedule AI computing tasks, and run a distributed rule engine for collaborative decision-making;

[0021] S4: After the collaborative decision is triggered, a unique event responsible node is elected, the event context related to the collaborative decision is aggregated, and a structured event report is generated and reported.

[0022] This invention provides a multimodal data aggregation-assisted alarm system based on the HarmonyOS power system. It has the following beneficial effects:

[0023] 1. This invention utilizes multi-view data fusion analysis to perform confidence fusion and spatial consistency verification on initial detection results from different nodes. This method leverages cross-validation of multi-source information to effectively suppress misjudgments caused by limitations in perspective or environmental interference from a single data source, thereby improving the accuracy and reliability of alarm decisions.

[0024] 2. This invention utilizes a dynamic scheduling mechanism for AI computing tasks to migrate tasks to the optimal node based on the node's real-time load and network conditions, achieving load balancing of computing resources. Simultaneously, by performing semantic aggregation and structured reporting of events at the edge, it significantly reduces the amount of data transmitted to the cloud, lowers network bandwidth consumption, and improves the overall utilization efficiency of system resources.

[0025] 3. This invention, through a distributed rule engine and an event-responsible node election mechanism, enables decision-making logic to run without a central node, avoiding the risk of single point of failure. The uniqueness of the event-responsible node ensures that the aggregation and reporting tasks for the same event are not executed repeatedly, effectively avoiding duplicate alarms and control conflicts in a distributed environment, and guaranteeing the consistency and robustness of the system's collaborative operation. Attached Figure Description

[0026] Figure 1 This is a system architecture diagram of the present invention;

[0027] Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Example:

[0030] Please see the appendix Figure 1 This invention provides a multimodal data aggregation-assisted early warning system based on the HarmonyOS power system, comprising:

[0031] The module includes: 10 distributed virtualization networking and resource pooling; 20 multimodal data sharing and subscription; 30 distributed collaborative AI analysis and decision-making; and 40 collaborative event aggregation and reporting.

[0032] After the system of this invention is started, the distributed virtualization networking and resource pooling module 10 deployed on each edge computing node in the substation performs self-discovery and security authentication to build a decentralized trusted communication network. After authentication, the module 10 registers the locally connected hardware resources of each node, such as sensors, actuators and AI computing units, as services, and aggregates them to form a globally shared hardware resource pool.

[0033] Based on the aforementioned network and resource pool, each node encapsulates its collected local data into standardized messages through the multimodal data sharing and subscription module 20 and publishes them to the distributed soft bus. Other nodes within the network can subscribe to the required data streams through this module 20 according to their business needs, thereby establishing a transparent data sharing path across nodes.

[0034] The distributed collaborative AI analysis and decision-making module 30 utilizes data provided by the multimodal data sharing and subscription module 20. This module 30 can invoke multiple data sources from different nodes for multi-perspective fusion analysis, or, when the AI ​​computing task load is too high, dynamically schedule tasks to other lightly loaded nodes based on the status information in the hardware resource pool. Simultaneously, the distributed rule engine built into this module 30 makes decisions on cross-node events that meet the trigger conditions based on preset linkage logic.

[0035] After a collaborative decision-making and control process is completed, the collaborative event aggregation and reporting module 40 is activated. This module 40 is responsible for collecting contextual information from all relevant nodes involved in the event, including raw sensing data, intermediate analysis results, and final execution actions, and aggregating them into a unique, structured, comprehensive event record. This record is then uniformly reported to the upper-level monitoring system.

[0036] Reference Appendix Figure 2 This invention provides a multimodal data aggregation-assisted early warning method based on the HarmonyOS power system, which may include the following steps:

[0037] S1, each edge computing node performs self-discovery and security authentication, builds a distributed virtual network, and registers the local hardware resources of each node to the network to form a globally shared hardware resource pool;

[0038] S2: Each node collects local multimodal data and publishes it to the distributed soft bus. At the same time, it subscribes to the data streams of one or more nodes according to business needs to achieve cross-node data sharing.

[0039] S3, based on shared data flow, performs distributed collaborative analysis, including calling data from multiple nodes for multi-perspective fusion analysis, or dynamically scheduling AI computing tasks when the computing load on a specific node is too high;

[0040] S4 monitors the data and analysis results on the distributed soft bus in real time. When the preset cross-node linkage rules are met, it directly triggers the action of one or more executors on the corresponding nodes.

[0041] S5: After the collaborative action is completed, the responsible node of the event collects and aggregates the context information of this collaborative task, generates a single, structured comprehensive event record, and reports it to the upper-level monitoring system.

[0042] One specific function of the distributed virtualization networking and resource pooling module 10 is to perform node self-discovery and trusted network construction after the multimodal edge computing aggregation board in the system is started.

[0043] Each node enters network discovery state and periodically sends node announcement messages to a preset LAN multicast address. These announcement messages are used to demonstrate their presence and capabilities to other devices in the network. A specific announcement message... It can be defined as:

[0044] ;

[0045] in:

[0046] This is a globally unique identifier for the current node, such as its hardware MAC address or serial number.

[0047] This is a service type identifier for nodes, used to indicate their hardware configuration or role in the system.

[0048] This is the timestamp when the message was generated, used to prevent message replay attacks.

[0049] It is a pre-defined hash function.

[0050] This is the node's private key.

[0051] For digital signature functions, use the node's private key. The hash value of the message content is signed to ensure the integrity of the message and the non-repudiation of its origin.

[0052] When a node Received from another node Announcement Then, it first uses nodes. Corresponding public key For signature Perform verification. Once verification is successful, the node... It is believed that a legitimate potential peer node has been found, and the node is... Initiate a two-way authentication request to establish a trusted connection.

[0053] The authentication process can employ a challenge-response mechanism. Node Generate a random number as the challenge code. and send it to the node. .node After receiving the challenge code, use its private key. Alternatively, the challenge code can be encrypted or signed using a pre-shared key between the two parties to generate a response. and return to For example, a response can be generated as follows:

[0054] ;

[0055] in:

[0056] This is a symmetric encryption function.

[0057] For nodes and Pre-shared symmetric key.

[0058] The response includes the requester. The ID can further enhance security.

[0059] node Upon receiving a response Then, use the same shared key. Decrypt the code and combine the decrypted content with the challenge code you sent. It compares its own ID with the node's own ID. If they match, then the node... Confirmation Node A node is considered a trusted node. Conversely, a node is considered an untrusted node. Also, treat the nodes in the same way. Perform authentication.

[0060] After both authentications are successful, the two nodes will exchange identifiers. Information such as communication addresses is added to each node's locally maintained list of trusted nodes. This list forms the basis of distributed soft bus communication. Simultaneously, an encrypted communication session is established between nodes, and all subsequent data interactions take place within this encrypted channel to ensure confidentiality. This trusted network is dynamic; nodes maintain connections through periodic heartbeat messages. If a node's heartbeat is not received within a preset time, it is removed from the trusted list.

[0061] The hash function, digital signature algorithm, or symmetric encryption algorithm used in the authentication process can be selected by those skilled in the art based on the system security level requirements.

[0062] The second specific function of the distributed virtualization networking and resource pooling module 10 is that, after a node successfully builds a trusted network, the node abstracts and registers its locally connected hardware resources as services. This process is achieved through the following steps:

[0063] When a node starts up, its operating system-level device drivers or hardware abstraction layer automatically detect and identify all physically connected hardware devices. These devices include, but are not limited to, image sensors, environmental sensors, industrial bus interfaces, and actuators. Simultaneously, the node identifies its internally integrated computing resources, particularly its AI acceleration unit and its computing power.

[0064] For each identified hardware device or internal resource, the node encapsulates it into a standardized service. The core of this service encapsulation lies in defining a resource descriptor. Resource descriptor It contains information such as the resource's type, unique identifier, capabilities, access interface, and current status. A specific resource descriptor. It can be defined as:

[0065] ;

[0066] in:

[0067] It is a globally unique identifier for the resource throughout the entire distributed network. This identifier is determined by the node's... and the local identifier of resources within the node Combination, for example _ .

[0068] Indicates the type of resource.

[0069] Describe the capabilities and characteristics of the resource. For example, for the Camera type, its capabilities may include resolution, frame rate, and whether PTZ is supported; for the NPU type, its capabilities may include peak computing power and supported AI model formats.

[0070] Describes the unified interface for accessing this resource. For local resources, this is typically a local service call interface; for other nodes in a distributed system, it is a logical address that can be routed via a soft bus.

[0071] Describes the current state of the resource. For a Camera, the state may include Streaming or Idle; for an NPU, the state may include the current load rate, remaining computing power, etc.

[0072] It is a unique identifier for the node that owns the resource.

[0073] Nodes use a distributed soft bus to distribute all their locally encapsulated resource descriptors. The resource descriptors are published globally to complete the registration process. This registration process is idempotent, meaning that repeated registrations will not produce side effects. Other nodes or upper-layer applications can discover and obtain these resource descriptors by querying the distributed soft bus. Through resource descriptors, upper-layer applications or collaborative modules do not need to know the physical node where the resource is located or the underlying implementation details, and can call and access it through a unified interface, realizing the abstraction and service-oriented approach to hardware resources.

[0074] The third specific function of the distributed virtualization networking and resource pooling module 10 is to build and dynamically maintain a globally shared hardware resource pool after each node completes the service registration of its resources. This resource pool aggregates the resource descriptors registered by all nodes to form a distributed, logically unified view of hardware capabilities.

[0075] The construction of the global resource pool. Logically, it refers to all resource descriptors registered by all active nodes in the network. A set of.

[0076] ;

[0077] in, This resource pool is the collection of all active nodes in the current trusted network. It is not stored on a central server but exists as distributed data replicas across each node. Each node maintains a local view of the resource pool that is synchronized with other nodes. This distributed data synchronization is achieved through a synchronization mechanism based on the Gossip protocol. When a node registers its resources, it encapsulates its resource descriptor information into a synchronization message and sends it to a predetermined number of peer nodes. Nodes receiving the message update their local resource pool view and continue forwarding the information to their own peer nodes until the resource information is disseminated throughout the network.

[0078] Dynamic maintenance of the resource pool. This maintenance process includes responding to node joining, node leaving, and resource state changes. When a new node completes authentication and joins the trusted network, it executes the aforementioned resource registration process and broadcasts its resource descriptor to the entire network through a synchronization mechanism. Other nodes in the network, upon receiving the new resource descriptor, add it to their respective local resource pool views, thus completing the dynamic addition of the new resource.

[0079] When a node goes offline normally as scheduled, it sends a resource deregistration message to the network, which contains unique identifiers for all the resources it owns. Upon receiving this message, other nodes remove the corresponding resource descriptors from their local resource pool views. If a node goes offline abnormally due to a fault, its status will be detected through the heartbeat mechanism between nodes. Once a node is determined to be offline, other nodes in the network will remove all resource descriptors it owns from their respective local resource pool views.

[0080] Updates to some dynamic information within the resource pool. Resource descriptor. Resource status For example, the load rate of an AI acceleration unit or the operating status of a camera are dynamic. The node that owns the resource will periodically or when its status changes significantly will publish a resource status update message to the network. .

[0081] ;

[0082] in:

[0083] It is a unique identifier for a resource whose state has changed.

[0084] This is the latest status data for the resource.

[0085] It is the timestamp when the update message was generated, used to ensure the timeliness of the status.

[0086] This update message is disseminated across the entire network through the same synchronization mechanism as resource registration, enabling all nodes to obtain near real-time status of other nodes' resources, providing data support for subsequent dynamic scheduling and collaborative decision-making of AI computing tasks.

[0087] For distributed data synchronization mechanisms, those skilled in the art can adopt a variety of specific implementations, such as the anti-entropy Gossip protocol or the chain-replication protocol. Their specific implementations are well-known technologies in the field and will not be elaborated here.

[0088] One specific function of the multimodal data sharing and subscription module 20 is to standardize and encapsulate the heterogeneous data collected by the nodes and publish it to the distributed soft bus for other nodes in the network to subscribe to and use. This process is achieved through the following steps:

[0089] A node acquires raw data from physically connected devices through its hardware interface. This raw data has heterogeneous formats, such as RTSP video streams from cameras acquired via an Ethernet interface, or Modbus register data from a fire alarm control panel read via an RS485 interface. The node's device driver performs preliminary parsing of this raw data, converting it into a structured data format.

[0090] The parsed structured data is further encapsulated into a standardized data message. This eliminates the heterogeneity of the underlying data, facilitating unified processing by all nodes within the network. A specific data message It can be defined as:

[0091] ;

[0092] in:

[0093] It is a globally unique identifier for the source resource that generated the data. This identifier corresponds to the resource identifier in the global resource pool, allowing any recipient to clearly identify the source of the data.

[0094] It is the unique identifier of the source node that published the message.

[0095] It is a high-precision timestamp when the data is generated or collected, which is used for subsequent time series alignment and correlation analysis of multi-source data.

[0096] This is a type identifier that indicates the format of the data payload. This identifier explicitly tells the receiver how to parse the data payload. .

[0097] This is the actual data payload, which consists of parsed structured data. To facilitate network transmission, this data payload is serialized into a byte stream.

[0098] After encapsulation, the multimodal data sharing and subscription module 20 will send the data message. The submission is made to the underlying distributed soft bus communication component for publication. During publication, the source resource identifier is included in the message. This topic is used as the subject for the message publication. Other nodes in the network can subsequently receive the data stream generated by this resource by subscribing to this topic.

[0099] For the serialization format of the data payload, those skilled in the art can choose according to the requirements of transmission efficiency and readability.

[0100] The second specific function of the multimodal data sharing and subscription module 20 is to realize topic-based cross-node data routing and subscription, thereby building a decoupled and transparent data exchange bus.

[0101] A node (hereinafter referred to as a subscription node) Based on its business logic requirements, determine the data sources it needs to receive. Subscription Node By querying the aforementioned global resource pool Obtain the globally unique identifier of the target resource of interest. This node Maintain a local subscription set This set contains all the resource identifiers that it currently needs to subscribe to:

[0102] ;

[0103] Subscription Node Its subscription set The information is published to the distributed soft bus in the form of one or more subscription messages to declare its data demand to the entire network. This subscription information is disseminated throughout the network and used to dynamically build and maintain a distributed topic routing table. Logically, this routing table is a mapping where the key is a resource identifier (i.e., a topic), and the value is a set of nodes that have subscribed to that topic.

[0104] ;

[0105] Similar to the global resource pool, this topic routing table It also exists as a distributed data replica on each node, and maintains the consistency of the view among nodes through the same synchronization mechanism as the resource pool. When a node publishes a subscription or unsubscribe request, the local routing table replicas of all nodes in the network are updated accordingly.

[0106] When a source node Publish a resource containing the source resource identifier Data messages At that time, the source node The distributed soft bus communication component on the device will query its local topic routing table. It is with Using the key, retrieve the set of nodes that have subscribed to this topic. .

[0107] ;

[0108] Subsequently, the communication component sends the data message Copy and send to the collection separately Each subscription node in the process. This routing process is content-based, meaning the destination of a message is determined by the message's own topic (…). The decision is made by the publisher, not by the publisher specifying it in advance.

[0109] When subscribing nodes When data from a resource is no longer needed, it publishes an unsubscribe message to the network. This message will trigger an update to the distributed topic routing table. The update will change the node The identifier is removed from the subscriber set of the corresponding resource topic. After this, the source node will no longer send requests to the node. Forward the data from this resource.

[0110] This mechanism achieves spatiotemporal decoupling between publishers and subscribers. Any node can subscribe to data published by any other node in the network on demand, without needing to know the physical location of the data source or the network topology, thus forming a flexible and scalable distributed data sharing architecture.

[0111] The aforementioned global resource pool, combined with a topic-based publish / subscribe mechanism, constructs a cross-node data bus that is transparent to upper-layer applications. This bus allows applications on any node to access data from any other node in the network in a manner similar to accessing local resources, shielding them from the complexities of underlying distributed communication. This transparency is achieved through the following steps:

[0112] An upper-layer application module, such as the distributed collaborative AI analysis and decision-making module 30, generates a data requirement based on its business logic. This requirement is initially logical; for example, it may require acquiring video data from cameras located in "Main Transformer Area A".

[0113] This application module queries the synchronized local global resource pool copy of its node. It can use resource type, preset geographic location tags, or other metadata as query criteria to retrieve and obtain globally unique identifiers of resources that meet its logical requirements. This query process is completed entirely locally, without incurring any additional network overhead.

[0114] The application module calls a high-level subscription interface function provided by the multimodal data sharing and subscription module 20. This function only requires the identifier of the target resource. It takes a local callback function or data queue as input parameters to receive the resource. The application module does not need to know the physical node location, network address, or underlying communication protocol of the resource.

[0115] Upon receiving the higher-level subscription call, the multimodal data sharing and subscription module 20 internally executes the aforementioned complete subscription and route establishment process. This module is responsible for handling all the underlying distributed communication details, including updating the local subscription set, publishing subscription requests to the network to update the distributed topic routing table, and establishing data receiving channels.

[0116] Once the subscription relationship is established, the source node publishes data messages. The data is routed to the subscription node via the distributed soft bus. The multimodal data sharing and subscription module 20 on the subscription node receives the underlying network data packets, parses them, and delivers the extracted structured data payload to the upper-layer application module through a specified local callback function or data queue.

[0117] From the perspective of the upper-layer application module, the entire process is simplified to two steps: "querying the local resource directory" and "initiating a local data subscription". The data appears to flow directly from a logical, unified data bus, while the distributed system issues involved behind this bus, such as node discovery, data routing, and network communication, are all encapsulated by the lower-layer module, thus achieving transparency in data access.

[0118] One specific function of the distributed collaborative AI analysis and decision-making module 30 is to utilize multi-source data provided by a transparent cross-node data center to perform multi-perspective data fusion analysis, thereby improving the accuracy of complex event identification. Taking fire identification as an example, this process is achieved through the following steps:

[0119] A node AI analytics applications on the device utilize its locally connected camera resources. The generated video data stream is analyzed in real time. When its built-in AI model detects a target resembling a flame, it generates a local, initial detection result. The result is defined as:

[0120] ;

[0121] in:

[0122] These are the bounding box coordinates of the target in the current 2D image frame.

[0123] It is the identified target category.

[0124] This is the confidence score for this identification. If this score is lower than a preset fusion trigger threshold... If not, the subsequent process will not be initiated.

[0125] The location tag is obtained from the resource descriptor of the camera resource.

[0126] node The local test results Encapsulate as a candidate event message This message is then published to a specific topic on the distributed soft bus. The publication of this message announces a potential unconfirmed event to other nodes in the network that may be involved in the fusion decision-making process.

[0127] One or more nodes in the network, configured to perform fusion analysis tasks, have subscribed to the aforementioned candidate event topics. When a fusion node receives a candidate event message... Then, it uses this as a trigger, within a preset time window. Inside, collect tags from the same location. All other candidate event messages. These detection results, which are related in time and logical location, together constitute a set of detection results to be fused. .

[0128] Merge node pair set The confidence scores of multiple detection results are fused. A specific implementation method is to use a weighted average method to calculate a combined confidence score. :

[0129] ;

[0130] in:

[0131] It is the first in the set The confidence level of each test result.

[0132] Is assigned to the first The weight of each detection result. This weight can be set based on the preset static parameters or dynamic state of the camera that generated the result. This parameter information can be obtained from the global resource pool. Descriptor of the corresponding resource Retrieved internally.

[0133] The fusion node performs a spatial consistency check to confirm that the target observed by multiple cameras is the same target in physical space. This check process utilizes resource descriptors pre-stored in each camera. The camera calibration parameters are used in the fusion node to generate the 2D bounding box of each detection result. Through coordinate system transformation, the coordinates are projected onto the unified three-dimensional world coordinate system of the substation or a common two-dimensional ground plane to obtain the corresponding three-dimensional or two-dimensional region. Subsequently, the geometric relationships between these projected regions are calculated to derive a spatial consistency factor. .

[0134] The fusion node calculates the final fusion confidence score. This confidence level is determined by both the combined confidence level and the spatial consistency factor:

[0135] ;

[0136] Finally, the fusion confidence level With a higher and more stringent final decision threshold Compare. If If the system determines that a high-confidence fire event has occurred, it will generate a confirmed collaborative event alarm to trigger subsequent linkage control processes.

[0137] The second specific function of the distributed collaborative AI analysis and decision-making module 30 is to dynamically schedule and migrate AI computing tasks when the AI ​​computing resource load of a node is too high, so as to achieve load balancing of the computing capacity of the entire network. This process is achieved through the following steps:

[0138] A node (hereinafter referred to as the source node) The distributed collaborative AI analysis and decision-making module 30 on the platform continuously monitors the resource load rate of its local AI acceleration unit. This load rate can be calculated based on one or more parameters, such as NPU real-time utilization, inference task queue length, or processing frame rate. When this load rate... For an extended period of time, the migration trigger threshold is exceeded. At that time, the task migration process will be initiated.

[0139] Source node Query its locally maintained global resource pool replica This process filters out all nodes that have AI acceleration units and are not currently under high load, forming a candidate target node set. .

[0140] For the set of candidate target nodes Each node in Source node Calculate a migration cost function This cost function is used to evaluate the overall cost of migrating the task to the candidate node. A specific cost function can be defined as:

[0141] ;

[0142] in:

[0143] It is a candidate node The normalized load factor. It can be seen from its resource pool The dynamic state of the corresponding NPU resource descriptor The normalization operation maps the load rates of NPUs with different performance levels to a unified range.

[0144] It is the source node With candidate nodes The normalized network latency between nodes. This latency can be measured in real time through periodic probe messages between nodes and then normalized.

[0145] and These are preset weighting coefficients, and These two coefficients are used to adjust the relative importance of computational load and network communication in migration decisions.

[0146] Source node From the set of candidate target nodes In the middle, choose migration cost The smallest node is used as the final target node. .

[0147] ;

[0148] Source node To the target node Send a task migration request. The request contains details of the AI ​​task to be migrated, such as the identifier of the required AI model, the publishing topic of the input data stream required by the task, and the target topic to which the computation results of the task should be published.

[0149] target node After receiving and verifying the request, the task loading is executed. First, based on the input data topic in the request, it subscribes to the corresponding data stream through the multimodal data sharing and subscription module 20. Simultaneously, it loads and initializes the specified AI model. Once the task is ready, it begins processing the received data and encapsulates the AI ​​inference results into standardized data messages, which are then published to the target topic specified in the request.

[0150] After confirming the target node After successfully taking over the task and starting to publish valid computation results, the source node... Stop local computation of the task and release the corresponding computing resources to complete a dynamic migration of an AI computing task.

[0151] The third specific function of the distributed collaborative AI analysis and decision-making module 30 is to provide a distributed rule engine that makes decisions and responds to cross-node events that meet the triggering conditions based on preset linkage logic, thereby achieving automated collaborative control.

[0152] The system predefines one or more sets of collaborative rules. Each rule consists of one or more triggering conditions and one or more execution actions. A rule can be defined as a structured data object. For example, using JSON or XML format for description:

[0153] ;

[0154] in:

[0155] It is the unique identifier of the rule.

[0156] This is a set of triggering conditions that define the prerequisites for a rule to be activated. The conditions in this set can be combined using logical AND or logical OR. Each atomic condition... All are defined in the form of. Among them, It specifies the resources that serve as the source of conditions, which can be distributed across any different nodes in the network.

[0157] It is a sequence of actions to be performed. When When satisfied, the actions in the sequence will be executed sequentially. Each atomic action... All are defined as forms.

[0158] Each node in the system loads all collaborative rules. During the initialization phase, each node's rule engine parses all rules. For each rule, the rule engine extracts its... All resource identifiers that the part depends on Subsequently, the rules engine automatically subscribes to all data topics corresponding to these resource identifiers through the multimodal data sharing and subscription module 20.

[0159] During system operation, the node's rule engine receives the data messages it subscribes to. Each time a message is received, the rule engine updates its internally maintained atomic conditions related to the message source resource. The current state.

[0160] After each update of the atomic condition state, the rule engine re-evaluates the overall triggering conditions of all rules that depend on that atomic condition. Because each node subscribes to all the data sources required by the rule, each node can independently and completely evaluate the triggering conditions of any rule without the need for a central node to coordinate.

[0161] When a node's rule engine determines the overall triggering conditions of a rule When the condition is met, the node becomes the triggering node for the rule. The triggering node then begins executing the sequence of actions defined by the rule. .

[0162] For each atomic action in the sequence The trigger node constructs a standardized instruction message. The target subject of this instruction message is set to the target resource identifier specified in the atomic action. The message payload contains the command and parameters. This command message is then published via a distributed soft bus.

[0163] Having target resources Since the target node's rule engine also loads all rules, it must also subscribe to instruction topics related to its own resources. The target node receives the instruction message. Then, its local resource service module parses the instruction and calls the corresponding device driver interface to perform the specified operation on the physical hardware, thereby completing a cross-node collaborative decision-making and control closed loop.

[0164] One specific function of the collaborative event aggregation and reporting module 40 is to collect all data and status information related to the collaborative task and construct a complete collaborative event context after the distributed rule engine completes a collaborative decision and executes an action. This process is achieved through the following steps:

[0165] When a distributed rule engine successfully triggers and executes a collaborative rule—that is, after a node's rule engine determines that a rule's condition is met and issues an action instruction—the triggering node first generates a globally unique transaction ID for this collaborative task. This transaction ID will persist throughout the entire lifecycle of the collaborative event.

[0166] The triggering node stores the transaction ID locally and begins collecting all event data directly related to this collaborative task. This data includes:

[0167] Trigger condition snapshot: All atomic conditions that satisfy the condition when the rule is triggered. The specific value or state.

[0168] Trigger Time: The precise timestamp when the collaborative task starts. .

[0169] Trigger Rule Identifier: A unique identifier for the rule that triggered this collaboration. .

[0170] For each execution action defined in the coordination rules The triggering node issues a command message. At that time, the transaction ID of this collaborative task will be... ) is attached to the instruction message.

[0171] The target node that receives and executes the action command will generate an action execution result report after completing the hardware action. .

[0172] Report on the execution result of this action It will also carry the transaction ID ( ), and is published to a specific topic on the distributed soft bus.

[0173] When it receives a message carrying the same transaction ID ( After all the action execution results are reported, it will associate and integrate these results with the collected information such as trigger condition snapshots and trigger times.

[0174] In addition, to provide richer contextual information, the collaborative event aggregation and reporting module will also base its reporting on the transaction ID ( This involves tracing back the original sensing data and intermediate analysis results related to this collaborative event. For example, if this collaboration was triggered by a high-confidence fire event identified by the aforementioned multi-view data fusion analysis module, then that module will trace and collect all camera image frames or video clips that led to the determination of the fire event, as well as key data from the multi-source confidence fusion decision-making process. This data is also published to the bus with a transaction ID or retrieved by the responsible node by querying its local data cache.

[0175] Ultimately, all of these are in the transaction ID ( The information linked together constitutes the complete context of this collaborative event. This context It is a structured data collection that includes the causes and consequences of an event, the basis for decision-making, the execution process, and the results.

[0176] The second specific function of the collaborative event aggregation and reporting module 40 is to designate a unique node as the primary responsible node for this collaborative event through a distributed election mechanism when multiple nodes may simultaneously perceive the event or participate in collaborative decision-making. This node is responsible for context aggregation and reporting, avoiding duplicate alarms and resource waste. This mechanism is implemented through the following steps:

[0177] When a node's rule engine first determines that the triggering condition of a certain collaborative rule has been met, and thus generates a transaction ID ( When a node begins executing an action, it considers itself the potential principal responsible for the transaction. It immediately publishes a principal responsible node declaration message to a predefined topic on the distributed soft bus. .

[0178] ;

[0179] in:

[0180] It is the unique identifier for this collaborative transaction.

[0181] It is a declaration node A unique identifier.

[0182] It is the timestamp of the announcement message.

[0183] When a node receives a request for the same transaction ID ( When the primary node declares a message, it compares itself against other nodes. This comparison is based on a pre-defined, network-wide unified election criterion. This criterion aims to ensure that, in a distributed network, when all nodes compare the same set of declared messages, a unique winner can be determined. A specific election criterion could be:

[0184] ;

[0185] in:

[0186] It is a node The preset role priority. For example, some high-performance nodes may be assigned higher priority.

[0187] It is a priority factor based on the node's current resource status, such as the node's NPU idle computing power and network bandwidth, and is determined from its resource descriptor in the global resource pool. get.

[0188] Based on node unique identifier Fixed priority. For example, it can be used directly. The numerical value of the priority factor. With all other priority factors being equal, this fixed priority ensures the determinism of the election.

[0189] A node that receives a declaration message, if its own priority is higher than that of node VkVk in the declaration message, will also publish its own primary responsibility node declaration message. If its own priority is lower than or equal to... Then it accepts It becomes a potential primary responsible node and enters a waiting confirmation period.

[0190] Each node maintains an ID for each transaction. The timer is set. During the timer period, the node continuously listens for information about this... The primary responsible node declaration message. If, after the timer period ends, a node does not receive a declaration message with a higher priority than itself or a potential primary responsible node it has accepted, and it has already issued its own highest priority declaration, then the node will self-identify as the sole primary responsible node for this collaborative transaction.

[0191] Once a node is identified as the primary responsible node, it will send a primary responsible node confirmation message to the network. This includes the transaction ID ( The message includes the node's own identifier. Upon receiving the confirmation message, other nodes, if they had previously participated in the election or self-declaration, will cancel their own election or declaration status and record the primary responsible node specified in the confirmation message as the ultimate responsible party for the transaction.

[0192] The above mechanism ensures that only one node is responsible for context aggregation and reporting for the same collaborative event at the same time, thus avoiding duplicate processing and resource competition in a distributed environment.

[0193] The third specific function of the collaborative event aggregation and reporting module 40 is that the event-responsible node performs event semantic aggregation, integrating scattered event context information into a structured event report with high-level semantics, and reporting it to the cloud or centralized monitoring system. This process is achieved through the following steps:

[0194] The event's primary responsible node, elected through election, continuously receives and aggregates transaction IDs related to this collaborative event. All relevant context data. This data includes: trigger condition snapshots, and reports of all action execution results. And other traceable raw sensing data and intermediate analysis results. The responsible node integrates these heterogeneous and scattered data into a complete collaborative event context. .

[0195] The responsible node has access to the aggregated collaborative event context. Semantic interpretation and abstraction are performed to form an event summary with high-level semantics.

[0196] The responsible node will provide a semantic summary and a complete collaborative event context. Encapsulate it into a standardized event report. The report uses a unified, scalable structured format, such as an industry-standard event management format. A specific event report. It can be defined as:

[0197] ;

[0198] in:

[0199] It is the unique identifier for this collaborative transaction.

[0200] It is the start time of the event.

[0201] This is the identifier of the node primarily responsible for this incident.

[0202] It is the type of event.

[0203] It indicates the severity level of the incident.

[0204] It refers to the physical location where the event occurred.

[0205] It is a semantic summary of the event.

[0206] It is a complete collaborative event context. It can be a link to detailed data stored at the edge or in distributed storage, or a directly embedded structured data object.

[0207] The responsible node should report this structured event. The report is uploaded to a cloud platform or centralized monitoring system. The reporting mechanism can use standard IoT messaging protocols or HTTP(S) API. The report content is encrypted to ensure security during transmission.

[0208] Through this edge-side event semantic aggregation and structured reporting mechanism, cloud or centralized monitoring systems no longer need to process massive amounts of raw edge data. Instead, they directly receive high-level semantic events that have undergone edge intelligent analysis and collaborative decision-making, greatly reducing the data processing burden on the cloud and improving the efficiency of response to abnormal events. Simultaneously, a complete collaborative event context is provided. This also provides detailed data support for subsequent event auditing, backtracking analysis, and model optimization.

[0209] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multimodal data aggregation-assisted alarm system based on the HarmonyOS power system, characterized in that, Including deployments across multiple nodes in a distributed network: The distributed virtualization networking and resource pooling module is used to abstract the hardware resources of the nodes into resource descriptors containing resource identifiers, and to build a distributed global resource pool shared by all nodes. The multimodal data sharing and subscription module is used to encapsulate the data collected by the hardware resources into standardized data messages, and to realize cross-node data sharing based on the resource identifier as the topic through a publish / subscribe mechanism; The distributed collaborative AI analysis and decision-making module is used to perform collaborative AI analysis and decision-making based on the cross-node data sharing. The collaborative AI analysis and decision-making includes: performing multi-view data fusion analysis, performing dynamic scheduling of AI computing tasks, and running a distributed rule engine. The collaborative event aggregation and reporting module is used to aggregate the event context related to the collaborative decision by electing a unique event responsible node after the distributed rule engine triggers the collaborative decision, and generate a structured event report for reporting.

2. The system as described in claim 1, characterized in that, The distributed virtualization networking and resource pooling module is also used for: The global resource pool is dynamically updated in response to the joining or leaving of nodes through a synchronization mechanism. Resource status update messages are periodically published to maintain the consistency of dynamic information in the resource descriptor.

3. The system as described in claim 1, characterized in that, The multimodal data sharing and subscription module is specifically used for: Each node maintains a distributed topic routing table, which maps the resource identifier to the set of subscribing nodes, in the form of distributed data replicas. When the standardized data message is published, the message is routed to all subscribed nodes according to the distributed topic routing table.

4. The system as described in claim 1, characterized in that, The multi-perspective data fusion analysis includes: At one or more fusion nodes, initial detection results are collected from multiple hardware resources that have the same location tag; The confidence scores of the multiple initial detection results are merged, and the spatial consistency of the multiple initial detection results is checked using the camera calibration parameters pre-stored in the resource descriptor to generate a fused confidence score.

5. The system as described in claim 1, characterized in that, The dynamic scheduling of the AI ​​computing tasks includes: when the AI ​​computing resource load of the source node exceeds the migration trigger threshold, calculating the migration cost function based on the resource load rate of the candidate target node and the network latency between the target node and the source node, and selecting the candidate target node with the lowest cost for task migration.

6. The system as described in claim 1, characterized in that, When the distributed rule engine triggers a collaborative decision, it generates a globally unique transaction ID for that decision. The target node executing the action performs instruction deduplication based on the transaction ID to ensure the unique execution of the action.

7. The system as described in claim 1, characterized in that, The election mechanism for the event's primary responsible node includes: Nodes that meet the conditions for triggering collaborative decision-making publish a declaration message from the responsible node. All nodes in the network determine a unique event-responsible node based on a preset election criterion that integrates node role priority, real-time resource status, and fixed identifiers.

8. The system as described in claim 1, characterized in that, The event context includes: the transaction ID associated with the collaborative decision, a snapshot of the conditions that triggered the decision, result reports of all executed actions, and raw perception data related to the decision.

9. The system as described in claim 1, characterized in that, The triggering conditions of the distributed rule engine include comparing the fusion confidence score generated by the multi-perspective data fusion analysis with a preset decision threshold.

10. A multi-modal data aggregation-assisted early warning method based on the HarmonyOS power system, characterized in that, Includes the following steps: S1: Abstract the hardware resources of multiple nodes in a distributed network into resource descriptors containing resource identifiers, and build a distributed global resource pool shared by all nodes. S2: Encapsulate the data collected by the hardware resources into standardized data messages, and use the resource identifier as the topic to achieve cross-node data sharing through a publish / subscribe mechanism; S3: Based on the cross-node data sharing, perform collaborative AI analysis and decision-making, which includes: performing multi-view data fusion analysis, performing dynamic scheduling of AI computing tasks, and running a distributed rule engine; S4: After the distributed rule engine triggers a collaborative decision, a unique event responsible node is elected, the event context related to the collaborative decision is aggregated, and a structured event report is generated and reported.

Images