Router, gateway and camera edge AI collaborative decision method

Abstract

The application provides a router, a gateway and a camera edge AI collaborative decision-making method in the technical field of smart home, which comprises the following steps: collecting visual, physical quantity and network state data of a home environment through a camera, a gateway and a router respectively, performing time synchronization on multi-source data and establishing spatial correlation between devices after time stamp calibration and format standardization; an edge inference engine performs spatio-temporal consistency verification according to the spatial relationship, dynamically calculates the trust degree of each device, and generates a collaborative decision-making result by weighting and fusing the verification data; finally, a local control is executed by the router or a smart execution device, and the execution result is fed back to update the trust degree, forming a closed-loop feedback; the whole process realizes an intelligent home decision-making link from multi-source perception, spatio-temporal alignment, collaborative reasoning to execution adaptation, continuously optimizing the fusion quality and response reliability. The application has the advantages that the comprehensive decision-making efficiency of the home intelligent system is comprehensively improved.

Description

Technical Field

[0001] This invention relates to the field of smart home technology, and in particular to an edge AI collaborative decision-making method for routers, gateways, and cameras. Background Technology

[0002] With the rapid development of smart home technology, various edge devices with sensing and computing capabilities, such as cameras, gateways, and routers, are typically deployed in home environments. Cameras collect visual data to enable functions such as human detection, facial expression recognition, and posture change monitoring; gateways aggregate environmental physical quantity sensor data such as temperature and humidity, smoke concentration, infrared sensing, and gas concentration; and routers monitor network status information such as network traffic, device online status, and bandwidth usage. These devices provide fundamental data support for smart applications such as home security, elderly care, and child behavior management.

[0003] Currently, the mainstream decision-making methods for smart home systems can be summarized into the following three categories: The first type is the cloud-based centralized decision-making approach. Each edge device uploads raw data (including video streams, sensor readings, network logs, etc.) to a cloud server via a gateway. A deep learning model or rule engine deployed in the cloud performs unified reasoning and judgment, and then distributes the decision results to execution devices such as door locks, lights, and speakers. This approach relies on the powerful computing capabilities of the cloud to achieve complex reasoning.

[0004] The second type is the independent decision-making method of a single device. Each sensing device has a built-in lightweight algorithm model, which independently completes the detection task and triggers the corresponding action. For example, a camera starts recording after detecting a human figure, a smoke sensor triggers a local alarm when the threshold is exceeded, and a router executes parental control policies based on traffic statistics. There is no information exchange between the devices, and the decision-making process does not affect each other.

[0005] The third type is the simple rule engine approach. IFTTT (If This Then That) conditional rules are preset in the gateway or smart hub, triggering linked actions based on a single device event, such as "camera detects movement → turn on lighting" or "smoke alarm → send notification message". These rules are usually manually configured by the user, and the logical relationships are relatively simple.

[0006] However, the aforementioned existing technical solutions have the following significant drawbacks in practical applications: (i) The problem of data silos is serious. Cameras, sensors and routers collect multimodal data such as visual data, environmental physical quantities and network status, but each type of data is stored and processed independently in its own device. There is a lack of cross-device data sharing and fusion mechanisms, which makes it impossible for the system to form a unified understanding of the home scene, and the potential value of multi-source information is seriously wasted.

[0007] (ii) Insufficient real-time performance makes it difficult to meet the response requirements of emergency scenarios. Cloud-based centralized decision-making is limited by network transmission latency, cloud task queuing, and downlink latency, with response times typically reaching hundreds of milliseconds or even several seconds. In emergency scenarios such as fall detection, unauthorized intrusion alarms, and gas leaks, such delays can directly lead to safety hazards, and the latency problem is further exacerbated when the network is congested.

[0008] (iii) Significant privacy and security risks. Cloud-based decision-making solutions require uploading sensitive family data such as video, audio, and activity trajectories to public cloud servers, which poses security threats such as data leakage, misuse, and cyberattacks. However, existing technologies lack effective localized data processing and privacy protection mechanisms.

[0009] (iv) Lack of self-governance capability when the network is down. When the home broadband is interrupted or the cloud connection fails, the centralized decision-making system in the cloud is completely paralyzed; although the simple rule engine can run locally, its logic is fixed and cannot be dynamically adjusted according to real-time data from multiple devices, creating a security protection gap.

[0010] (v) The rules are rigid and cannot handle complex spatiotemporal scenarios. IFTTT-type rules only support single-condition triggering, making it difficult to express complex scenarios involving multiple devices, multimodal data, and temporal and spatial constraints, such as "an elderly person falls with cries for help and the indoor temperature rises abnormally". Existing solutions lack the ability to comprehensively reason about the temporal sequence and spatial relationship of events.

[0011] In summary, existing smart home decision-making solutions have significant shortcomings in areas such as cross-device collaboration, real-time performance, privacy and security, offline autonomy, and reasoning in complex scenarios. Therefore, providing an edge AI collaborative decision-making method involving routers, gateways, and cameras to achieve efficient fusion of multi-source data, real-time reasoning, privacy protection, and offline autonomy at the edge, thereby comprehensively improving the overall decision-making efficiency of home intelligent systems, has become an urgent technical problem to be solved. Summary of the Invention

[0012] The technical problem to be solved by this invention is to provide an edge AI collaborative decision-making method for routers, gateways and cameras, so as to achieve efficient fusion of multi-source data, real-time inference, privacy protection and offline autonomy at the edge, thereby comprehensively improving the overall decision-making efficiency of home intelligent systems.

[0013] This invention provides a collaborative decision-making method for edge AI of routers, gateways, and cameras, comprising the following steps: Step S1: Multi-source sensing and preprocessing step, where cameras, gateways and routers, which are used as sensing devices, collect visual data streams, environmental physical quantity sensing data streams and network status data streams of the home environment, respectively. The timestamp calibration and data format standardization of each data stream are performed to obtain standardized multi-source data. Step S2: Spatiotemporal alignment and spatial association step, which involves synchronizing the data streams of each sensing device in the standardized multi-source data in time and determining the spatial association relationship between each sensing device; Step S3: Collaborative reasoning step, whereby the edge reasoning engine deployed on the gateway performs spatiotemporal consistency verification on the standardized multi-source data after spatiotemporal alignment and spatial correlation based on the spatial correlation, determines the dynamic trust level corresponding to each sensing device, and performs weighted fusion on the verified standardized multi-source data according to the dynamic trust level to generate collaborative decision results; Step S4: Execution and feedback step, whereby the router or intelligent execution device performs local control operations based on the collaborative decision results, sends the execution results back to the edge inference engine, and updates the dynamic trust level based on the execution results.

[0014] Furthermore, in step S2, the spatial association is characterized by constructing a spatial topological association matrix, which includes the relative distances and azimuths between each sensing device, as well as the coverage affiliation of each sensing device relative to the camera's field of view. The coverage affiliation is determined by pre-calibrating the coordinates of each sensing device in the home space and mapping the coordinates to the image pixel coordinate system using the camera's intrinsic and extrinsic parameter matrix.

[0015] Furthermore, in step S2, the time synchronization is implemented using a time synchronization protocol, and the width of the time alignment window is dynamically adjusted according to the bandwidth utilization and transmission delay in the network status data stream: when the uplink bandwidth utilization is greater than or equal to 80% of the maximum bandwidth, the time alignment window is expanded to 1.5 times the default value; when the transmission delay is greater than 100 milliseconds, the time alignment window is expanded to 2 times the default value.

[0016] Furthermore, in step S3, the dynamic trust level is determined jointly based on the historical trust score and current data quality score of each sensing device; the current data quality score is negatively correlated with the network packet loss rate; the fusion weight of each sensing device is determined by the proportion of the dynamic trust level of that device in the sum of the dynamic trust levels of all participating devices.

[0017] Furthermore, in step S3, the weighted fusion includes: using the detection result of the visual data stream as a correction factor to correct the fusion confidence of the environmental physical quantity data in the standardized multi-source data within the camera's field of view, wherein the fusion confidence is a confidence value determined for the environmental physical quantity data in the weighted fusion; when the confidence of the visual detection result is higher than a preset threshold, the fusion confidence of the environmental physical quantity data is increased, and when it is lower than the preset threshold, the fusion confidence is decreased; the correction magnitude is non-linearly positively correlated with the confidence of the visual detection result.

[0018] Furthermore, in step S3, the collaborative reasoning step also supports low-code logic orchestration rules. The low-code logic orchestration rules support the combination of "AND" and "OR" logic between multiple device events and support constraints on the temporal order of events. When the visual human detection event and the gas concentration exceeding the standard data are in the same spatiotemporal window and the temporal order satisfies "gas concentration exceeding the standard first, human appearance later", it is determined to be a personnel entering a dangerous area event.

[0019] Furthermore, when a connection interruption with the cloud is detected, the system automatically switches to the network-disconnected autonomous mode. In the network-disconnected autonomous mode, the edge inference engine switches from a data-driven fusion mode to a rule-driven deductive mode. The security rule library, which was synchronized from the cloud and solidified locally before the network disconnection, serves as the primary decision-making basis. It also mandates that environmental physical quantity data in the standardized multi-source data within the camera's field of view must be visually detected and confirmed before security actions can be triggered.

[0020] Furthermore, in step S4, the local control operation includes at least one of the following: the router disconnects the Wi-Fi network connection of the designated terminal device, the smart lights switch the lighting status, the router pushes an alarm message containing the captured image to the user terminal through the cellular backup link, and the smart speaker plays a preset voice alarm.

[0021] Furthermore, in step S4, the specific method for updating the dynamic trust level is as follows: when the execution result is manually canceled by the user within a preset time, the dynamic trust level of each sensing device involved in the decision is increased with a positive reward value; when the standardized multi-source data corresponding to the execution result is not recovered within a preset time and the user does not manually cancel it, the dynamic trust level of the camera involved in the decision is decreased with a penalty value; the dynamic trust level decays exponentially on a weekly basis to prioritize reflecting the performance of each sensing device in the most recent decay cycle.

[0022] Furthermore, the method can be applied to scenarios such as home security intrusion detection, monitoring of abnormal behavior of elderly people living alone, or management of children's online behavior, to achieve multi-device collaborative decision-making based on spatiotemporal consistency verification and dynamic trust degree fusion at the edge. The advantages of this invention are: 1. By deploying an edge inference engine on the gateway, the three types of data streams—visual data, environmental physical quantities, and network status data—collected by cameras, gateways, and routers, are timestamped and standardized within the home network. A time synchronization protocol is used to dynamically adjust the alignment window and construct a spatial topology association matrix that includes relative distances and coverage affiliations between devices, achieving spatiotemporal alignment and association of multi-source data. The edge inference engine then performs weighted fusion of the spatiotemporally consistent data streams based on dynamic trust levels. Simultaneously, visual detection results are used as a correction factor to adjust the fusion confidence of environmental physical quantity data, forming a collaborative decision-making result that considers multimodal semantics. The router or intelligent execution device then directly executes control operations locally. Thus, the entire perception-inference-execution closed loop is completed at the edge, avoiding the limitations of traditional methods. This eliminates the privacy risks associated with uploading sensitive data such as video to the cloud, while also eliminating cloud round-trip delays, thus meeting the real-time response requirements in emergency scenarios. Furthermore, it automatically switches to a network-disconnected autonomous mode when a cloud connection interruption is detected, shifting from data-driven fusion to rule-driven deduction. It relies on a locally fixed security rule library and a mandatory visual confirmation mechanism to maintain decision-making capabilities. At the same time, it supports AND / OR logic and timing constraints between events from multiple devices through low-code logic orchestration, enabling the system to handle complex spatiotemporal scenarios such as "gas leak first, human figure appears later." Ultimately, a unified edge collaboration architecture simultaneously solves five major problems: data silos, insufficient real-time performance, privacy risks, network outage paralysis, and rigid rules, comprehensively improving the overall decision-making efficiency of home intelligent systems in scenarios such as security intrusion detection and monitoring of abnormal behavior in the elderly.

[0023] 2. Heterogeneous data streams are collected by three types of devices, and timestamp calibration and format standardization are performed. Then, spatiotemporal alignment is performed and a spatial topology association matrix is ​​constructed to clarify the relative distance, azimuth angle and coverage affiliation between devices. The edge inference engine performs spatiotemporal consistency verification based on spatial association and assigns dynamic trust to each device before weighted fusion. This technology enables the joint analysis of originally fragmented visual data, physical quantity sensing data and network status data under a unified spatiotemporal reference system. The system can form a comprehensive understanding of the home scene, effectively explore the potential value of multi-source information and overcome the problem of data silos.

[0024] 3. By deploying collaborative reasoning steps on the gateway's edge inference engine, all spatiotemporal consistency checks, dynamic trust calculations, and weighted fusion are completed locally at the edge, eliminating the need to upload raw data to the cloud. Local control operations that the router can perform, including disconnecting Wi-Fi connections, controlling lighting, pushing alarms, and playing voice alerts, are all executed locally in a closed loop. The entire decision-making chain, from perception and reasoning to execution, is completed within the home LAN, avoiding wide area network transmission and cloud processing. Response time can be compressed to the millisecond level, achieving an order-of-magnitude reduction in latency compared to cloud solutions, effectively ensuring timely response capabilities in emergency scenarios.

[0025] 4. The visual data streams captured by the cameras, the environmental physical quantity data aggregated by the gateway, and the network status data of the router are all preprocessed, spatiotemporally aligned, inferred, fused, and executed at the edge (gateway and router). Throughout the process, the original video streams, sensor readings, and other privacy data do not need to leave the home local network. Even in the offline autonomous mode, it can still run independently based on the locally fixed security rule base, and there is no need to upload data. This edge localized processing architecture fundamentally eliminates the privacy leakage risk caused by the transmission of sensitive data. In stark contrast to cloud solutions, it significantly improves the privacy and security level of the home smart system.

[0026] 5. When a connection interruption with the cloud is detected, the system automatically switches to the autonomous mode without internet access. The edge inference engine switches from a data-driven fusion mode to a rule-driven deductive mode, using the security rule base synchronized from the cloud and solidified locally before the internet outage as the primary decision-making basis. At the same time, it mandates that environmental physical quantity data within the camera's field of view must be visually verified before triggering security actions. This mechanism ensures that the system still has basic decision-making and execution capabilities in the absence of internet access, enhances the reliability of security triggering through visual verification, and avoids security protection gaps caused by network recovery delays by using a rule base solidified before the internet outage, thus completely eliminating the potential risks during the internet outage window.

[0027] 6. The spatial topological association matrix accurately depicts the relative distance, azimuth, and field of view coverage among devices. Low-code logic orchestration rules are introduced to support the combination of "AND" and "OR" logic between multiple device events, and to constrain the temporal order of events. In a specific example, when a visual human detection event and gas concentration exceeding the standard data are in the same spatiotemporal window and satisfy the order of "gas concentration exceeding the standard first, human appearance later", it is determined to be a person entering a dangerous area event. This technical means of combining spatial location association, time synchronization alignment, and temporal logic constraints enables the system to accurately identify complex scenarios involving multiple devices, multimodal data, and strict spatiotemporal conditions, which is far superior to simple single-condition rules. This significantly improves the system's reasoning ability for complex application scenarios such as home security and elderly monitoring.

[0028] 7. Dynamic trust level is jointly determined by the historical trust score and current data quality score of each device. The data quality score is negatively correlated with the network packet loss rate, enabling the trust level to reflect the current working status of the device in real time. The execution result is sent back to the inference engine. When the execution result is manually canceled by the user, a positive reward value is added. When the data is not recovered and the user does not cancel, a penalty value is applied to the camera trust level. The trust level decays exponentially on a weekly basis to prioritize reflecting recent performance. This closed-loop feedback mechanism enables the system to adaptively identify and rely on high-reliability devices and suppress the impact of low-quality or faulty devices. This allows for continuous optimization of the accuracy and robustness of fusion decisions in long-term operation, overcoming the inherent defect that static rules cannot be dynamically adjusted.

[0029] 8. Unlike traditional one-way data acquisition or cloud-based command delivery modes, this solution sends the execution results back and uses them to update the dynamic trust level, so that the system no longer relies on static thresholds or fixed models. Specifically, this feedback mechanism introduces user behavior as an implicit supervision signal—a positive reward is given when the execution result is manually canceled by the user, and a penalty is imposed when the anomaly is not recovered and not canceled, while in conjunction with a weekly exponential decay factor. This design enables the system's fusion weights to keenly track the recent performance degradation or environmental changes of each sensing device, effectively avoiding decision-making degradation problems caused by long-term operation such as sensor drift or camera field of view obstruction, and ensuring the long-term steady-state operation of the system in complex home scenarios.

[0030] 9. In addition to using conventional time synchronization, it creatively constructs a spatial topological correlation matrix that includes relative distance, azimuth angle, and "coverage affiliation". In particular, it uses the camera's intrinsic and extrinsic parameter matrix to map physical coordinates to the image pixel coordinate system. The deep advantage of this design is that it unifies visual two-dimensional semantics (pixel position) and environmental physical quantity sensing data (three-dimensional spatial coordinates) into the same spatiotemporal coordinate system. Combined with spatiotemporal consistency verification, the system can accurately determine whether "the temperature change point is within the camera's field of view" or "whether the moving target and the gas leak point are in the same physical area", thereby filtering out non-identical and asynchronous interference signals from the source and significantly reducing the false alarm rate in cross-modal data fusion.

[0031] 10. In edge computing scenarios, network jitter is the main bottleneck affecting the timing synchronization of multiple devices. By utilizing the "network status data stream" (bandwidth utilization, transmission latency) it can react to the width of the time alignment window. When the uplink bandwidth is congested (≥80%) or the transmission latency is too high (>100ms), the alignment window is actively expanded to 1.5 or 2 times the default value. This adaptive sliding window mechanism is equivalent to adding a "network congestion buffer pool" to the system, which can effectively avoid the erroneous discarding of valid data due to sudden network packet loss or latency, and ensure that during peak load periods of home networks (such as downloading large files or playing online games on multiple devices), collaborative decision-making can still make reasonable judgments based on sufficient time span data samples.

[0032] 11. Dynamic trust level is jointly determined by historical scores and current data quality (negatively correlated with packet loss rate), which ensures the objectivity of weight allocation; the high confidence results of visual inspection are used to perform nonlinear positive correlation correction on the fusion confidence of environmental physical quantity data; this means that when the camera clearly captures the hazard source, even if the physical sensor signal is weak, the system will increase its weight to respond quickly; conversely, if the visual confidence is low due to strong light or obstruction, the system will automatically reduce the weight of physical data that relies on visual verification; this "visual-guided" cross-modal correction strategy greatly optimizes the decision-making fault tolerance boundary in extreme scenarios compared to traditional equal weight or fixed weighting.

[0033] 12. A highly forward-looking disaster recovery mechanism has been designed to address the unreliability of home broadband. When the cloud connection is interrupted, the system automatically switches to "rule-driven deductive mode" and runs on a locally fixed security rule base. Crucially, this mode mandates that environmental physical data within the camera's field of view must be "visually verified before triggering security actions." This mandatory visual verification mechanism is equivalent to building a physical logic "AND gate" in the state of network outage "isolation," effectively preventing high-risk actions such as accidental triggering of solenoid valve shut-off due to simple sensor failure (such as a pet touching a gas alarm) when unattended. This ensures self-governance capabilities while strictly adhering to the bottom line of safety. Attached Figure Description

[0034] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0035] Figure 1 This is a flowchart of an edge AI collaborative decision-making method for routers, gateways, and cameras according to the present invention. Detailed Implementation

[0036] The overall approach of the technical solution in this application is as follows: Using an edge inference engine deployed on the gateway as the core decision-making hub, the originally independent cameras, gateways, and routers are transformed into collaborative sensing nodes for three types of data: visual data, environmental physical quantities, and network status data. Precise spatiotemporal alignment is achieved by performing timestamp calibration and format standardization on the three heterogeneous data streams, dynamically adjusting the time synchronization window based on network status, and constructing a spatial topology association matrix that includes the relative distance between devices and the camera's field of view coverage. Then, the edge inference engine performs spatiotemporal consistency verification under a unified spatiotemporal reference system. Based on a dynamic trust level jointly determined by historical trust scores and current data quality (negatively correlated with packet loss rate), the data streams are weighted and fused. Simultaneously, visual detection results are used as a correction factor to adjust the environment within the camera's field of view. The fusion confidence of physical quantity data is corrected by nonlinear positive correlation, and low-code rule orchestration is supported for multi-device events and / or logical and temporal constraints. Finally, the router or intelligent execution device performs control operations locally in a closed loop and sends back the execution results to update the dynamic trust through positive reward and punishment mechanisms, which decays exponentially on a weekly basis. At the same time, when the system detects a cloud connection interruption, it automatically switches to a network-disconnected autonomous mode. The data-driven fusion switches to rule-led deduction and requires visual detection confirmation before security actions can be triggered. Thus, with a complete edge-side perception-alignment-inference-execution-feedback closed-loop architecture, it simultaneously achieves multi-source information fusion, millisecond-level real-time response, localized retention of privacy data, reliable autonomy in network-disconnected states, and accurate inference in complex spatiotemporal scenarios, comprehensively improving the overall decision-making efficiency of the home intelligent system.

[0037] Please refer to Figure 1 As shown, a preferred embodiment of the edge AI collaborative decision-making method for routers, gateways, and cameras of the present invention includes the following steps: Step S1: Multi-source sensing and preprocessing step, where cameras, gateways and routers, which are used as sensing devices, collect visual data streams, environmental physical quantity sensing data streams and network status data streams of the home environment, respectively. The timestamp calibration and data format standardization of each data stream are performed to obtain standardized multi-source data. Specifically, the visual data stream includes RGB image sequences or video frame streams captured by cameras; the environmental physical quantity sensing data stream includes, but is not limited to, sensing values ​​and status quantities reported by temperature and humidity sensors, smoke concentration sensors, gas concentration sensors, infrared human body sensors, and door and window magnetic sensors; the network status data stream includes real-time traffic statistics of the router's WAN and LAN ports, uplink and downlink rates of each access terminal device, packet loss rate, transmission latency, and device online / offline status. The timestamp calibration refers to the gateway acting as a global time reference node, broadcasting a reference time to cameras and routers via PTP (Precise Time Protocol) or NTP (Network Time Protocol). Each device uniformly converts its own collected data's local timestamps to timestamps referenced to the gateway's reference time, ensuring a unified time reference system for subsequent time synchronization. The data format standardization refers to the unified encapsulation of raw data reported by various devices into a standardized JSON or Protobuf message format. This message format includes at least a device identifier field, a timestamp field, a data type field, and a data payload field, enabling the edge inference engine to uniformly parse and process multi-source data from different devices.

[0038] Step S2: Spatiotemporal alignment and spatial association step, which involves synchronizing the data streams of each sensing device in the standardized multi-source data in time and determining the spatial association relationship between each sensing device; The spatial relationships are represented by constructing a spatial topology association matrix M. Assuming N sensing devices (including cameras, gateway built-in sensors, and routers) are deployed in the home space, then M is an N×N matrix, where each element m_ij represents the spatial association metric between device i and device j. The construction of the spatial topology association matrix includes the following sub-steps: Sub-step S21: Establish a three-dimensional world coordinate system with a fixed reference point in the home space (such as the location of the gateway) as the origin, and obtain the three-dimensional coordinates (x_i, y_i, z_i) of each sensing device in the world coordinate system by manual measurement or automatic calibration. Sub-step S22: Calculate the relative distance d_ij = √((x_i-x_j)² + (y_i-y_j)² + (z_i-z_j)²) and the azimuth angle θ_ij between each sensing device, where the azimuth angle is based on the positive x-axis direction of the world coordinate system; Sub-step S23: Using the camera's intrinsic parameter matrix K (containing focal lengths f_x, f_y, principal point coordinates c_x, c_y, and distortion coefficients) and extrinsic parameter matrix [R|t] (containing rotation matrix R and translation vector t), map the world coordinates (X_w, Y_w, Z_w) of each sensing device to the image pixel coordinate system (u, v). The mapping relationship is: [u, v, 1]^T = K[R|t][X_w, Y_w, Z_w, 1]^T. If the mapped pixel coordinates (u, v) fall within the effective pixel area of ​​the camera image (i.e., 0 ≤ u < image width, 0 ≤ v < image height), then the device is determined to be within the camera's field of view coverage, and this coverage membership relationship is recorded in the spatial topological association matrix.

[0039] Step S3: Collaborative reasoning step, whereby the edge reasoning engine deployed on the gateway performs spatiotemporal consistency verification on the standardized multi-source data after spatiotemporal alignment and spatial correlation based on the spatial correlation, determines the dynamic trust level corresponding to each sensing device, and performs weighted fusion on the verified standardized multi-source data according to the dynamic trust level to generate collaborative decision results; Specifically, the spatiotemporal consistency verification includes the following verification dimensions: temporal continuity verification, which involves checking the data from the same sensing device within a continuous time window. If the data value undergoes a jump that does not conform to physical laws within a very short period of time, it is determined to be temporally inconsistent; and spatial logical self-consistency verification, which involves verifying whether the data from different sensing devices within the same spatiotemporal window corroborate each other based on the spatial topological correlation matrix. For example, if the visual detection result of the camera does not detect a human figure, but the infrared sensor within the coverage area of ​​the camera reports human movement, it is determined that the spatial logic of this set of data is inconsistent; conversely, if there is a reasonable temporal sequence relationship between the visual detection result and the infrared sensor report, it is determined to be spatiotemporally consistent.

[0040] Step S4: Execution and feedback step, whereby the router or intelligent execution device performs local control operations based on the collaborative decision results, sends the execution results back to the edge inference engine, and updates the dynamic trust level based on the execution results.

[0041] The execution result includes status information indicating whether the operation was successful or failed. Whether the planned operation was successfully executed or failed due to reasons such as device offline or network failure, the system sends the execution result and its related status code back to the edge inference engine. The edge inference engine treats operation failure as an important negative feedback signal, used to trigger an update of the trust level of the relevant execution device and the sensing devices involved in the decision-making process.

[0042] In step S2, the spatial association is characterized by constructing a spatial topological association matrix, which includes the relative distance and azimuth angle between each sensing device and the coverage membership of each sensing device relative to the camera's field of view. The coverage membership is determined by pre-calibrating the coordinates of each sensing device in the home space and mapping the coordinates to the image pixel coordinate system using the camera's intrinsic and extrinsic parameter matrix.

[0043] The camera's intrinsic parameter matrix includes parameters such as focal length, principal point coordinates, and distortion coefficients, while its extrinsic parameter matrix includes the camera's rotation matrix and translation vector relative to the home space's world coordinate system. Using camera calibration techniques such as the Zhang Zhengyou calibration method, these intrinsic and extrinsic parameter matrices can be obtained in advance. This allows the world coordinates of any device in the home space to be transformed to the corresponding pixel position in the camera's image pixel coordinate system. If this pixel position falls within the effective pixel area of ​​the image, the device is determined to be within the camera's field of view.

[0044] In step S2, the time synchronization is implemented using a time synchronization protocol, and the width of the time alignment window is dynamically adjusted according to the bandwidth utilization and transmission delay in the network status data stream: when the uplink bandwidth utilization is greater than or equal to 80% of the maximum bandwidth, the time alignment window is expanded to 1.5 times the default value; when the transmission delay is greater than 100 milliseconds, the time alignment window is expanded to 2 times the default value.

[0045] It should be noted that when the uplink bandwidth utilization rate is greater than or equal to 80% of the maximum bandwidth and the transmission delay is greater than 100 milliseconds, the width of the time alignment window is multiplied by a factor of 3, that is, expanded to 1.5 × 2 = 3 times the default value, in order to cope with packet out-of-order and delay jitter caused by severe network congestion.

[0046] The preferred time synchronization protocol is the IEEE 1588 PTP protocol, which achieves sub-millisecond time synchronization accuracy under good network conditions. The time alignment window defines the allowable deviation range in the time dimension for each data stream participating in joint inference; the default window width is set to 500 milliseconds based on actual system deployment experience. Data arriving within the window is considered time-aligned and processed uniformly by the edge inference engine; data exceeding the window range is cached for the next processing cycle or discarded to prevent expired data from misleading inference decisions. The mechanism of dynamically adjusting the alignment window ensures that the system can adaptively balance decision timeliness and data inclusiveness when network conditions change.

[0047] The network state data stream is collected by the router, and at least includes the upstream bandwidth occupancy rate and the data packet transmission delay of the WAN (Wide Area Network) port of the router. The upstream bandwidth occupancy rate reflects the link congestion degree of the home network transmitting data to the cloud, and the transmission delay reflects the round-trip time of data from the gateway to the cloud server, both of which jointly serve as the basis for adjusting the size of the time alignment window.

[0048] In step S3, the dynamic trust degree is determined jointly according to the historical trust score and the current data quality score of each sensing device; the current data quality score is negatively correlated with the network packet loss rate; the fusion weight of each sensing device is determined by the proportion of the dynamic trust degree of the device in the sum of the dynamic trust degrees of all participating devices.

[0049] In step S3, the weighted fusion includes: using the detection result of the visual data stream as a correction factor to correct the fusion confidence of the environmental physical quantity data in the standardized multi-source data located within the visible range of the camera, wherein the fusion confidence is the confidence value determined for the environmental physical quantity data in the weighted fusion; when the confidence of the visual detection result is higher than a preset threshold, the fusion confidence of the environmental physical quantity data is increased, and when the confidence is lower than the preset threshold, the fusion confidence is decreased; the correction amplitude is non-linearly positively correlated with the confidence of the visual detection result.

[0050] The non-linear positive correlation is preferably implemented by an S-type function (Sigmoid function) or a logarithmic function. Specifically, let the confidence of the visual detection result be c_v (value range from 0 to 1), and the preset threshold be T (T∈[0.7, 0.9]), then the calculation formula of the correction coefficient α is: α=1 / (1+e^(-k(c_v-T))), where k is a preset steepness coefficient, and the value range of k is 5 to 15. When c_v>T, α>1, the fusion confidence is increased; when c_v<T, α<1, the fusion confidence is decreased; when c_v=T, α=1, the original fusion confidence is maintained. This non-linear correction mechanism enables the change of the confidence of the visual detection result near the threshold to produce a smooth and sensitive adjustment effect on the fusion confidence, and the correction amplitude tends to be saturated when the confidence is much higher or much lower than the threshold, so as to prevent over-correction.

[0051] Wherein, the preset threshold is used to determine whether the visual detection result is sufficiently credible, and its value is a confidence value within the range of 0.7 to 0.9, preferably 0.85. This threshold can be dynamically adjusted according to actual application scenarios (such as security level requirements).

[0052] In step S3, the collaborative reasoning step also supports low-code logic orchestration rules. The low-code logic orchestration rules support the combination of "AND" and "OR" logic between multiple device events and support constraints on the temporal order of events. When the visual human detection event and the gas concentration exceeding the standard data are in the same spatiotemporal window and the temporal order satisfies "gas concentration exceeding the standard first, human appearance later", it is determined to be a personnel entering a dangerous area event.

[0053] The implementation of the low-code logic orchestration rules includes: the system providing users or developers with a visual rule orchestration interface or configuration file interface. This interface or interface defines a series of event sources (such as "visual human detection event" and "gas concentration exceeding the standard event"), logical operators ("AND", "OR", and "NOT"), temporal relational operators ("first", "last", and "simultaneous"), and action executors. Users combine these elements by dragging and dropping or configuring them. The system backend parses these combinations into an executable conditional decision tree or finite state machine. For example, for the rule "gas concentration exceeding the standard first, human appearance second," the system parses it into a two-state sequence detector. When the "concentration exceeding the standard" and "human appearance" event states are detected sequentially within a preset time window, the corresponding decision is triggered.

[0054] When a connection interruption with the cloud is detected, the system automatically switches to the autonomous mode without network access. In the autonomous mode without network access, the edge inference engine switches from the data-driven fusion mode to the rule-driven deductive mode. The security rule library, which was synchronized from the cloud and solidified locally before the network disconnection, serves as the primary decision-making basis. Furthermore, it mandates that environmental physical quantity data in the standardized multi-source data within the camera's field of view must be visually detected and confirmed before security actions can be triggered.

[0055] As a preferred implementation, when the system is in a normal connection with the cloud, the edge inference engine will periodically (e.g., every morning) or when there are updates to the cloud rule base, proactively synchronize the latest security rule base from the cloud and store it in the gateway's local non-volatile memory to replace or supplement older rule versions. This mechanism ensures that the locally stored rule base is as up-to-date as possible before network outages, improving the timeliness and accuracy of decision-making in the autonomous mode during network outages.

[0056] In step S4, the local control operation includes at least one of the following: the router disconnects the Wi-Fi network connection of the designated terminal device, the smart lights switch the lighting status, the router pushes an alarm message containing the captured image to the user terminal through the cellular backup link, and the smart speaker plays a preset voice alarm.

[0057] The cellular backup link refers to the router's built-in 4G / 5G cellular communication module. When the main home broadband link is interrupted, the router automatically activates the cellular backup link as an emergency communication channel to ensure that alarm messages can be pushed to user terminals in real time. The captured image is generated by JPEG compression of key frames of the visual data stream captured by the camera at the moment of decision triggering. The alarm message also includes the type of triggering event, the time of occurrence, and a location description. The preset voice alarm is stored in the local memory of the gateway or smart speaker. The corresponding voice file is selected and played according to the type of decision result. For example, for a gas leak event, a voice prompt "Abnormal gas concentration detected, please check immediately" is played.

[0058] In step S4, the specific method for updating the dynamic trust level is as follows: when the execution result is manually canceled by the user within a preset time, the dynamic trust level of each sensing device involved in the decision is increased with a positive reward value; when the standardized multi-source data corresponding to the execution result is not recovered within a preset time and the user does not manually cancel it, the dynamic trust level of the camera involved in the decision is reduced with a penalty value; the dynamic trust level decays exponentially on a weekly basis to prioritize reflecting the performance of each sensing device in the most recent decay cycle.

[0059] The preset time is preferably 2 minutes. The positive reward value is calculated as follows: ΔR_reward = β × (1 - current trust level / trust level cap), where β is the preset base reward coefficient, and the value of β ranges from 5 to 20. This calculation method allows devices with lower trust levels to receive a larger reward increase when making correct decisions, thus encouraging and quickly restoring the weight of low-trust devices. The penalty value is calculated as follows: ΔR_penalty = γ × (current trust level / trust level cap), where γ is the preset base penalty coefficient, and the value of γ ranges from 10 to 30. This calculation method allows devices with higher trust levels to suffer a larger penalty reduction when making incorrect decisions, thus constraining high-trust devices and preventing them from continuously obtaining undue high weight due to historically high scores.

[0060] The specific implementation of the exponential decay of dynamic trust on a weekly basis is as follows: A timestamp field is assigned to the dynamic trust value of each sensing device to record the time of its most recent update. Every week, the system performs a decay calculation on all trust values ​​using an exponential decay function: new trust = current trust × e^(-λ), where λ is a preset decay coefficient, ranging from 0.1 to 0.5, with λ=0.3 being preferred. This mechanism ensures that the contribution of historical trust values ​​older than approximately 8 weeks will decay to less than 10% of their original impact, guaranteeing that the trust value sensitively reflects the recent performance of each sensing device.

[0061] The preset time is used to determine whether the user has interfered with the system's decision-making. It can be a preset value in the range of 30 seconds to 5 minutes, preferably 2 minutes.

[0062] The method is applied to scenarios such as home security intrusion detection, monitoring of abnormal behavior of elderly people living alone, or management of children's online behavior, and is used to achieve multi-device collaborative decision-making based on spatiotemporal consistency verification and dynamic trust degree fusion at the edge.

[0063] In the embodiments described above, the router, gateway, and camera are all edge devices with independent network addresses within the home LAN. They are connected to the same home LAN via wired (Ethernet) or wireless (Wi-Fi) connections, enabling interconnection between devices. The edge inference engine is deployed as software at the gateway's operating system layer, continuously running as a system daemon to monitor and process data streams from each sensing device. In normal operation, the system primarily uses a data-driven fusion mode, where the edge inference engine integrates the spatiotemporal consistency verification results and dynamic trust-weighted fusion results of the three types of data streams for inference decisions. In network-disconnected autonomous mode, the system automatically switches to a rule-driven deductive mode, ensuring basic security protection capabilities even when cloud connection is abnormal. The switching between the two modes is monitored by a heartbeat detection module built into the gateway. This module sends heartbeat requests to the cloud server at a fixed frequency (preferably once every 30 seconds). If three consecutive heartbeat requests time out without response, the cloud connection is considered interrupted, triggering a mode switch.

[0064] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A collaborative decision-making method for edge AI of router, gateway, and camera, characterized in that, Includes the following steps: Step S1: Multi-source sensing and preprocessing step, where cameras, gateways and routers, which are used as sensing devices, collect visual data streams, environmental physical quantity sensing data streams and network status data streams of the home environment, respectively. The timestamp calibration and data format standardization of each data stream are performed to obtain standardized multi-source data. Step S2: Spatiotemporal alignment and spatial association step, which involves synchronizing the data streams of each sensing device in the standardized multi-source data in time and determining the spatial association relationship between each sensing device; Step S3: Collaborative reasoning step, whereby the edge reasoning engine deployed on the gateway performs spatiotemporal consistency verification on the standardized multi-source data after spatiotemporal alignment and spatial correlation based on the spatial correlation, determines the dynamic trust level corresponding to each sensing device, and performs weighted fusion on the verified standardized multi-source data according to the dynamic trust level to generate collaborative decision results; Step S4: Execution and feedback step, whereby the router or intelligent execution device performs local control operations based on the collaborative decision results, sends the execution results back to the edge inference engine, and updates the dynamic trust level based on the execution results.

2. The edge AI collaborative decision-making method for routers, gateways, and cameras as described in claim 1, characterized in that, In step S2, the spatial association is characterized by constructing a spatial topological association matrix, which includes the relative distance and azimuth angle between each sensing device and the coverage membership of each sensing device relative to the camera's field of view. The coverage membership is determined by pre-calibrating the coordinates of each sensing device in the home space and mapping the coordinates to the image pixel coordinate system using the camera's intrinsic and extrinsic parameter matrix.

3. The edge AI collaborative decision-making method for routers, gateways, and cameras as described in claim 1, characterized in that, In step S2, the time synchronization is implemented using a time synchronization protocol, and the width of the time alignment window is dynamically adjusted according to the bandwidth utilization and transmission delay in the network status data stream: when the uplink bandwidth utilization is greater than or equal to 80% of the maximum bandwidth, the time alignment window is expanded to 1.5 times the default value; when the transmission delay is greater than 100 milliseconds, the time alignment window is expanded to 2 times the default value.

4. The edge AI collaborative decision-making method for routers, gateways, and cameras as described in claim 1, characterized in that, In step S3, the dynamic trust level is determined jointly based on the historical trust score and the current data quality score of each sensing device; the current data quality score is negatively correlated with the network packet loss rate; the fusion weight of each sensing device is determined by the proportion of the dynamic trust level of that device in the sum of the dynamic trust levels of all participating devices.

5. The edge AI collaborative decision-making method for routers, gateways, and cameras as described in claim 1, characterized in that, In step S3, the weighted fusion includes: using the detection result of the visual data stream as a correction factor to correct the fusion confidence of the environmental physical quantity data in the standardized multi-source data within the camera's field of view, wherein the fusion confidence is a confidence value determined for the environmental physical quantity data in the weighted fusion; when the confidence of the visual detection result is higher than a preset threshold, the fusion confidence of the environmental physical quantity data is increased, and when it is lower than the preset threshold, the fusion confidence is decreased; the correction magnitude is non-linearly positively correlated with the confidence of the visual detection result.

6. The edge AI collaborative decision-making method for routers, gateways, and cameras as described in claim 1, characterized in that, In step S3, the collaborative reasoning step also supports low-code logic orchestration rules. The low-code logic orchestration rules support the combination of "AND" and "OR" logic between multiple device events and support constraints on the temporal order of events. When the visual human detection event and the gas concentration exceeding the standard data are in the same spatiotemporal window and the temporal order satisfies "gas concentration exceeding the standard first, human appearance later", it is determined to be a personnel entering a dangerous area event.

7. The edge AI collaborative decision-making method for routers, gateways, and cameras as described in claim 1, characterized in that, When a connection interruption with the cloud is detected, the system automatically switches to the autonomous mode without network access. In the autonomous mode without network access, the edge inference engine switches from the data-driven fusion mode to the rule-driven deductive mode. The security rule library, which was synchronized from the cloud and solidified locally before the network disconnection, serves as the primary decision-making basis. Furthermore, it mandates that environmental physical quantity data in the standardized multi-source data within the camera's field of view must be visually detected and confirmed before security actions can be triggered.

8. The edge AI collaborative decision-making method for routers, gateways, and cameras as described in claim 1, characterized in that, In step S4, the local control operation includes at least one of the following: the router disconnects the Wi-Fi network connection of the designated terminal device, the smart lights switch the lighting status, the router pushes an alarm message containing the captured image to the user terminal through the cellular backup link, and the smart speaker plays a preset voice alarm.

9. The edge AI collaborative decision-making method for routers, gateways, and cameras as described in claim 1, characterized in that, In step S4, the specific method for updating the dynamic trust level is as follows: when the execution result is manually canceled by the user within a preset time, the dynamic trust level of each sensing device involved in the decision is increased with a positive reward value; when the standardized multi-source data corresponding to the execution result is not recovered within a preset time and the user does not manually cancel it, the dynamic trust level of the camera involved in the decision is reduced with a penalty value; the dynamic trust level decays exponentially on a weekly basis to prioritize reflecting the performance of each sensing device in the most recent decay cycle.

10. A router, gateway, and camera edge AI collaborative decision-making method as described in any one of claims 1-9, characterized in that, The method is applied to scenarios such as home security intrusion detection, monitoring of abnormal behavior of elderly people living alone, or management of children's online behavior, and is used to realize multi-device collaborative decision-making based on spatiotemporal consistency verification and dynamic trust degree fusion at the edge.

Images