Multi-modal industrial Internet of Things intelligent gateway based on edge computing and implementation method thereof

Abstract

The invention relates to the technical field of intelligent gateways, and discloses a multi-mode industrial Internet of Things intelligent gateway based on edge computing and an implementation method thereof, and the method comprises the steps: obtaining sensor data streams of a door magnetic sensor, a human body sensor, a temperature and humidity sensor and a smoke detector in an intelligent region; constructing a space-time fusion data matrix based on the sensor data stream; inputting the space-time fusion data matrix into an edge layer quantization neural network, a fog layer recurrent neural network and a cloud layer large model for distributed reasoning to obtain a reasoning result set; and performing priority queue scheduling and zero-copy transmission on emergency events, important events and conventional events in combination with the reasoning result set to generate a control instruction sequence, so that parallel operation of data acquisition and processing is realized, and the intelligent level, the response performance and the operation reliability of an intelligent regional Internet of Things system are improved.

Description

Technical Field

[0001] This application relates to the field of smart gateway technology, and in particular to a multimodal industrial IoT smart gateway based on edge computing and its implementation method. Background Technology

[0002] The intelligent area is equipped with a large number of heterogeneous sensor devices, including door magnetic sensors, human body sensors, temperature and humidity sensors, smoke detectors, and other types of sensing devices. These sensors have different sampling frequencies, data formats, and spatial distribution characteristics. Traditional IoT data processing methods typically adopt a centralized architecture, directly uploading all sensor data to the cloud for unified processing. However, this approach faces problems such as data transmission bandwidth limitations, uncontrollable network latency, and competition for cloud computing resources, making it difficult to meet the stringent requirements of real-time performance and reliability for intelligent management. Summary of the Invention

[0003] This application provides a multimodal industrial IoT smart gateway based on edge computing and its implementation method. The invention realizes parallel operation of data acquisition and processing, and improves the intelligence level, response performance and operational reliability of intelligent regional IoT systems.

[0004] The first aspect of this application provides a method for implementing a multimodal industrial IoT smart gateway based on edge computing, the method comprising: Acquire sensor data streams from door magnetic sensors, human body sensors, temperature and humidity sensors, and smoke detectors within the intelligent area; A spatiotemporal fusion data matrix is ​​constructed based on the sensor data stream; The spatiotemporal fusion data matrix is ​​input into the edge layer quantization neural network, the fog layer recurrent neural network, and the cloud layer large model for distributed inference to obtain an inference result set. Based on the inference result set, priority queue scheduling and zero-copy transmission are performed on emergency events, important events, and routine events to generate a sequence of control instructions.

[0005] In conjunction with the first aspect, in a first implementation of the first aspect of the present invention, acquiring the sensor data streams of the door magnetic sensor, human body sensor, temperature and humidity sensor, and smoke detector within the intelligent area includes: A three-dimensional spatial coordinate system for the intelligent area is constructed, and based on the three-dimensional spatial coordinate system, the door magnetic sensor, human body sensor, temperature and humidity sensor and smoke detector are mapped to the length, width and height coordinate positions to obtain the device position matrix; Based on the device location matrix, a dual-buffer data storage architecture including a main buffer and a secondary buffer is configured; Based on the dual-buffer data storage architecture, event-triggered sampling is performed on the door magnetic sensor and smoke detector, frequency sampling is performed on the human body sensor, and frequency sampling is performed on the temperature and humidity sensor to obtain the original signal set; The original signal set is encapsulated according to the format of device identifier, timestamp, data value and spatial coordinate to obtain sensor data stream.

[0006] In conjunction with the first aspect, in a second implementation of the first aspect of the present invention, the configuration of a dual-buffer data storage architecture comprising a primary buffer and a secondary buffer based on the device location matrix includes: Calculate the buffer configuration parameters based on the number of sensors and spatial distribution density in the device location matrix; Based on the buffer configuration parameters, a main buffer responsible for real-time data writing operations and a secondary buffer responsible for data reading and processing operations are created respectively. A timed switching mechanism is implemented for the main buffer and the secondary buffer. When the main buffer performs a write operation, the secondary buffer performs a read operation synchronously. After the switching time arrives, the read and write roles of the main buffer and the secondary buffer are interchanged, thus obtaining the buffer switching control mechanism. The buffer switching control mechanism is integrated with the main buffer and the secondary buffer to form a dual-buffer data storage architecture.

[0007] In conjunction with the first aspect, in a third implementation of the first aspect of the present invention, the step of constructing a spatiotemporal fusion data matrix based on the sensor data stream includes: Extract the spatial coordinate information of each sensor in the sensor data stream and calculate the Euclidean distance between the sensors, and convert the Euclidean distance into a spatial weight matrix based on the spatial coordinate information; The event trigger data of the door magnetic sensor and smoke detector, the first frequency data of the human body sensor and the second frequency data of the temperature and humidity sensor are identified from the sensor data stream. Cubic spline interpolation is performed using the spatial topology sensing temporal alignment algorithm to generate a temporal calibration dataset. The time-series calibration dataset is fused with the spatial weight matrix to produce a spatially weighted fusion result. The missing data in the spatially weighted fusion result is filled with numerical values ​​using a neighboring sensor spatial correlation prediction algorithm to form a spatiotemporal fusion data matrix.

[0008] In conjunction with the first aspect, in the fourth implementation of the first aspect of the present invention, the step of inputting the spatiotemporal fusion data matrix into the edge layer quantization neural network, the fog layer recurrent neural network, and the cloud layer large model for distributed inference to obtain an inference result set includes: The spatiotemporal fusion data matrix is ​​input into the edge layer quantization neural network for feature extraction and scene change prediction, generating edge layer inference results; The spatiotemporal fusion data matrix is ​​input into the fog layer recurrent neural network to perform behavior sequence recognition and generate fog layer inference results. The spatiotemporal fusion data matrix is ​​input into the cloud big model for preference learning and control optimization, and the cloud inference results are output. The device control commands in the edge layer inference results, the behavior recognition information in the fog layer inference results, and the optimization strategies in the cloud layer inference results are integrated to form an inference result set.

[0009] In conjunction with the first aspect, in a fifth implementation of the first aspect of the present invention, the step of inputting the spatiotemporal fusion data matrix into an edge layer quantization neural network for feature extraction and scene change prediction to generate an edge layer inference result includes: The spatiotemporal fusion data matrix is ​​input into the edge layer quantization neural network, and parallel computation is accelerated by enabling the NEON instruction set of the ARM Cortex-A72 processor to obtain the target input features; The target input features are input into the depth-separable convolutional layer and the inverse residual module to calculate the spatial correlation features and temporal evolution features of the multimodal sensors in the region layer by layer, so as to obtain the feature representation vector; The feature representation vector is used as the system state input. The Kalman filter prediction control module performs recursive state estimation to form scene change prediction data. Based on the feature representation vector and the scene change prediction data, the edge layer inference result containing device control instructions is output.

[0010] In conjunction with the first aspect, in a sixth implementation of the first aspect of the present invention, the step of inputting the spatiotemporal fusion data matrix into a fog layer recurrent neural network to perform behavioral sequence recognition and generate fog layer inference results includes: The spatiotemporal fusion data matrix is ​​reorganized and its feature dimensions are transformed to obtain behavioral sequence input data; The behavioral sequence input data is input into a fog layer recurrent neural network to extract behavioral temporal features and generate a bidirectional hidden state sequence. Based on the bidirectional hidden state sequence, a sliding time window attention mechanism is performed to generate an attention-weighted feature vector; The attention-weighted feature vector is input into the behavior classifier to perform guest behavior pattern recognition, and the output is a fog layer inference result containing behavior recognition information.

[0011] In conjunction with the first aspect, in the seventh implementation of the first aspect of the present invention, the step of inputting the spatiotemporal fusion data matrix into a large cloud model for preference learning and control optimization, and outputting cloud inference results, includes: Semantic encoding is performed on the regional environmental state information and guest behavior characteristics in the spatiotemporal fusion data matrix to obtain a semantically encoded input sequence; The semantically encoded input sequence is input into a cloud-based large model for personalized behavior pattern analysis to generate guest preference features. Based on the guest preference features, a federated learning framework is initiated to perform distributed model parameter updates, generating a personalized inference model, and then cloud-based inference results containing optimization strategies are generated through the personalized inference model.

[0012] In conjunction with the first aspect, in the eighth implementation of the first aspect of the present invention, the step of performing priority queue scheduling and zero-copy transmission on emergency events, important events, and routine events based on the inference result set to generate a control instruction sequence includes: Based on the reasoning results, the urgency of the reasoning instructions is classified into emergency events (door magnetic triggering and smoke alarm), important events (human body induction response), and routine events (temperature and humidity control). A queue scheduling mechanism is established based on the token bucket dynamic allocation algorithm applied to emergency events, important events, and regular events. The system monitors the event data processed by the queue scheduling mechanism. When the event data is ready, the sendfile zero-copy function is activated to perform data migration operations in the kernel space, thereby completing efficient data transmission processing. The control command sequence generated by the efficient data transmission processing is obtained, and the target device is directly reached based on the control command sequence using DPDK technology.

[0013] A second aspect of this application provides a multimodal industrial IoT smart gateway based on edge computing, wherein the multimodal industrial IoT smart gateway based on edge computing is used to execute the above-described multimodal industrial IoT smart gateway implementation method based on edge computing.

[0014] Compared with existing technologies, this application has the following advantages: By constructing a spatial weight matrix and a temporal alignment algorithm, it effectively solves the problem of spatiotemporal inconsistency in multi-source heterogeneous sensor data, enabling fused data to accurately reflect the true state of the region. The dual-buffer data storage architecture achieves parallel operation of data acquisition and processing through a timed switching mechanism between the primary and secondary buffers, eliminating the data blocking problem of the traditional single-buffer mode. The three-level distributed inference engine, combined with edge-layer quantized neural networks, fog-layer recurrent neural networks, and cloud-layer large models, achieves collaborative intelligent processing of local real-time feature extraction, behavior sequence recognition, and personalized preference learning, reducing dependence on network connectivity. The priority queue scheduling mechanism, through a three-level queue architecture and a token bucket dynamic allocation algorithm, differentiates processing based on the urgency of events, ensuring priority response to urgent events. Zero-copy technology and the DPDK user-space protocol stack bypass the processing overhead of traditional network protocol stacks, significantly reducing data transmission latency. This invention improves the intelligence level, response performance, and operational reliability of intelligent regional IoT systems. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] The structures, proportions, sizes, etc., shown in the accompanying drawings of this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed in the specification, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0017] Figure 1 This is a flowchart illustrating the implementation method of a multimodal industrial IoT smart gateway based on edge computing provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0020] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0021] It should also be further understood that the term "and / or" as used in this application specification and the appended claims refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes such combinations. See also Figure 1 One embodiment of the multimodal industrial IoT smart gateway implementation method based on edge computing in this application includes: Step 100: Acquire sensor data streams from door magnetic sensors, human body sensors, temperature and humidity sensors, and smoke detectors within the intelligent area; Specifically, a three-dimensional spatial coordinate system covering length, width, and height is established based on the actual physical layout of the intelligent area. The installation positions of door magnetic sensors, human body sensors, temperature and humidity sensors, and smoke detectors are then calibrated within this system. By mapping the physical installation point of each sensor to its corresponding length, width, and height coordinates, a device location matrix containing the spatial distribution information of all sensors is generated. Based on this location matrix, a dual-buffer data storage architecture with a main buffer and a secondary buffer is configured. The main buffer receives and writes the latest sensor data in real time, while the secondary buffer reads and processes data in parallel. Periodic switching between the main and secondary buffers decouples data acquisition and processing, ensuring data continuity and real-time performance even with high-frequency updates. Regarding the data acquisition strategy, event-triggered sampling is used for door magnetic sensors and smoke detectors based on their operating characteristics, generating data only when the state changes to reduce unnecessary acquisition. Higher-frequency periodic sampling is used for human body sensors to capture instantaneous changes in human activity, while a moderate sampling frequency is used for temperature and humidity sensors to stably acquire environmental state parameters, resulting in the raw signal set. The raw signals are processed according to a unified encapsulation format, which includes a device identification field to indicate the data source, a high-precision timestamp to reflect the time sequence of data generation, a data value field to record the sensor's detection results, and the device's position parameters in the three-dimensional spatial coordinate system, forming a sensor data stream.

[0022] Step 200: Construct a spatiotemporal fusion data matrix based on sensor data streams; Specifically, the position information of each sensor in the three-dimensional spatial coordinate system is extracted from the sensor data stream one by one. The Euclidean distance between any two sensors is calculated using the spatial coordinates as a parameter to measure the physical proximity between sensors, reflecting the correlation of different sensors in spatial layout. The Euclidean distance is transformed using exponential decay or other spatial weighting formulas to obtain a spatial weight matrix describing the degree of spatial correlation between sensors within the region. The larger the value of each element in the matrix, the closer the spatial connection between the two sensors. The sensor data stream is type-identified, with the outputs of door magnetic sensors and smoke detectors marked as event-triggered data, the outputs of human body sensors marked as first-frequency data, and the outputs of temperature and humidity sensors marked as second-frequency data. For data with different characteristics, a spatial topology sensing temporal alignment algorithm is called to finely calibrate the sampling time sequence through cubic spline interpolation. This aligns multi-source data with inconsistent sampling frequencies or irregular trigger time distributions at a unified time granularity, generating a temporal calibration dataset with a consistent time axis. The temporal calibration dataset is fused with a spatial weight matrix. During the calculation, the data from each sensor are weighted and summed according to their spatial weights to generate a spatially weighted fusion result that comprehensively considers spatial location relationships and temporal consistency. For any missing data in the fusion result, a prediction algorithm based on the spatial correlation of neighboring sensors is used to perform numerical inference and imputation using data from nearby and highly correlated sensors. This ensures that the data matrix remains continuous and complete in both temporal and spatial dimensions, forming a spatiotemporal fused data matrix.

[0023] Step 300: Input the spatiotemporal fusion data matrix into the edge layer quantization neural network, the fog layer recurrent neural network, and the cloud layer large model for distributed inference to obtain the inference result set; Specifically, the spatiotemporal fusion data matrix is ​​input into a quantized neural network model deployed in the edge layer. This model accelerates computation and compresses storage through low-bit quantization of parameters and weights, effectively reducing latency and energy consumption while maintaining recognition accuracy. The edge-layer quantized neural network performs multi-layer convolution and non-linear mapping operations on the input data matrix, extracting multi-dimensional feature vectors reflecting the real-time state of the region. Combined with historical data sequences, it predicts scene change trends, generating edge-layer inference results that can directly drive local device responses. The spatiotemporal fusion data matrix is ​​then input into a recurrent neural network located in the fog layer. This recurrent neural network uses long short-term memory units or a bidirectional structure to capture temporal-dependent features in the data. Through recursive processing of continuous data segments, it identifies user behavior patterns and interaction sequences in the region, outputting fog-layer inference results containing behavioral labels, probability of occurrence, and duration. Simultaneously, the spatiotemporal fusion data matrix is ​​transmitted to a large-scale pre-trained model running in the cloud. This model performs deep analysis using big data from across rooms and time periods, performs preference learning tailored to different guest characteristics, and uses the predictive analysis results to generate globally optimal control and resource allocation strategies, outputting cloud-layer inference results with personalized and globally optimized characteristics. The real-time device control commands in the edge layer inference results, the behavior recognition information in the fog layer inference results, and the optimization strategies in the cloud layer inference results are integrated according to a unified data structure to form an inference result set covering three types of information: real-time control, behavior understanding, and long-term optimization.

[0024] Step 400: Combine the inference result set to perform priority queue scheduling and zero-copy transmission for emergency events, important events and routine events, and generate a sequence of control instructions.

[0025] Specifically, based on the risk and timeliness level corresponding to each inference instruction in the inference result set, door magnetic triggers and smoke alarms are categorized as emergency events, human body induction responses as important events, and temperature and humidity regulation as routine events. Priority tags and timestamps are written into these events before they enter the scheduling system. A hierarchical queue is established at the edge, and token buckets are used for dynamic resource allocation: higher token quotas and shorter processing times are set for emergency events. Combined with the depth and time limit parameters of the P0, P1, and P2 queues (e.g., P0 queue depth approximately 10, typical processing time approximately 5 milliseconds; P1 queue depth approximately 50, typical processing time approximately 20 milliseconds; P2 queue depth approximately 200, typical processing time approximately 100 milliseconds), the token ratio is flexibly adjusted based on real-time load (e.g., increasing P0 allocation when the proportion of emergency events is high) to ensure that high-risk events can still be prioritized for clearing and queue jitter suppressed under peak concurrency. During the event transport and disk write phase, the entire link employs non-blocking I / O and multiplexed listening. When the scheduler detects that the target event data is ready, it directly calls the sendfile function to complete the data page transfer in kernel space, bypassing user-space buffer copying. This reduces context switching and copy overhead, and, combined with the cache hit advantage brought by zero-copy transmission, ensures stable millisecond-level end-to-end response. During the network fast forwarding and downlink control issuance phase, the control instruction sequence, which has completed zero-copy processing and includes priority markers and execution windows, is injected into the user-space high-speed forwarding stack. Combined with DPDK's polling-style transmission and reception and huge-page memory mechanism, the network interface queue directly connected to the target device enables data packets to be sent with minimal protocol stack overhead, forming a short-path closed loop of "scheduling-transmission-execution." Simultaneously, a fast preemption strategy for urgent events and a batch merging strategy for regular events are retained to maintain optimal coordination between overall throughput and critical latency. The output control instruction sequence is structured and encapsulated with timestamp, priority, target device identifier, and execution result as fields.

[0026] In one specific embodiment, the process of performing step 100 may specifically include the following steps: Construct a three-dimensional spatial coordinate system for the intelligent area, and based on the three-dimensional spatial coordinate system, map the door magnetic sensor, human body sensor, temperature and humidity sensor and smoke detector to the length, width and height coordinate positions to obtain the device position matrix; A dual-buffer data storage architecture, including a primary buffer and a secondary buffer, is configured based on the device location matrix. Based on the dual-buffer data storage architecture, event-triggered sampling is performed on the door magnetic sensor and smoke detector, frequency sampling is performed on the human body sensor, and frequency sampling is performed on the temperature and humidity sensor to obtain the original signal set; The original signal set is encapsulated according to the format of device identifier, timestamp, data value and spatial coordinate to obtain sensor data stream.

[0027] Specifically, a three-dimensional spatial coordinate system is established within the physical space of the intelligent area, using the area's length, width, and height as reference axes. Digital modeling maps the area's building plan and facade structure into a mathematical model. Within this three-dimensional coordinate system, the actual installation positions of door magnetic sensors, human body sensors, temperature and humidity sensors, and smoke detectors are calibrated. The installation point of each sensor is converted into corresponding length, width, and height coordinate values, which are then stored sequentially to form a device position matrix. This matrix characterizes the spatial distribution of each sensor. After obtaining the device position matrix, a dual-buffer data storage architecture, including a main buffer and a secondary buffer, is configured. The main buffer receives and writes data collected by the sensors in real time, while the secondary buffer is used by the data processing thread to read and analyze the collected data. A circular queue structure enables continuous data storage within the buffers. The main and secondary buffers switch at fixed time intervals, such as every 50 milliseconds, to achieve parallel operation of data acquisition and processing, ensuring continuous data reception and real-time processing. Regarding the data acquisition strategy, for door magnetic sensors and smoke detectors, which primarily detect changes in state, an event-triggered sampling mode is adopted, acquiring and uploading data immediately upon the occurrence of a state change. For human body sensors, which need to capture details of human activity, a high-frequency fixed sampling mode is used to improve the sensitivity of activity detection. For temperature and humidity sensors, which monitor objects that change relatively slowly, a low-frequency fixed sampling mode is used to reduce data redundancy. During the acquisition process, all sensor output data is accompanied by a timestamp. In the data encapsulation stage, the acquired raw signal set is encapsulated in a unified structured format. The encapsulation format includes a device identification field, an acquisition timestamp field, a data value field, and a corresponding spatial coordinate field, used to identify the data source device, record the acquisition time, store the detection results, and specify the source location of the data in three-dimensional space. The sensor data stream formed after encapsulation simultaneously possesses time attributes, spatial attributes, and measurement value attributes.

[0028] The process includes: performing cubic spline interpolation using a spatial topology-aware temporal alignment algorithm; and multi-sensor clock synchronization calibration processing based on Network Time Protocol (NTP). This includes: deploying a high-precision clock source as a reference time server in an intelligent regional IoT gateway; sending clock synchronization signals per second to door magnetic sensors, human body sensors, temperature and humidity sensors, and smoke detectors via NTP; calculating the round-trip delay and clock offset of each sensor node relative to the reference clock; establishing a sensor clock deviation matrix; performing clock calibration calculations on each sensor node based on the sensor clock deviation matrix; estimating the clock drift rate and initial offset using the least squares method; establishing a sensor clock correction function; converting the local timestamps of each sensor to a unified network timestamp; and obtaining clock synchronization calibration parameters; and using the clock synchronization calibration parameters to perform clock calibration on the door magnetic sensor. The event trigger timestamps of sensors and smoke detectors, the first frequency sampling timestamp of human body sensors, and the second frequency sampling timestamp of temperature and humidity sensors are uniformly calibrated to eliminate time deviations caused by clock asynchrony and generate standardized timestamp data. The standardized timestamp data is used as the time reference for the cubic spline interpolation algorithm. By constructing a piecewise cubic polynomial function, precise interpolation calculations are performed on the time axis for sensor data with different sampling frequencies to ensure strict alignment of all sensor data in the time dimension and produce high-precision time alignment results. A clock synchronization quality monitoring mechanism is established to continuously monitor the clock deviation changes and synchronization accuracy of each sensor node. When a clock deviation exceeding the 1ms threshold is detected, a recalibration process is automatically triggered. By dynamically adjusting the synchronization period and calibration parameters, the long-term stability and accuracy of the time alignment algorithm are guaranteed.

[0029] In one specific embodiment, the process of configuring a dual-buffer data storage architecture including a primary buffer and a secondary buffer based on the device location matrix may specifically include the following steps: Calculate the buffer configuration parameters based on the number of sensors and their spatial distribution density in the device location matrix; Based on the buffer configuration parameters, a main buffer responsible for real-time data writing operations and a secondary buffer responsible for data reading and processing operations are created respectively. A timed switching mechanism is implemented for the main buffer and the secondary buffer. When the main buffer performs a write operation, the secondary buffer performs a read operation synchronously. After the switching time arrives, the read and write roles of the main buffer and the secondary buffer are interchanged, thus obtaining the buffer switching control mechanism. The buffer switching control mechanism is integrated with the main buffer and the secondary buffer to form a dual-buffer data storage architecture.

[0030] Specifically, the buffer configuration parameters are calculated based on the number of sensors recorded in the device location matrix and the sensor distribution density in the three-dimensional spatial coordinate system. The device location matrix contains the length, width, and height spatial coordinates of each sensor, and also reflects the spatial layout relationship and density between different sensors. When calculating the buffer configuration parameters, the total number of sensors, the average data generation rate of a single sensor, the difference in event triggering frequency caused by spatial distribution, and the bandwidth usage under different sampling strategies are comprehensively considered. For areas with high distribution density, since the associated events between sensors are more likely to be triggered simultaneously, larger storage space and higher data write bandwidth are allocated to the corresponding buffer; for areas with low distribution density, the buffer space is relatively reduced to maintain a balance in the overall resource allocation. Through the above analysis, buffer configuration parameters including buffer capacity, number of data pages, circular queue length, write bandwidth threshold, and data refresh cycle are obtained. Based on the buffer configuration parameters, two independent buffer instances are created, where the primary buffer is used to perform real-time data write operations, and the secondary buffer is used to perform data read and processing operations. The primary buffer needs high-speed write capability and sufficient burst storage space to handle peak periods of simultaneous data uploads from multiple sensors. The secondary buffer requires efficient random and sequential read capabilities to ensure that the data processing thread can complete batch data analysis and preprocessing within a specified time. These two buffers are physically independent but logically managed by the same scheduling module, ensuring no access conflicts occur on the write and read paths. To achieve parallel operation of data acquisition and processing, a timed switching mechanism is introduced between the primary and secondary buffers. In this mechanism, the two buffers take on the roles of write and read respectively, and switch functions when a preset switching period arrives. For example, within a switching period, the primary buffer continuously receives data streams from various sensors and writes them in real time, while the secondary buffer simultaneously provides its internally cached data to the processing unit for format parsing, timing analysis, or preliminary fusion operations. When the switching time arrives, the system control module pauses the current operations of both buffers, freezes the write pointer of the primary buffer, terminates the read task of the secondary buffer, and then swaps their roles, switching the primary buffer to data read mode and the secondary buffer to data write mode. To avoid data loss or misalignment during switching, the switching control mechanism introduces bidirectional synchronization signals and buffer status flags. This ensures that role switching only occurs after both buffers have completed their current tasks, achieving seamless switching within a millisecond-level time window. The buffer switching control mechanism is integrated with the primary and secondary buffers to form a dual-buffer data storage architecture.

[0031] In one specific embodiment, the process of performing step 200 may specifically include the following steps: Extract the spatial coordinate information of each sensor in the sensor data stream and calculate the Euclidean distance between the sensors, and convert the Euclidean distance into a spatial weight matrix based on the spatial coordinate information; Event trigger data from door magnetic sensors and smoke detectors, first frequency data from human body sensors, and second frequency data from temperature and humidity sensors are identified from sensor data streams. Cubic spline interpolation is performed using a spatial topology sensing temporal alignment algorithm to generate a temporal calibration dataset. The temporal calibration dataset is fused with the spatial weight matrix to produce a spatially weighted fusion result; The missing data in the spatially weighted fusion results are filled with numerical values ​​using a spatial correlation prediction algorithm of neighboring sensors to form a spatiotemporal fusion data matrix.

[0032] Specifically, the sensor data streams are analyzed line by line to extract the length, width, and height position parameters of each sensor in the three-dimensional spatial coordinate system of the intelligent area. Using this spatial coordinate information, the Euclidean distance between any two sensors is calculated, reflecting their linear distance relationship in three-dimensional space and quantifying the physical proximity of different sensors. Based on the principle of spatial attenuation or an exponential weighting function, the Euclidean distance matrix between sensors is converted into a spatial weight matrix according to a preset attenuation model. Each element in the matrix represents the relative weight of the two sensors in the spatial fusion operation; the closer the distance, the higher the weight. The sensor data streams are then processed for data type identification and classification. Output data from door magnetic sensors and smoke detectors are identified as event-triggered data, which is generated only when the state changes and is irregularly distributed on the time axis. Output data from human body sensors is identified as first-frequency data, with a high sampling frequency to capture instantaneous activity changes. Output data from temperature and humidity sensors is identified as second-frequency data, with a lower sampling frequency to reduce redundant information. Since the inconsistency in the temporal sampling of different types of sensors can affect the joint analysis of multi-source data, a temporal alignment algorithm based on spatial topology sensing is used to perform time unification processing on data of different frequencies and triggering modes. Using a uniform time granularity (e.g., 10 milliseconds) as a benchmark, cubic spline interpolation is performed on the original data to map discontinuous event-triggered data and periodic sampling data of different frequencies onto a unified time series, generating a time-series calibration dataset. This time-series calibration dataset is then fused with a spatial weight matrix. At each unified time slice, calibration data from all sensors are weighted according to the spatial weight matrix, assigning larger influence coefficients to neighboring sensor data with higher weights, generating a spatially weighted fusion result with higher fusion density and better noise suppression. Missing portions are detected and located in the fusion result, and numerical imputation is performed using a prediction algorithm based on the spatial correlation of neighboring sensors. The prediction algorithm finds the multiple sensors closest to the location of the missing data based on the spatial weight matrix, and performs weighted averaging or regression prediction based on observations at the same time slice, generating estimated values ​​consistent with the statistical characteristics of the original data. These estimated values ​​are then filled into the corresponding positions in the fusion result, forming a spatiotemporal fusion data matrix.

[0033] In one specific embodiment, the process of performing step 300 may specifically include the following steps: The spatiotemporal fusion data matrix is ​​input into the edge layer quantization neural network for feature extraction and scene change prediction, generating edge layer inference results; The spatiotemporal fusion data matrix is ​​input into the fog layer recurrent neural network to perform behavior sequence recognition and generate fog layer inference results; The spatiotemporal fusion data matrix is ​​input into the cloud layer large model for preference learning and control optimization, and the cloud layer inference results are output. The device control commands from the edge layer inference results, the behavior recognition information from the fog layer inference results, and the optimization strategies from the cloud layer inference results are integrated to form a shape inference result set.

[0034] Specifically, the spatiotemporal fusion data matrix is ​​input into a quantized neural network model deployed at the edge layer. The quantized neural network reduces storage and computational latency by quantizing weight parameters and activation values ​​(e.g., 8-bit integer quantization), minimizing the impact on inference accuracy while leveraging the parallel computing capabilities of the edge processor for rapid local execution. The quantized neural network performs deep feature extraction on the multidimensional features of the spatiotemporal fusion data matrix, mapping raw values ​​containing sensor states, spatial distribution patterns, and temporal trends into higher-level feature vectors. These feature vectors are used to predict scene changes, identifying impending state transitions by analyzing the feature evolution across consecutive time slices, such as a change from an unmanned to a manned area, environmental parameters exceeding thresholds, or safety monitoring trigger conditions being met. This generates edge-layer inference results with immediate responsiveness, including control commands that can be directly sent to the execution device and local control parameters related to short-term predictions. Simultaneously, the spatiotemporal fusion data matrix is ​​input into a recurrent neural network model located in the fog layer. The recurrent neural network employs a Long Short-Term Memory (LSTM) or bidirectional LSTM structure to capture long-term dependencies and contextual information in the data sequence. Recurrent neural networks recursively analyze the changing trajectory of the spatiotemporal fusion data matrix within a longer time window to identify behavioral patterns and action sequences of people within the area, such as continuous entry and exit, prolonged stays, frequent activities, or interactions with the environmental control system. The core of the fog layer inference process is behavioral sequence recognition, determining the type of behavior, providing the time range and probability assessment of the behavior's occurrence, and including behavioral contextual information in the results. The spatiotemporal fusion data matrix is ​​then input into a large-scale pre-trained model running in the cloud layer for higher-level analysis. The cloud layer model boasts billions of parameters and acquires extensive pattern recognition and inference capabilities through training on large datasets spanning multiple regions, users, and scenarios. The goal of cloud layer inference is to perform preference learning and control optimization. This involves analyzing long-term data accumulation across a single region or even the entire region to uncover guests' personalized preferences regarding environmental control, security needs, and service response. Simultaneously, it combines global operational status with constraints such as energy consumption and equipment lifespan to generate multi-objective optimization strategies for equipment operation. The results from the three inference layers are integrated to form a set of inference results containing multi-layered information. Real-time device control commands are extracted from the edge layer inference results as the most direct execution signals; behavioral recognition information from the fog layer inference results is added to the result set to provide behavioral context and decision-making reference; optimization strategies from the cloud layer inference results are appended to the result set to combine real-time control with long-term optimization. Meta-information such as timestamps, source identifiers, and priority tags are included when the result set is generated.

[0035] Before prioritizing and zero-copying the inference result set for emergency, important, and routine events, the process includes: adaptive anomaly detection and multiple fault-tolerant processing of the inference result set to obtain a reliable inference result set. This involves: inputting the edge layer inference results, fog layer inference results, and cloud layer inference results from the inference result set into the isolated forest anomaly detection module; constructing 100 isolated trees to calculate the path length anomaly score for each inference result; and dynamically adjusting the anomaly detection threshold based on the current time period, with a threshold coefficient of 2.5 for daytime and 3.0 for nighttime, resulting in anomaly detection labeling results. Based on the anomaly detection labeling results, a triple-redundant voting verification mechanism is initiated for inference instructions identified as key decisions. The inference results for emergency events such as fire alarm triggering and door magnetic alarm are simultaneously distributed to three independent verification modules for parallel computation. When at least two verification modules output consistent results, the decision is considered valid; otherwise, it is marked as an anomaly decision. The system generates redundant verification results. Based on these results, it detects sensor fault states. When continuous abnormal data is detected from door magnetic sensors, human body sensors, temperature and humidity sensors, or smoke detectors, it automatically triggers a downgraded operation mode switch. A pre-built sensor association graph is used to determine backup or neighboring sensors as alternative data sources, generating a fault switching strategy. Based on the spatial association relationships of sensors in the fault switching strategy, a graph convolutional network is used to perform spatial feature propagation calculations on missing or abnormal inference data. The weighted average of three neighboring normal sensors and historical data trends are used to predict and fill in missing values. Simultaneously, a confidence coefficient is calculated for each filled value to form a data repair result. The system integrates the anomaly detection and labeling results, redundant verification results, fault switching strategy, and data repair results. Abnormal inference instructions are removed, valid instructions that have passed verification are retained, backup data after fault switching is applied, and complete data after filling and repair is generated to produce a reliable inference result set that has undergone anomaly detection and fault tolerance processing.

[0036] In one specific embodiment, the process of inputting the spatiotemporal fusion data matrix into the edge layer quantization neural network for feature extraction and scene change prediction, and generating the edge layer inference result, may specifically include the following steps: The spatiotemporal fusion data matrix is ​​input into the edge layer quantized neural network, and parallel computation is accelerated by enabling the NEON instruction set of the ARM Cortex-A72 processor to obtain the target input features; The target input features are input into the depthwise separable convolutional layer and the inverse residual module to calculate the spatial correlation features and temporal evolution features of the multimodal sensors in the region layer by layer, and the feature representation vector is obtained. The feature representation vector is used as the system state input. The Kalman filter predictive control module performs recursive state estimation to form scene change prediction data. Based on feature representation vectors and scene change prediction data, the edge layer inference results containing device control commands are output.

[0037] Specifically, the spatiotemporal fusion data matrix undergoes pre-quantization processing and memory layout optimization at the input end. Within the edge gateway, the matrix is ​​offline calibrated for zeros and scales and integer quantized, ensuring the feature tensor resides in contiguous memory pages in int8 format with an NHWC layout to match the cache level of the ARM Cortex-A72 processor. Simultaneously, DMA prefetching and circular buffering strategies are employed during data transfer to reduce memory access jitter. At the operator level, parallel computation is accelerated using the NEON vectorization kernel, enabling core paths such as convolution, matrix multiplication and addition, and activation pruning to be batch-processed with a vector width as the stride, thereby compressing computational power and bandwidth usage without sacrificing effective accuracy. To further reduce end-to-end latency, batch normalization and activation functions are integrated during model export, constant operators are folded, and training branches are removed, ensuring the explicit inference graph retains only necessary computation nodes and maps them to a customized int8 kernel, resulting in a target input feature tensor that satisfies the resource constraints of edge devices. The target input features are input into a depthwise separable convolutional layer and an inverse residual module for computation. Local spatiotemporal texture and device autocorrelation features are extracted in each channel through depthwise convolution, and pointwise convolution is used to achieve linear mixing across channels to reconstruct multimodal correlations. In the inverse residual structure, pointwise convolution is used to increase dimensionality to expand the feature subspace, and then lightweight depthwise convolution is used to model the spatial proximity relationships and time-varying trajectories between different sensors in the increased high-dimensional space. Pointwise convolution is used to reduce dimensionality and establish a short-circuit path with the input to retain low-frequency structural information. This strengthens the coupling expression across modalities while ensuring controllable computational cost. To suppress disturbances in low signal-to-noise ratio channels, lightweight attention or channel recalibration units are inserted between convolutional blocks. Higher response weights are assigned to the high-frequency variation components of human body sensors and the event pulse components of smoke detectors and door magnets, while slow-drift components of temperature and humidity are subject to smoothing constraints. Simultaneously, block parallelism and pipelined techniques are employed in operator scheduling, enabling in-chip computation of depthwise and pointwise convolutions within the on-chip cache, avoiding repeated write-back to main memory. This results in a compact feature representation vector depicting the spatial correlation and temporal evolution of multimodal sensors within the region. Combining an edge-side quantization network with a hardware acceleration path, stable high-throughput inference output is achieved on edge devices with low power consumption. The feature representation vector is used as the system state input to the Kalman filter predictive control module to perform recursive state estimation. The module internally maintains the hidden states and covariance metrics of the regional scene. At each time step, it first performs prior prediction based on the state and control variables of the previous time step, then corrects and updates based on observations triggered by the current representation vector, obtaining a continuous predictive trajectory for short-term scene changes. To adapt to the input patterns of both event-driven and periodic sampling, the covariance of observation noise and process noise is updated online adaptively: when events such as door magnets and smoke detectors are triggered intensively, the sensitivity to sudden changes is increased to achieve rapid convergence; when the main input is slow changes in temperature and humidity, the gain is reduced to suppress the amplification of observation noise, and a smooth transition is achieved in conjunction with the historical window of the edge buffer.Based on feature representation vectors and scene change prediction data, the edge layer inference results, including device control commands, are output. For security-related door magnetic triggers and smoke detector anomalies, high-priority control commands and alarm reporting parameter packages are directly generated, and emergency response markers are added to the network stack. For human body sensing-related comfort linkages, target setpoints and gradual change sequences for lighting, air conditioning, and ventilation are given based on predicted entry, exit, and stay patterns. For slow-change temperature and humidity control, energy-saving priority adjustment commands are output by combining predicted trajectories and energy consumption constraints. The edge layer inference results are packaged with a unified timestamp, priority label, device identifier, and execution window.

[0038] In one specific embodiment, the process of inputting the spatiotemporal fusion data matrix into the fog layer recurrent neural network to perform behavioral sequence recognition and generate fog layer inference results can specifically include the following steps: The spatiotemporal fusion data matrix is ​​reorganized and its feature dimensions are transformed to obtain behavioral sequence input data. The behavioral sequence input data is fed into the fog layer recurrent neural network to extract behavioral temporal features and generate a bidirectional hidden state sequence. The attention-weighted feature vector is generated by performing a sliding time window attention mechanism based on the bidirectional hidden state sequence. The attention-weighted feature vector is input into the behavior classifier to perform guest behavior pattern recognition, and the output is a fog layer inference result containing behavior recognition information.

[0039] Specifically, the spatiotemporal fusion data matrix undergoes recombination and feature dimension transformation operations, converting the original data structure arranged according to sensor spatial layout and temporal granularity into behavioral sequence input data suitable for time-series modeling. The spatiotemporal fusion data matrix is ​​arranged with time slices as rows and sensor numbers as columns, with each element containing the measurement value and its preprocessed features corresponding to the time and spatial location. The spatiotemporal fusion data matrix is ​​divided into continuous fixed-length segments along the time axis (e.g., each segment covers 30 seconds or several sampling windows), and within each segment, the feature vectors of each sensor are concatenated in a preset order, allowing the network to receive multimodal combination features reflecting the global state of the region at a single time step. Simultaneously, normalization and optional feature compression are performed during the dimension transformation process to reduce the impact of numerical scale differences on the network convergence speed, and missing values ​​are replaced with estimated values ​​filled in during the previous fusion stage. The behavioral sequence input data is then fed into a recurrent neural network deployed in the fog layer for behavioral temporal feature extraction. The fog layer recurrent neural network employs a bidirectional long short-term memory network (Bi-LSTM) or bidirectional gated recurrent unit (Bi-GRU) structure to simultaneously capture the forward and backward dependencies of the time series. During forward propagation, the network processes time steps sequentially from the beginning to the end of the sequence, encoding the preceding state information into a hidden state vector. During backward propagation, it processes time steps from the end to the beginning of the sequence, encoding the subsequent state information into another set of hidden state vectors. By concatenating the forward and backward hidden states at each time step, the resulting bidirectional hidden state sequence can simultaneously contain past and future contextual information at the same point in time. A sliding time window attention mechanism is then applied based on this bidirectional hidden state sequence to further refine the information from key time slices. The sliding time window attention mechanism slides a fixed-length time window across the hidden state sequence, calculating attention weights for the hidden states at all time steps within each window. The attention weights are calculated based on the similarity measure between the current time step and the hidden states at other time steps within the window (e.g., through dot product, affine transformations with learnable parameters, or multi-head attention structures). The weights are normalized using the Softmax function, adaptively assigning higher weights to time steps with greater behavioral discriminative power and lower weights to time steps with higher noise or information redundancy. The hidden states at each time step are summed according to their attention weights to obtain the attention-weighted feature vector for the time window. This attention-weighted feature vector is then input into the behavior classifier module to perform guest behavior pattern recognition. The behavior classifier employs a multi-layer fully connected neural network structure, combining batch normalization and non-linear activation functions to enhance feature representation capabilities, and uses Softmax activation in the output layer to achieve multi-class probability prediction. During the training phase, the classifier optimizes its weight parameters based on labeled historical behavior data, enabling it to accurately distinguish different types of guest behavior patterns, such as entering a room, leaving a room, staying briefly in a room, resting for a long time, moving frequently, and calling for service.During the inference phase, the classifier receives the feature vectors output from the attention mechanism, calculates the probability distribution of various behavioral patterns, and takes the category with the highest probability as the recognition result, while outputting the confidence score of the category and the relevant time label.

[0040] In one specific embodiment, the process of inputting the spatiotemporal fusion data matrix into the cloud big model for preference learning and control optimization, and outputting the cloud inference result, can specifically include the following steps: Semantic encoding is performed on the regional environmental state information and guest behavior characteristics in the spatiotemporal fusion data matrix to obtain a semantically encoded input sequence; The semantically encoded input sequence is input into a large cloud model for personalized behavior pattern analysis to generate guest preference features. The federated learning framework is initiated based on guest preference features to perform distributed model parameter updates, generate personalized inference models, and then generate cloud-based inference results containing optimization strategies through these personalized inference models.

[0041] Specifically, semantic encoding is performed on the regional environmental status information and guest behavior characteristics in the spatiotemporal fusion data matrix. Environmental elements such as lighting brightness, temperature, humidity, air quality index, noise level, door magnetic status, and smoke alarm signs are extracted from the spatiotemporal fusion matrix and fused with contextual information such as behavior tags, occurrence time, confidence level, and stay duration from fog layer behavior recognition results. Discretization is completed using a domain vocabulary and attribute ontology, while retaining key continuous numerical ranges and variation slopes as numerical features to form a semantic encoding input sequence. In the semantic encoding stage, time alignment and regional normalization are performed, mapping timestamps from different time zones and room types to a unified relative time axis and scaling the dimensional shifts caused by equipment differences. At the same time, a privacy minimization strategy is introduced, carrying only the statistical summary and anonymization tags necessary for completing the analysis, without returning any personally identifiable information. In terms of input sequence organization, a structure of "environmental fragment - behavior fragment - response fragment" is adopted to preserve the causal adjacency of environmental state transitions and behavior sequences within a local window. Time sequence and source domain are marked through positional and segmental encoding, resulting in an aligned, anonymized, and structured semantically encoded input sequence that enters the cloud inference channel. This semantically encoded input sequence is then fed into the cloud-based large-scale model for personalized behavior pattern analysis. A model with seven billion parameters based on the GPT architecture is deployed on the cloud side. Combined with cross-room and cross-time period data, it possesses strong pattern capture and policy generation capabilities, and periodically updates parameters in a federated manner, with a typical update cycle of 24 hours. During the inference phase, the large model uses a self-attention mechanism to globally model three types of segments—environment, behavior, and device response—on long sequences, capturing the coupling relationship between guests' preferences for lighting, air conditioning, ventilation, and security at different times and in different contexts. Based on this, the cloud outputs a multi-dimensional guest preference feature vector, including comfortable temperature range, humidity tolerance range, light preference curve, sensitivity to noise, willingness to trade off energy consumption, reaction threshold to security events, and scene-triggered preferences (e.g., "automatic soft lighting upon entering the room at night, gradual brightening in the morning, automatic energy saving when no one is in the room"). To improve robustness, preference feature calculation combines fog layer recognition confidence with short-term predictions at the edge layer. Thresholding suppresses the interference of low-confidence segments on long-term preferences, while introducing weight decay for abnormal periods (such as sudden alarms or temporary maintenance) to prevent single anomalies from dominating the global preference profile. Based on the guest preference features, a federated learning framework is initiated to perform distributed model parameter updates, continuously obtaining a personalized inference model consistent with the guest profile without aggregating the original data.Each regional gateway or floor edge node calculates its local gradient or parameter increment based on local spatiotemporal fusion data and recent execution feedback, and performs privacy protection processing such as differential pruning and noise injection locally, only sending parameter summaries to the federated coordination end. The coordination end aggregates the parameter increments from each node according to a predetermined rhythm (e.g., every 24 hours). The aggregation strategy follows a weighted principle that emphasizes both sample size and data quality, ensuring that nodes with more abundant and more stable data occupy a more reasonable weight in the global update, while robustly suppressing abnormal drift nodes. After aggregation, the global update parameters are issued, and personalized sub-model versions for the current guest or room type are derived on the cloud side to form a personalized inference model. The personalized inference model is used to generate cloud-layer inference results containing optimization strategies, and the strategies are output in structured form to the downstream scheduling module and device control channel. The optimization strategy revolves around a multi-objective trade-off: it combines preference characteristics to provide target settings and gradual change sequences in the comfort dimension (such as target temperature ranges, lighting gradient curves, and quiet periods); it combines global energy consumption and equipment lifespan constraints to provide energy-saving and maintenance-friendly operating levels, and includes scene triggering rules and priority tags when necessary. Simultaneously, the cloud inference results include three metadata items: strategy activation conditions, expected effects, and rollback plans.

[0042] In one specific embodiment, the process of performing step 400 may specifically include the following steps: Based on the urgency of the reasoning instructions, door magnetic triggering and smoke alarms are classified as emergency events, human body induction response is classified as important events, and temperature and humidity control is classified as routine events. A queue scheduling mechanism is established based on a token bucket dynamic allocation algorithm for emergency events, important events, and routine events. The system monitors event data processed by the queue scheduling mechanism. When the event data is ready, it uses the sendfile zero-copy function to perform data migration operations in kernel space, thus completing efficient data transmission processing. It acquires the control command sequence generated by efficient data transmission processing, and directly transmits the control command sequence to the target device through DPDK technology.

[0043] Specifically, based on the urgency of the inference instructions in the inference result set, events are classified and processed in a hierarchical manner. Door magnetic sensor trigger events and smoke detector alarm events are classified as emergency events, human body sensor response events as important events, and temperature and humidity control events as routine events. Before an event enters the scheduling module, a priority marker and a timestamp accurate to milliseconds are added to each event record. During initialization, a three-level priority queue is established: queue P0 handles emergency events with a depth of 10 queues and a processing time limit of 5 milliseconds per event; queue P1 handles important events with a depth of 50 queues and a processing time limit of 20 milliseconds; and queue P2 handles routine events with a depth of 200 queues and a processing time limit of 100 milliseconds. A token bucket algorithm is used to dynamically allocate tokens to the three priority queues. Initially, queue P0 receives 80% of the total tokens, queue P1 receives 15%, and queue P2 receives 5%. When a large influx of urgent events is detected within a 50-millisecond scheduling cycle, the token percentage of queue P0 is increased to 90%, while the token percentage of queue P1 is decreased to 8%, and the token percentage of queue P2 is decreased to 2%. When the number of urgent events decreases while the number of important events increases, the token percentage of queue P1 is increased to 40%, the token percentage of queue P0 is decreased to 55%, and the token percentage of queue P2 remains at 5% within a scheduling cycle. After an event is delivered to the corresponding priority queue and passes threshold checks and quota verification, the I / O subsystem detects the readiness status of the event data in non-blocking listening mode. When the event data is ready, the sendfile() function is called to perform a zero-copy data migration operation in kernel space, directly transferring the file descriptor data page to the socket buffer, avoiding two data copies between user space and kernel space. To optimize the zero-copy transmission path, a large page memory pool with a page size of 2MB is allocated to reduce page table switching overhead, and the maximum batch size of the send buffer is set to 64 data packets to ensure that the transmission latency remains stable at the millisecond level under high concurrency scenarios. After zero-copy data transfer, the control command sequence directly enters the DPDK user-space protocol stack for high-speed forwarding. On the DPDK side, queues P0, P1, and P2 are each bound to an independent transmission queue. Queue P0 is bound to queue 0 and assigned the highest-priority processing thread; queue P1 is bound to queue 1 and assigned a medium-priority processing thread; and queue P2 is bound to queue 2 and assigned a low-priority batch processing thread. DPDK operates in polling-driven mode, with each transmission queue fixed to an independent CPU core to reduce context switching. The batch processing threshold for queue P0 is set to 16 packets for immediate transmission, queue P1 to 32 packets to balance throughput and latency, and queue P2 to 64 packets to improve link utilization through latency aggregation.The generated control command sequence adopts a structured encapsulation format, including an event category field (emergency event, important event, routine event), a priority level field (P0, P1, P2), a target device identifier field, an execution time window field (in milliseconds), a retry policy field (3 retries on failure, 50 milliseconds retry interval), and a transmission delay sample value field. Based on the structured format, the control command sequence directly reaches the target device through the DPDK protocol stack, achieving millisecond-level response.

[0044] This also includes: dynamic allocation optimization of computing resources based on deep Q-networks, including: constructing a system state space containing CPU utilization, memory usage, network bandwidth, task queue length, and energy consumption levels; defining an action space containing CPU frequency adjustment range of 0.8-2.4GHz, memory allocation range of 256MB-1GB, and network bandwidth range of 100Kbps-10Mbps; and establishing a reinforcement learning environment model; designing a composite reward function, which weights system performance rewards, energy consumption penalties, and stability rewards with weight coefficients of 0.5, 0.3, and 0.2, respectively, and applying negative average latency. Normalized calculations of negative power consumption and system runtime yield the reinforcement learning training objective. A two-layer fully connected deep Q-network is constructed with 512 and 256 hidden units, and an experience replay buffer stores 10,000 state transition samples. Training parameters are set with a batch size of 64, a learning rate of 0.001, and a discount factor of 0.95. Network weight update calculations are performed to generate a resource allocation strategy model. Based on the resource allocation strategy model, resource adjustment decisions are made every 100ms. An ε-greedy strategy is used to balance exploration and utilization, with the ε value decaying exponentially from 0.9 to 0.01. The Linux cgroup mechanism is used to dynamically adjust CPU and memory limits, and the tc command is used to control network bandwidth allocation to generate an optimized resource configuration scheme. The execution effect of the optimized resource configuration scheme is monitored, and performance indicators such as inference latency, system throughput, and energy consumption are collected in real time. Performance change trends are recorded in a time-series database to form resource optimization feedback data for continuous strategy improvement.

[0045] The acquisition of control command sequences generated by efficient data transmission processing also includes edge-side predictive caching and hotspot scene identification processing. This includes: performing scene type identification and analysis on the control command sequences generated by efficient data transmission processing, classifying high-frequency operations such as guest entering a room, guest leaving a room, temperature adjustment activation, and automatic lighting control as hotspot scenes, calculating the access frequency and response time of each scene type through statistical analysis, and establishing a scene priority ranking table; configuring 2GB of edge-side cache space based on the scene priority ranking table, using the LRU cache replacement algorithm to manage cache entries, pre-storing control commands for hotspot scenes with an access frequency exceeding 85% and their corresponding inference results in the cache, setting the cache entry expiration time to 300 seconds, and obtaining a hotspot scene cache database; building... A cache hit rate monitoring mechanism is established. When a sensor trigger within the region matches a pre-stored scene in the hot scene cache database, control commands are returned directly from the cache without re-executing inference calculations, keeping the cache hit response latency within 5ms and generating cache hit statistics. The cache strategy is dynamically adjusted based on the cache hit statistics. When the hit rate of a certain scene type continuously decreases, its cache priority is automatically reduced. When a new high-frequency scene appears, it is added to the cache in a timely manner. Scene access trends are calculated through a sliding time window to maintain the timeliness of cached content. The fast response results of cache hits are integrated with the real-time inference results of new scenes to ensure that the control command sequence includes both the efficient response of pre-cached scenes and the accurate control of real-time calculations, forming an optimized control command sequence with a hybrid response mode.

[0046] The above describes the implementation method of the multimodal industrial IoT smart gateway based on edge computing in the embodiments of this application. The following describes the multimodal industrial IoT smart gateway based on edge computing in the embodiments of this application. The multimodal industrial IoT smart gateway based on edge computing in the embodiments of this application is used to execute any of the above-mentioned implementation methods of the multimodal industrial IoT smart gateway based on edge computing.

[0047] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0048] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0049] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for implementing a multimodal industrial IoT smart gateway based on edge computing, characterized in that, include: Acquire sensor data streams from door magnetic sensors, human body sensors, temperature and humidity sensors, and smoke detectors within the intelligent area; A spatiotemporal fusion data matrix is ​​constructed based on the sensor data stream; The spatiotemporal fusion data matrix is ​​input into the edge layer quantization neural network, the fog layer recurrent neural network, and the cloud layer large model for distributed inference to obtain an inference result set. Based on the inference result set, priority queue scheduling and zero-copy transmission are performed on emergency events, important events, and routine events to generate a sequence of control instructions.

2. The method for implementing a multimodal industrial IoT smart gateway based on edge computing according to claim 1, characterized in that, The acquisition of sensor data streams from the door magnetic sensor, human body sensor, temperature and humidity sensor, and smoke detector within the intelligent area includes: A three-dimensional spatial coordinate system for the intelligent area is constructed, and based on the three-dimensional spatial coordinate system, the door magnetic sensor, human body sensor, temperature and humidity sensor and smoke detector are mapped to the length, width and height coordinate positions to obtain the device position matrix; Based on the device location matrix, a dual-buffer data storage architecture including a main buffer and a secondary buffer is configured; Based on the dual-buffer data storage architecture, event-triggered sampling is performed on the door magnetic sensor and smoke detector, frequency sampling is performed on the human body sensor, and frequency sampling is performed on the temperature and humidity sensor to obtain the original signal set; The original signal set is encapsulated according to the format of device identifier, timestamp, data value and spatial coordinate to obtain sensor data stream.

3. The method for implementing a multimodal industrial IoT smart gateway based on edge computing according to claim 2, characterized in that, The dual-buffer data storage architecture, which includes a primary buffer and a secondary buffer, is configured based on the device location matrix, including: Calculate the buffer configuration parameters based on the number of sensors and spatial distribution density in the device location matrix; Based on the buffer configuration parameters, a main buffer responsible for real-time data writing operations and a secondary buffer responsible for data reading and processing operations are created respectively. A timed switching mechanism is implemented for the main buffer and the secondary buffer. When the main buffer performs a write operation, the secondary buffer performs a read operation synchronously. After the switching time arrives, the read and write roles of the main buffer and the secondary buffer are interchanged, thus obtaining the buffer switching control mechanism. The buffer switching control mechanism is integrated with the main buffer and the secondary buffer to form a dual-buffer data storage architecture.

4. The method for implementing a multimodal industrial IoT smart gateway based on edge computing according to claim 1, characterized in that, The construction of the spatiotemporal fusion data matrix based on the sensor data stream includes: Extract the spatial coordinate information of each sensor in the sensor data stream and calculate the Euclidean distance between the sensors, and convert the Euclidean distance into a spatial weight matrix based on the spatial coordinate information; The event trigger data of the door magnetic sensor and smoke detector, the first frequency data of the human body sensor and the second frequency data of the temperature and humidity sensor are identified from the sensor data stream. Cubic spline interpolation is performed using the spatial topology sensing temporal alignment algorithm to generate a temporal calibration dataset. The time-series calibration dataset is fused with the spatial weight matrix to produce a spatially weighted fusion result. The missing data in the spatially weighted fusion result is filled with numerical values ​​using a neighboring sensor spatial correlation prediction algorithm to form a spatiotemporal fusion data matrix.

5. The method for implementing a multimodal industrial IoT smart gateway based on edge computing according to claim 4, characterized in that, The process involves inputting the spatiotemporal fusion data matrix into an edge layer quantization neural network, a fog layer recurrent neural network, and a cloud layer large model for distributed inference, resulting in an inference result set, including: The spatiotemporal fusion data matrix is ​​input into the edge layer quantization neural network for feature extraction and scene change prediction, generating edge layer inference results; The spatiotemporal fusion data matrix is ​​input into the fog layer recurrent neural network to perform behavior sequence recognition and generate fog layer inference results. The spatiotemporal fusion data matrix is ​​input into the cloud big model for preference learning and control optimization, and the cloud inference results are output. The device control commands in the edge layer inference results, the behavior recognition information in the fog layer inference results, and the optimization strategies in the cloud layer inference results are integrated to form an inference result set.

6. The method for implementing a multimodal industrial IoT smart gateway based on edge computing according to claim 5, characterized in that, The step of inputting the spatiotemporal fusion data matrix into the edge layer quantization neural network for feature extraction and scene change prediction, and generating edge layer inference results, includes: The spatiotemporal fusion data matrix is ​​input into the edge layer quantization neural network, and parallel computation is accelerated by enabling the NEON instruction set of the ARM Cortex-A72 processor to obtain the target input features; The target input features are input into the depth-separable convolutional layer and the inverse residual module to calculate the spatial correlation features and temporal evolution features of the multimodal sensors in the region layer by layer, so as to obtain the feature representation vector; The feature representation vector is used as the system state input. The Kalman filter prediction control module performs recursive state estimation to form scene change prediction data. Based on the feature representation vector and the scene change prediction data, the edge layer inference result containing device control instructions is output.

7. The method for implementing a multimodal industrial IoT smart gateway based on edge computing according to claim 6, characterized in that, The step of inputting the spatiotemporal fusion data matrix into a fog layer recurrent neural network to perform behavior sequence recognition and generate fog layer inference results includes: The spatiotemporal fusion data matrix is ​​reorganized and its feature dimensions are transformed to obtain behavioral sequence input data; The behavioral sequence input data is input into a fog layer recurrent neural network to extract behavioral temporal features and generate a bidirectional hidden state sequence. Based on the bidirectional hidden state sequence, a sliding time window attention mechanism is performed to generate an attention-weighted feature vector; The attention-weighted feature vector is input into the behavior classifier to perform guest behavior pattern recognition, and the output is a fog layer inference result containing behavior recognition information.

8. The method for implementing a multimodal industrial IoT smart gateway based on edge computing according to claim 7, characterized in that, The process of inputting the spatiotemporal fusion data matrix into a large cloud model for preference learning and control optimization, and outputting cloud inference results, includes: Semantic encoding is performed on the regional environmental state information and guest behavior characteristics in the spatiotemporal fusion data matrix to obtain a semantically encoded input sequence; The semantically encoded input sequence is input into a cloud-based large model for personalized behavior pattern analysis to generate guest preference features. Based on the guest preference features, a federated learning framework is initiated to perform distributed model parameter updates, generating a personalized inference model, and then cloud-based inference results containing optimization strategies are generated through the personalized inference model.

9. The method for implementing a multimodal industrial IoT smart gateway based on edge computing according to claim 1, characterized in that, The process of combining the inference result set to perform priority queue scheduling and zero-copy transmission for emergency events, important events, and routine events, and generating a control instruction sequence, includes: Based on the reasoning results, the urgency of the reasoning instructions is classified into emergency events (door magnetic triggering and smoke alarm), important events (human body induction response), and routine events (temperature and humidity control). A queue scheduling mechanism is established based on the token bucket dynamic allocation algorithm applied to emergency events, important events, and regular events. The system monitors the event data processed by the queue scheduling mechanism. When the event data is ready, the sendfile zero-copy function is activated to perform data migration operations in the kernel space, thereby completing efficient data transmission processing. The control command sequence generated by the efficient data transmission processing is obtained, and the target device is directly reached based on the control command sequence using DPDK technology.

10. A multimodal industrial IoT smart gateway based on edge computing, characterized in that, This method is used to implement a multimodal industrial IoT smart gateway based on edge computing as described in any one of claims 1-9.

Images