A remote monitoring method based on edge computing and multi-source sensing data fusion

Abstract

This invention discloses a remote monitoring method for multi-source sensing data fusion based on edge computing, belonging to the field of edge computing technology. This method operates on edge computing nodes. First, it utilizes the energy distribution characteristics of high-frequency one-dimensional signals to perform physical weighted interpolation reconstruction of missing frames in low-frequency images at transient impact moments. Second, it extracts the visual texture entropy of the reconstructed frames and the physical spectral entropy of the corresponding signals, and calculates the matching confidence of the data through a consistency check function to eliminate environmental interference. Finally, differential data encoding and uploading are performed only when the data is reliable and the Euclidean distance between the current feature and the locally maintained reference state vector exceeds a threshold. This invention solves the spatiotemporal semantic uncertainty problem of heterogeneous sensing data at the edge by synergistically combining temporal reconstruction, spatial verification, and semantic differential transmission, achieving high-fidelity monitoring under low bandwidth conditions.

Description

Technical Field

[0001] This invention relates to the field of edge computing technology, and in particular to a remote monitoring method based on edge computing for multi-source sensing data fusion. Background Technology

[0002] In industrial remote monitoring, high-frequency one-dimensional signals and low-frequency two-dimensional images are typically used for heterogeneous perception of equipment status. However, existing technologies face a core contradiction: the conflict between the spatiotemporal attributes of heterogeneous data and the physical limitations of transmission bandwidth. Specifically, the asynchronous sampling times of high-frequency signals and low-frequency images prevent the physical and visual features of transient faults from aligning in the temporal domain; while unstructured interference from the environment (such as lighting and electromagnetic noise) renders single-modal data lacking in reliability in the spatial domain. Existing edge processing methods often address these issues piecemeal, lacking a unified mechanism that can simultaneously complete missing features in the temporal domain, remove interference noise in the spatial domain, and filter redundant information semantically. This makes it difficult for remote monitoring systems to acquire accurate and low-volume effective data when facing transient and complex operating conditions. Summary of the Invention

[0003] This invention provides a remote monitoring method based on edge computing for multi-source sensing data fusion, aiming to solve the technical problem in the prior art that edge nodes have difficulty acquiring and transmitting deterministic monitoring data due to the mismatch of spatiotemporal attributes of heterogeneous data and environmental interference.

[0004] In view of the above problems, the present invention provides a remote monitoring method for multi-source sensing data fusion based on edge computing. This method runs on an edge computing node configured with a spatiotemporal cognitive collaborative bus, which resides in the memory of the edge node. The method includes the following steps: S1. Reconstruction steps based on energy distribution: Real-time monitoring of one-dimensional signal sequences and two-dimensional image sequences written into the heterogeneous tensor pool of the spatiotemporal cognitive collaborative bus; When the amplitude of the one-dimensional signal sequence is detected to exceed the first threshold, and the moment is located between two adjacent frames of the two-dimensional image sequence, the interpolation weight coefficient of that moment relative to the two adjacent frames is calculated. Based on the interpolation weight coefficients, a weighted interpolation operation is performed on the two adjacent frames to generate the reconstructed image frame at that moment, and the reconstructed image frame is written back to the heterogeneous tensor pool. S2. Arbitration steps based on entropy verification: Extract the visual texture entropy of the reconstructed image frame and calculate the physical spectral entropy of the one-dimensional signal sequence within the corresponding time window; Substitute the visual texture entropy and the physical spectrum entropy into the physical consistency verification function to calculate the matching confidence. The matching confidence score is stored in the confidence register configured in the spatiotemporal cognitive collaboration bus; S3. Transmission steps based on shadow state: Read the matching confidence score from the confidence register; If the matching confidence is higher than the second threshold, then the Euclidean distance between the current feature vector and the locally stored reference state vector is calculated. If the Euclidean distance exceeds the third threshold, the difference data between the current feature vector and the reference state vector is encoded and uploaded, and the reference state vector is updated using the current feature vector.

[0005] The present invention also provides a remote monitoring system for multi-source sensing data fusion based on edge computing, comprising: an acquisition module configured to acquire the one-dimensional signal sequence and the two-dimensional image sequence; a memory configured to store a computer program and construct the spatiotemporal cognitive collaborative bus; and a processor connected to the acquisition module and the memory, configured to execute the computer program to implement the above method.

[0006] The technical solution provided in this application has at least the following technical effects: This invention implements a deterministic acquisition and transmission mechanism for heterogeneous sensing data by constructing a spatiotemporal cognitive collaborative bus at edge computing nodes. This method utilizes the physical energy continuity of high-frequency signals to correct the temporal sampling defects of low-frequency images; utilizes the physical consistency of multimodal entropy values ​​to correct the spatial disturbance rejection defects of single-modality data; and utilizes the difference comparison of local reference state vectors to correct the bandwidth utilization defects of blind transmission. These three steps are interdependent and jointly ensure that the data finally uploaded to the cloud contains complete transient fault characteristics while eliminating environmental noise and redundant information. This systematically solves the spatiotemporal semantic uncertainty problem of heterogeneous sensing data at the edge, improving the effectiveness and real-time performance of remote monitoring.

[0007] This invention establishes a heterogeneous entropy mapping relationship between one-dimensional time-domain signals and two-dimensional spatial-domain images by introducing a physical consistency verification function. It utilizes objective entropy distance to quantify the causal correlation between physical impacts (vibration / sound) and visual changes (image texture). This effectively filters out false positives caused by either 'audible but invisible' (e.g., internal noises from equipment without external appearance changes) or 'visible but silent' (e.g., changes in external lighting or camera shake). By coupling the confidence level to the transmission control layer, this invention achieves bandwidth compression at the edge. Uplink bandwidth is only consumed when physical and visual features exhibit a strong correlation in time, space, and information content, and significant state drift occurs. This reduces the invalid data transmission rate by an order of magnitude while ensuring monitoring accuracy. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating a remote monitoring method based on edge computing and multi-source sensing data fusion, provided in an embodiment of the present invention. Detailed Implementation

[0009] The above technical solutions will now be described in detail with reference to the accompanying drawings and specific embodiments to provide a better understanding of them. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. It should be understood that the present invention is not limited to the exemplary embodiments used only to explain the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. Furthermore, it should be noted that, for ease of description, only the parts related to the present invention are shown in the drawings, not all of them.

[0010] For examples, please refer to Figure 1 This invention provides a remote monitoring method based on edge computing and multi-source sensing data fusion. The method operates on an edge computing node configured with a spatiotemporal cognitive collaborative bus, and performs closed-loop processing on heterogeneous one-dimensional signal sequences and two-dimensional image sequences through the spatiotemporal cognitive collaborative bus residing in the edge node's memory.

[0011] Specifically, the method includes the following core steps: First, the S1 energy distribution-based reconstruction step is executed, that is, the one-dimensional signal sequence and the two-dimensional image sequence written to the spatiotemporal cognitive collaborative bus are monitored in real time. When the amplitude of the one-dimensional signal sequence exceeds the first threshold and the moment is located between two adjacent frames of the two-dimensional image sequence, the interpolation weight coefficient of the moment relative to the two adjacent frames is calculated, and the weighted interpolation operation is performed on the two adjacent frames based on the interpolation weight coefficient to generate the reconstructed image frame and write it back to the bus. Next, the S2 arbitration step based on entropy verification is executed, which is to extract the visual texture entropy of the reconstructed image frame and the physical spectrum entropy of the corresponding one-dimensional signal sequence, substitute the two into the physical consistency verification function to calculate the matching confidence, and store it in the confidence register. Finally, the S3 shadow state-based transmission step is executed, which involves reading the matching confidence. If it is higher than the second threshold, the Euclidean distance between the current feature vector and the locally stored reference state vector is calculated. When the Euclidean distance exceeds the third threshold, the differential data is encoded, uploaded, and the reference state vector is updated.

[0012] Bus registration and time-domain reconstruction of heterogeneous data: The spatiotemporal cognitive collaborative bus first receives high-frequency one-dimensional signal sequences and low-frequency two-dimensional image sequences input in parallel by the acquisition module. After the heterogeneous tensor pool of the spatiotemporal cognitive collaborative bus receives the high-frequency one-dimensional signal sequences and low-frequency two-dimensional image sequences, the edge computing nodes immediately perform time synchronization operations, assigning nanosecond-level timestamps to each sampling point in the high-frequency one-dimensional signal sequence and each frame in the low-frequency two-dimensional image sequence, and writing the timestamped high-frequency one-dimensional signal sequences and low-frequency two-dimensional image sequences into the circular storage space of the heterogeneous tensor pool in chronological order. As the high-frequency one-dimensional signal sequence is continuously written into the heterogeneous tensor pool, the edge computing nodes monitor the amplitude changes of the high-frequency one-dimensional signal sequence in real time. When the absolute value of the amplitude of the high-frequency one-dimensional signal sequence at a specific moment exceeds a preset first threshold, the edge computing node determines that specific moment as a transient impact moment.

[0013] Next, the edge computing node retrieves the low-frequency two-dimensional image sequence from the heterogeneous tensor pool and determines whether the transient impact moment is located between two adjacent frames of the low-frequency two-dimensional image sequence. If the timestamp of the transient impact moment is greater than the timestamp of a certain previous frame in the low-frequency two-dimensional image sequence and less than the timestamp of the immediately following frame, the edge computing node initiates the reconstruction calculation process based on energy distribution.

[0014] To obtain accurate interpolation weighting coefficients During the process, the edge computing nodes perform the following specific integral operations: The edge computing nodes first determine the moment of the transient impact. And lock the moment of the immediately preceding frame on the timeline. and the next frame of the image .

[0015] Edge computing nodes define the amplitude function of a high-frequency one-dimensional signal sequence as follows: Edge computing nodes compute from arrive Within the time interval The definite integral is used as the numerator; simultaneously, the definite integral from... arrive Within the time interval The definite integral is used as the denominator. Edge computing nodes utilize the formula... The energy distribution weights are calculated. Then, the edge computing nodes invoke image processing algorithms, using the formula... Generate reconstructed image frames ,in and These are the previous and next image frames, respectively. Finally, the edge computing node adds the same timestamp as the moment of the transient impact to the reconstructed image frame and writes it into the heterogeneous tensor pool of the spatiotemporal cognitive collaborative bus for subsequent arbitration calls.

[0016] Entropy verification arbitration for multimodal physical consistency: After the reconstructed image frame is stored in the heterogeneous tensor pool of the spatiotemporal cognitive collaborative bus, the edge computing node immediately initiates the multimodal entropy consistency arbitration process. The edge computing node first reads the reconstructed image frame and a high-frequency one-dimensional signal sequence segment aligned with the timestamp of the reconstructed image frame from the heterogeneous tensor pool. The length of this sequence segment corresponds to the interval between image frames, or preferably a fixed 1024 or 2048 sampling points. Before feature calculation, the edge computing node performs preprocessing operations on the read data: converting the reconstructed image frame into a grayscale image matrix and mapping the pixel values ​​of the grayscale image matrix to a floating-point range of 0 to 1; simultaneously, truncating the high-frequency one-dimensional signal sequence segment using a Hanning window function to reduce spectral leakage.

[0017] After preprocessing, the edge computing nodes perform normalized extraction of heterogeneous entropy features. For the reconstructed image frame, the edge computing nodes use a sliding window algorithm (preferably with a sliding window size of 5×5 to 9×9 pixels) to calculate the local information entropy of the grayscale image matrix, and then average all the calculated local information entropies to obtain the original visual texture entropy. Subsequently, the edge computing nodes divide the original visual texture entropy by the maximum theoretical entropy value corresponding to the image bit depth to obtain the normalized first entropy value. For a high-frequency one-dimensional signal sequence segment, the edge computing node performs a Fast Fourier Transform (FFT) to convert the time-domain signal into a frequency-domain amplitude spectrum, and calculates the information entropy of this frequency-domain amplitude spectrum to obtain the original physical spectrum entropy. The edge computing node then divides the original physical spectrum entropy by the maximum theoretical entropy value corresponding to the signal sampling length to obtain a normalized second entropy value. .

[0018] After obtaining the first entropy value With the second entropy value Subsequently, the edge computing node accesses the historical state data stored in the spatiotemporal cognitive collaboration bus and performs a recursive update of the historical consistency momentum. The edge computing node calculates the first entropy value at the current moment. With the second entropy value After adjustment coefficient The absolute value of the weighted difference is used as the modal conflict value at the current moment. To quantify the system's stability over long-term operation, edge computing nodes use the Exponentially Weighted Moving Average (EWMA) algorithm to update the historical conflict mean. .

[0019] Specifically, edge computing nodes read the historical average of collisions from the previous moment. Using the formula Update, among which This is a preset decay factor. Subsequently, the edge computing node defines the historical consistency momentum as... (Assuming conflict values ​​have been normalized, and when) The time truncation is 1). This definition indicates that, The value is positively correlated with the stability of modal matching within a preset time window: the smaller the conflict, the greater the momentum.

[0020] Based on updated historical consistent momentum and the first entropy value at the current moment. Second entropy value Edge computing nodes are substituted into the physical consistency verification function to calculate the matching confidence. The specific mathematical expression of this physical consistency verification function is as follows: In this formula, This is the normalized visual texture entropy (first entropy value). The normalized physical spectrum entropy (second entropy value); It characterizes the degree of modal conflict between visual and physical signals at the current moment. The absolute value of the modal conflict degree ensures that no matter which modality has a higher entropy value, as long as the difference between the two increases, the conflict term will increase. The historical consistent momentum calculated in the above steps; It is the basic tolerance constant, used to set the minimum allowable noise floor of the system; It is a non-zero constant used to prevent division by zero anomalies; This is a preset adjustment coefficient. After calculation, the edge computing nodes will be matched with the confidence level. Write it into the confidence register of the spatiotemporal cognitive collaboration bus.

[0021] Shadow transmission based on bidirectional state synchronization: During system initialization or initial startup, the edge computing node resets the shadow state buffer in the spatiotemporal cognitive collaboration bus to a zero vector, or forces a full feature upload to establish an initial reference state vector.

[0022] Subsequently, after completing the arbitration calculations and matching confidence levels... After being stored in the confidence register, the edge computing node enters the transmission decision-making phase. The edge computing node first reads the matching confidence from the confidence register of the spatiotemporal cognitive collaboration bus. And perform a deadlock logic check for physical blocking. The edge computing node will read the matching confidence score. Compare the numerical value with a preset second threshold. If a match is found, the confidence level is... If the value is less than the second threshold, the edge computing node determines that the current reconstructed image frame is environmental interference or invalid data, directly releases the heterogeneous tensor pool memory space occupied by the reconstructed image frame, and no longer performs subsequent feature extraction and transmission calculations, thereby physically blocking invalid data from occupying transmission bandwidth.

[0023] If the confidence level matches If the value is greater than or equal to the second threshold, the edge computing node continues to perform the cognitive distance calculation and dead zone determination process. The edge computing node calls a feature extraction algorithm (such as the output of the global average pooling layer of the lightweight convolutional neural network MobileNetV3, or the traditional Histogram of Oriented Gradients (HOG) feature) to calculate the reconstructed image frame and generate the current feature vector.

[0024] Subsequently, the edge computing node accesses the shadow state buffer of the spatiotemporal cognitive collaborative bus and reads the reference state vector stored therein. The edge computing node calculates the Euclidean distance between the current feature vector and the reference state vector as the cognitive distance quantifying the degree of difference between the two. The edge computing node compares this cognitive distance with a preset third threshold. If the cognitive distance is less than or equal to the third threshold, the edge computing node determines that the current device state has not changed significantly relative to the known state in the cloud, terminates the data upload process, and only sends a heartbeat keep-alive signal. Only when the cognitive distance is greater than the third threshold does the edge computing node perform differential data encoding and state synchronization update operations. The edge computing node subtracts the reference state vector from the current feature vector to obtain the feature difference vector. The edge computing node uses an entropy encoding algorithm to compress and encode this feature difference vector, generates a transmission data packet, and sends it to the remote cloud through the network interface. After confirming that the transmission data packet has been successfully sent, the edge computing node uses the current feature vector to overwrite the original reference state vector in the shadow state buffer of the spatiotemporal cognitive collaborative bus, completing the local shadow state synchronization update.

[0025] System hardware-level collaborative work: The operation of this method relies on precise hardware-level collaboration between edge computing nodes and heterogeneous acquisition modules. Edge computing nodes have built-in physical memory units, which are logically divided into an operating system area, an application area, and a dedicated data area. The spatiotemporal cognitive collaboration bus resides in this dedicated data area and is further divided into three consecutive logical address segments. The first logical address segment is configured as a heterogeneous tensor pool, employing a first-in-first-out (FIFO) ring data structure. Its memory capacity is dynamically allocated based on the sampling rate of the high-frequency one-dimensional signal sequence and the low-frequency two-dimensional image sequence, as well as the preset time window length, to ensure that it can accommodate a complete data block containing at least two consecutive image frames and their corresponding one-dimensional signals within the time period. The second logical address segment is configured as a confidence register, occupying a fixed floating-point storage space, used for atomically reading and writing the matching confidence at the current moment. The third logical address segment is configured as a shadow state buffer, its size strictly matching the dimension of the feature vector, used for persistently storing a copy of the most recently successfully uploaded cloud shadow state vector.

[0026] To ensure the temporal alignment accuracy of the data in the heterogeneous tensor pool, the heterogeneous acquisition module employs a hardware-triggered clock synchronization mechanism. The high-frequency one-dimensional signal acquisition card and the low-frequency two-dimensional image acquisition card in the heterogeneous acquisition module are connected to the general-purpose input / output interface (GPIO) of the edge computing node via the same hardware trigger bus. A high-precision timer inside the edge computing node periodically sends synchronization pulse signals to the hardware trigger bus. Upon detecting the rising edge of this synchronization pulse signal, the high-frequency one-dimensional signal acquisition card and the low-frequency two-dimensional image acquisition card reset their respective internal sampling counters and latch the current nanosecond-level system time as a reference timestamp. Subsequently, the high-frequency one-dimensional signal acquisition card accumulates a timestamp based on the sampling interval for each data point acquired; the low-frequency two-dimensional image acquisition card also accumulates a timestamp based on the frame rate for each image frame exposed. This mechanism eliminates software latency caused by operating system scheduling, ensuring that the alignment error on the time axis between the high-frequency one-dimensional signal sequence and the low-frequency two-dimensional image sequence written to the heterogeneous tensor pool is less than microseconds, thus providing a reliable spatiotemporal reference for subsequent interpolation and reconstruction calculations.

[0027] System parameter configuration example: To enable those skilled in the art to better understand and implement this method, a set of parameter configuration examples verified to be effective in actual tests of this invention are provided below. It should be noted that the specific values ​​of these parameters should not be considered as limitations on the scope of protection of this invention. In the reconstruction process, the first threshold is set to 3 to 5 times the root mean square (RMS) amplitude of the high-frequency one-dimensional signal of the monitored object under normal and stable operating conditions, to ensure that reconstruction is triggered only by significant physical shocks. In the arbitration process, the adjustment coefficient of the physical consistency verification function... The recommended value range is [5, 20] (e.g., 10) to control the steepness of the Sigmoid function; adjustment coefficient The recommended value range is [0.5, 2.0] (e.g., 1.2) to balance the dimensional differences of different modal entropy values; the basic tolerance constant. The recommended value is 0.1; non-zero constant. Recommended value The attenuation factor of the historical conflict mean. The recommended value range is [0.01, 0.1] (e.g., 0.05), which means the system is affected by the stability of the past 20 to 100 time steps. In the transmission process, the second threshold (i.e., the physical blocking threshold) is recommended to be set to 0.5, meaning that when the matching confidence is below 50%, it is considered interference; the third threshold (i.e., the transmission dead zone threshold) is recommended to be set to 5% to 10% of the feature vector magnitude, meaning that when the feature change is less than this proportion, it is considered a silent state. The above thresholds and adjustment coefficients can be adaptively adjusted according to the signal-to-noise ratio of the actual application scenario, and are not limited to the example values ​​mentioned above.

[0028] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote monitoring method based on edge computing and multi-source sensing data fusion, characterized in that, This method operates on an edge computing node configured with a spatiotemporal cognitive collaborative bus, which resides in the edge node's memory. The method includes the following steps: S1. Reconstruction steps based on energy distribution: Real-time monitoring of one-dimensional signal sequences and two-dimensional image sequences written into the heterogeneous tensor pool of the spatiotemporal cognitive collaborative bus; When the amplitude of the one-dimensional signal sequence is detected to exceed the first threshold, and the moment is located between two adjacent frames of the two-dimensional image sequence, the interpolation weight coefficient of that moment relative to the two adjacent frames is calculated. Based on the interpolation weight coefficients, a weighted interpolation operation is performed on the two adjacent frames to generate the reconstructed image frame at that moment, and the reconstructed image frame is written back to the heterogeneous tensor pool. S2. Arbitration steps based on entropy verification: Extract the visual texture entropy of the reconstructed image frame and calculate the physical spectral entropy of the one-dimensional signal sequence within the corresponding time window; Substitute the visual texture entropy and the physical spectrum entropy into the physical consistency verification function to calculate the matching confidence. The matching confidence score is stored in the confidence register configured in the spatiotemporal cognitive collaboration bus; S3. Transmission steps based on shadow state: Read the matching confidence from the confidence register; If the matching confidence is higher than the second threshold, then the Euclidean distance between the current feature vector and the locally stored reference state vector is calculated. If the Euclidean distance exceeds the third threshold, the difference data between the current feature vector and the reference state vector is encoded and uploaded, and the reference state vector is updated using the current feature vector.

2. The method according to claim 1, characterized in that, The spatiotemporal cognitive collaborative bus is configured as a data structure in the memory of the edge computing node, and the data structure includes: Heterogeneous tensor pool: Used to store the timestamped one-dimensional signal sequence, the two-dimensional image sequence, and the reconstructed image frame; The confidence register is used to store the matching confidence score output in step S2. Shadow state buffer: used to store the reference state vector.

3. The method according to claim 1, characterized in that, In step S1, the interpolation weighting coefficients The calculation formula is: in, The moment when the amplitude of the one-dimensional signal sequence exceeds the first threshold. The time of the previous frame image, For the time of the next frame image, Let be the amplitude function of the one-dimensional signal sequence; The weighted interpolation operation is performed according to the formula Execution, in which For the reconstructed image frame, and These are the two adjacent frames.

4. The method according to claim 1, characterized in that, In step S2, the physical consistency verification function is: in, The matching confidence level; The normalized visual texture entropy, This refers to the normalized physical spectrum entropy; This is the historical consistency momentum, with a value of 1 minus the value within a preset time window in the past. The moving average; Based on the tolerance constant, It is a non-zero constant. This is the adjustment coefficient.

5. The method according to claim 1, characterized in that, In step S3, the specific logic of the transmission step is as follows: Compare the matching confidence level with the second threshold; When the matching confidence is less than the second threshold, the reconstructed image frame is deleted and the Euclidean distance calculation is not performed; When the matching confidence is greater than or equal to the second threshold and the Euclidean distance is greater than the third threshold, the encoding upload is performed.

6. A remote monitoring system for multi-source sensing data fusion based on edge computing, characterized in that, include: The acquisition module is configured to acquire the one-dimensional signal sequence and the two-dimensional image sequence; The memory is configured to store computer programs and construct the spatiotemporal cognitive collaborative bus. A processor, connected to the acquisition module and the memory, is configured to execute the computer program to implement the method as described in any one of claims 1 to 5.

Images