Data fusion processing method and system based on edge computing

Abstract

The application discloses a data fusion processing method and system based on edge computing, and relates to the technical field of data fusion processing. The method comprises the following steps: collecting heterogeneous data streams from a sensor array deployed by an edge node; performing spatio-temporal alignment processing on the heterogeneous data streams to generate a data tensor in a unified spatio-temporal dimension; decomposing the data tensor into a plurality of low-rank factor matrices; calculating an information complementarity measure between edge nodes based on the low-rank factor matrices to generate dynamic fusion weights; weighting and reorganizing the low-rank factor matrices by using the dynamic fusion weights to obtain a local fusion tensor of the edge node; and aggregating the local fusion tensors of each edge node to a center node to perform cross-node tensor completion by the center node to generate complete fusion data. The system comprises a data acquisition module, a spatio-temporal alignment module, a tensor decomposition module, a weight calculation module, a local fusion module and a global fusion module, which jointly realize the above method.

Description

Technical Field

[0001] This invention relates to the field of data fusion processing technology, specifically to a data fusion processing method and system based on edge computing. Background Technology

[0002] In edge computing environments, large-scale sensor arrays continuously generate multi-source heterogeneous data streams. These data differ in temporal sampling frequency and spatial reference coordinate system, hindering the collaborative utilization of multi-node data. Existing technologies for fusing heterogeneous data streams often employ weighted averaging methods based on fixed weights or splicing methods based on single-modal features. These methods cannot measure the complementarity of data content between different edge nodes, resulting in both information redundancy and information loss in the fusion results. Furthermore, edge nodes are limited by communication bandwidth and storage resources, and the data uploaded to the central node often has missing dimensions. Existing solutions typically use matrix completion or interpolation methods to handle missing data. These methods ignore the global structural relationships between multi-node data, and the completion results fail to recreate the multimodal dependencies of the real physical scene. In the local fusion stage at edge nodes, existing technologies fail to adaptively adjust the fusion weights according to the dynamically changing information complementarity relationships between nodes, resulting in highly complementary data not being sufficiently enhanced and low-complementary data not being effectively suppressed. In the global fusion stage at the central node, existing technologies struggle to recover complete multidimensional structures from fragmented tensor data across nodes. The completion process is independent of the potential correlation features between nodes, failing to maintain consistency in tensor element reconstruction from a global perspective. This invention addresses how to dynamically quantify the degree of information complementarity between edge nodes and generate adaptively adjustable fusion weights, as well as how to utilize the global latent space structure at the central node to perform consistent completion of cross-node tensors. Summary of the Invention

[0003] This invention provides a data fusion processing method and system based on edge computing, aiming to achieve accurate quantification of the dynamic information complementarity relationship between edge nodes and thereby drive the adaptive generation of fusion weights; at the same time, it realizes the consistent recovery of global multidimensional fusion data from local incomplete tensors at the central node.

[0004] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a data fusion processing method based on edge computing. The method includes: acquiring heterogeneous data streams from sensor arrays deployed at edge nodes; performing spatiotemporal alignment processing on the heterogeneous data streams to generate a data tensor with a unified spatiotemporal dimension; decomposing the data tensor into multiple low-rank factor matrices, each low-rank factor matrix corresponding to a data mode; calculating the information complementarity metric between edge nodes based on the low-rank factor matrices to generate dynamic fusion weights; using the dynamic fusion weights to weightedly reorganize the low-rank factor matrices to obtain local fusion tensors of the edge nodes; aggregating the local fusion tensors of each edge node to a central node, and performing cross-node tensor completion through the central node to generate complete fused data. This method performs structured tensor representation and low-rank decomposition of multi-source heterogeneous data at edge nodes, dynamically adjusts the fusion weights using the information complementarity relationship between nodes, reduces redundant information transmission while preserving local data features, and completes missing dimension completion by the central node based on the global latent space representation, effectively improving the accuracy and completeness of data fusion.

[0005] As a technical solution of the present invention, the spatiotemporal alignment processing of the heterogeneous data stream specifically includes: extracting the timestamp information of each data component in the heterogeneous data stream; performing linear interpolation resampling on the data components based on the timestamp information to generate a time-synchronized multi-source data sequence; obtaining the spatial coordinate parameters of each sensor in the sensor array; and performing a coordinate system-uniform transformation on the multi-source data sequence according to the spatial coordinate parameters to generate a spatially normalized data tensor. This achieves precise alignment of the multi-source data in both time and space dimensions, providing a consistent benchmark framework for subsequent tensor decomposition.

[0006] Preferably, the step of decomposing the data tensor into multiple low-rank factor matrices specifically includes: performing a multidimensional expansion operation on the data tensor to generate matrix slices expanded along each mode; calling a pre-configured alternating least squares optimizer to perform non-negative matrix decomposition on each matrix slice to generate the basis matrix and coefficient matrix corresponding to each mode; combining the basis matrix and coefficient matrix of each mode into the low-rank factor matrix, and recording the rank estimate of each low-rank factor matrix. This decomposition process can preserve the essential structural features of the data and enhance the interpretability of the factors through non-negativity constraints.

[0007] Furthermore, the step of calculating the information complementarity metric between edge nodes based on the low-rank factor matrix specifically includes: extracting the row vectors of the low-rank factor matrix corresponding to each edge node as node feature representations; calculating the cosine similarity between the feature representations of each pair of edge nodes to generate an initial similarity matrix; and performing sparsification on the initial similarity matrix, retaining node pairs with similarity exceeding a preset threshold to generate a sparse complementarity metric matrix. This complementarity metric effectively identifies the degree of correlation between data content between nodes and suppresses the interference of weakly correlated nodes on the fusion process.

[0008] Furthermore, the generation of dynamic fusion weights specifically includes: performing softmax normalization on each row of the sparse complementarity metric matrix to generate a relative complementarity weight vector between each edge node and other nodes; performing element-wise multiplication of the relative complementarity weight vector with the low-rank factor matrix of the corresponding node to obtain a node-level dynamic fusion weight vector; and extending the dynamic fusion weight vector along the modality dimension to generate a dynamic fusion weight matrix with the same dimension as the low-rank factor matrix. Thus, the fusion weights of each edge node can adaptively reflect the degree of complementarity between its current observation information and other nodes, achieving dynamic adjustment based on data characteristics.

[0009] In the local fusion stage, the weighted recombination of the low-rank factor matrix using the dynamic fusion weights specifically includes: performing a Hadamard product operation on the low-rank factor matrix of each edge node and the corresponding dynamic fusion weight matrix to generate a weighted factor matrix; performing tensor superposition and summation on the weighted factor matrices of all edge nodes within the same modality to generate the local fusion factor matrix of that modality; and performing the inverse Kronecker product operation on the local fusion factor matrices of each modality to reassemble them, obtaining the local fusion tensor. This operation efficiently completes the compressed fusion of multi-source data at the edge side while preserving the structural relationships between modalities.

[0010] In the global fusion phase, the cross-node tensor completion performed through the central node specifically includes: concatenating the local fusion tensors of each edge node at the central node according to the edge node index into a cross-node tensor block; using a pre-trained variational autoencoder to perform latent variable encoding on the cross-node tensor block to generate a global latent space representation; decoding the tensor elements of the missing dimensions based on the global latent space representation to complete the missing positions in the cross-node tensor block, generating the complete fused data. Through the generative completion capability of the variational autoencoder, gaps caused by transmission limitations or local data loss are recovered, ensuring the global consistency and integrity of the fused data.

[0011] As a preferred implementation of the aforementioned global fusion, the step of encoding latent variables in the cross-node tensor block using a pre-trained variational autoencoder specifically includes: batch normalizing the cross-node tensor block to generate a normalized tensor block; inputting the normalized tensor block into the encoder network of the variational autoencoder, and outputting the mean vector and logarithmic variance vector of the latent variables through multi-layer convolution and downsampling operations; sampling random noise from the logarithmic variance vector, and adding the random noise to the mean vector to generate the global latent space representation. Correspondingly, the step of decoding and outputting tensor elements with missing dimensions based on the global latent space representation specifically includes: inputting the global latent space representation into the decoder network of the variational autoencoder, and generating a reconstructed tensor block through multi-layer deconvolution and upsampling operations; extracting sub-tensors from the reconstructed tensor block corresponding to the missing positions in the original cross-node tensor block; filling the missing positions with the sub-tensors to generate a completed cross-node tensor block as the complete fused data. This encoding and decoding mechanism models the global data distribution in the latent space, making the filling elements conform to the overall data probability distribution, thus improving the reliability of the completion results.

[0012] This invention also provides a data fusion processing system based on edge computing. The system includes: a data acquisition module for acquiring heterogeneous data streams from sensor arrays deployed at edge nodes; a spatiotemporal alignment module for performing spatiotemporal alignment processing on the heterogeneous data streams to generate a data tensor with a unified spatiotemporal dimension; a tensor decomposition module for decomposing the data tensor into multiple low-rank factor matrices, each low-rank factor matrix corresponding to a data mode; a weight calculation module for calculating the information complementarity metric between edge nodes based on the low-rank factor matrices to generate dynamic fusion weights; a local fusion module for weighted recombination of the low-rank factor matrices using the dynamic fusion weights to obtain local fusion tensors of the edge nodes; and a global fusion module for aggregating the local fusion tensors of each edge node to a central node, and performing cross-node tensor completion through the central node to generate complete fused data. The modules of this system work collaboratively, organically combining edge computing, tensor decomposition, and generative models. It completes the initial refinement and selective fusion of data at the edge and achieves missing information completion at the center, offering technical advantages such as low transmission overhead, high fusion accuracy, and strong robustness.

[0013] The technical effects and advantages provided by the present invention in the above technical solution are as follows: Feature representations of each edge node are extracted based on a low-rank factor matrix. A sparse complementarity metric matrix reflecting the complementary information relationship between nodes is constructed by calculating the cosine similarity of the feature representations between nodes and performing sparsification. Softmax normalization is applied row-by-row to this matrix to generate a relative complementary weight vector associated with each node. This vector is then element-wise operated on with the corresponding node's low-rank factor matrix and expanded along the modal dimension to obtain a dynamic fusion weight matrix. This process allows the weights to be adjusted in real-time according to the actual changes in the complementary information relationship between nodes, enhancing the utilization of highly complementary node data while reducing the interference of redundant nodes on the fusion results, thus improving the targeting and information density of edge data fusion. At the central node, the local fusion tensors of each edge node are concatenated into a cross-node tensor block according to the node index. This tensor block is then latently encoded using a pre-trained variational autoencoder. During the encoding process, the cross-node tensor block is batch normalized and mapped to a global latent space representation, which captures the global structural features of multi-node data in the latent space. The decoder reconstructs the data based on this global latent space representation, extracting sub-tensors corresponding to the original missing positions for filling. This completion method leverages the unified representation capability of latent space for multimodal data dependencies, ensuring that the missing dimensions of each node are reconstructed under the same global latent space constraints. The completion results maintain cross-node consistency in both mathematical structure and physical semantics, avoiding data fragmentation between nodes caused by independent interpolation. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0015] Figure 1 This is a flowchart of a data fusion processing method based on edge computing; Figure 2 This is a flowchart of spatiotemporal alignment of heterogeneous data and tensor nonnegative matrix decomposition; Figure 3 This is a schematic diagram of the dynamic fusion weighted recombination and local fusion tensor generation process; Figure 4 This is a flowchart of cross-node tensor completion based on variational autoencoder. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] See Figure 1 This invention provides a data fusion processing method based on edge computing, comprising: acquiring heterogeneous data streams from sensor arrays deployed at edge nodes; performing spatiotemporal alignment processing on the heterogeneous data streams to generate a data tensor with a unified spatiotemporal dimension; decomposing the data tensor into multiple low-rank factor matrices, each low-rank factor matrix corresponding to a data mode; calculating the information complementarity metric between edge nodes based on the low-rank factor matrices to generate dynamic fusion weights; using the dynamic fusion weights to perform weighted recombination of the low-rank factor matrices to obtain local fusion tensors of the edge nodes; aggregating the local fusion tensors of each edge node to a central node, and performing cross-node tensor completion through the central node to generate complete fused data.

[0018] Example 1: In specific implementation, please refer to Figure 2 The heterogeneous data stream collected from sensor arrays deployed at edge nodes contains multimodal data components generated by multiple sensors at different sampling times and spatial locations. When performing spatiotemporal alignment processing on the heterogeneous data stream, the timestamp information of each data component is extracted. The timestamp information records the acquisition time of each data component. Based on the timestamp information, linear interpolation resampling is performed on the data components to generate a time-synchronized multi-source data sequence. The linear interpolation resampling process is as follows: a unified reference time series is set, with time points distributed at fixed time intervals; for each data component, based on its original timestamp and data value, the interpolation result for that data component at each time point in the reference time series is calculated using a linear interpolation method. The interpolation result is used as the resampled data value, thus obtaining the time-synchronized data sequence corresponding to that data component; the time-synchronized data sequences obtained after linear interpolation resampling of all data components are combined to form the time-synchronized multi-source data sequence.

[0019] The spatial coordinate parameters of each sensor in the sensor array are obtained. These parameters include the sensor's three-dimensional coordinate position in a preset spatial coordinate system. A coordinate system-one transformation is performed on the multi-source data sequence based on these parameters to generate a spatially normalized data tensor. The coordinate system-one transformation process is as follows: a reference spatial coordinate system is determined, which is a globally unified Cartesian coordinate system; the transformation matrix for each sensor's spatial coordinate parameters is calculated from the original sensor local coordinate system to the reference spatial coordinate system, and this matrix includes rotation and translation components; the spatial markers related to the sensor's spatial position in the multi-source data sequence acquired by each sensor are transformed using the transformation matrix, and the transformed spatial positions are associated with the time-synchronized multi-source data sequence. Data elements are organized according to the time and spatial dimensions to form a three-dimensional data tensor. The three dimensions of this three-dimensional data tensor are the time dimension, the sensor spatial position dimension, and the data feature dimension. This three-dimensional data tensor is the spatially normalized data tensor.

[0020] When decomposing a data tensor into multiple low-rank factor matrices, a multidimensional expansion operation is performed on the data tensor to generate matrix slices expanded along each mode. The multidimensional expansion operation refers to matrix-based expansion of the data tensor according to different modes, fixing the mode indices except for one mode, and using the vector corresponding to that mode index as a column or row of a matrix, thereby organizing the data under that mode into a two-dimensional matrix form. For a third-order data tensor, expansion along the first mode yields a first-mode matrix slice, expansion along the second mode yields a second-mode matrix slice, and expansion along the third mode yields a third-mode matrix slice. Each mode corresponds to one observation dimension of the data; in this embodiment, modes include temporal mode, spatial mode, and feature mode.

[0021] For each modality's matrix slice, a pre-configured alternating least squares optimizer is invoked to perform non-negative matrix decomposition on the slice, generating the basis matrix and coefficient matrix corresponding to that mode. The pre-configured alternating least squares optimizer internally has an objective function for non-negative matrix decomposition, which minimizes the Euclidean distance between the matrix slice V and the product of the basis matrix W and the coefficient matrix H, with an additional non-negativity constraint. The objective function is: Where V represents the matrix slice of the current mode, with dimensions of m rows and n columns; W represents the basis matrix, with dimensions of m rows and k columns; H represents the coefficient matrix, with dimensions of k rows and n columns; k represents the preset decomposition rank parameter, which is set based on the number of singular values ​​when the cumulative energy ratio of singular values ​​first exceeds the preset energy threshold after singular value decomposition of the matrix slice V. The preset energy threshold is set to 0.95. This represents the Frobenius norm.

[0022] The alternating least squares optimizer iteratively updates by alternately fixing one matrix W and H and solving for the non-negative least squares solution of the other matrix. Initially, the elements of W and H are generated from random numbers uniformly distributed in the interval [0,1]. In the t-th iteration, the elements of W and H are fixed... ,calculate : Obtained through a nonnegative least squares solver Then fix ,calculate : Similarly, it is obtained through a nonnegative least squares solver. The iteration terminates when the maximum number of iterations (200) is reached, or when the relative change in the objective function value between two consecutive iterations is less than 1e-4. The outputs W and H after the iteration are the basis matrix and coefficient matrix, respectively.

[0023] The basis matrix and coefficient matrix generated for each mode are combined into a low-rank factor matrix. For a specific mode, the low-rank factor matrix is ​​obtained by concatenating the transposes of the basis matrix and coefficient matrix along a specific dimension, or by directly using the product of the basis matrix and coefficient matrix as the low-rank factor matrix representation of that mode. The dimension of the low-rank factor matrix is ​​consistent with the dimension of the original matrix slice V. The rank estimate of each low-rank factor matrix is ​​recorded, and the rank estimate uses the value of the decomposition rank parameter k determined during the decomposition process.

[0024] Example 2: In practical implementation, when calculating the information complementarity measure between edge nodes based on the low-rank factor matrix, the row vector of the low-rank factor matrix corresponding to each edge node is extracted as the node feature representation. The total number of edge nodes is denoted as... , No. The low-rank factor matrix corresponding to each edge node is denoted as . , The dimension is ,in For the first The number of rows in the low-rank factor matrix of each edge node. This represents the number of columns in the low-rank factor matrix. Extraction All row vectors, treating each row vector as an independent feature vector, together constitute the first... The set of node feature representations of edge nodes ,in for The row vectors. .

[0025] Calculate the cosine similarity between the feature representations of each pair of edge nodes to generate an initial similarity matrix. For the The edge node and the first For each edge node, the average cosine similarity between the feature representation sets of two edge nodes is calculated as the node similarity. The edge node and the first Similarity between edge nodes Calculated using the following formula: in, and For edge node indexing, ; For the first The number of rows in the low-rank factor matrix of each edge node; For the first The number of rows in the low-rank factor matrix of each edge node; For the first The edge node of the first Row vectors; For the first The edge node of the first Row vectors; This represents the vector dot product operation; Representing vectors Norm. Calculated for all edge node pairs. Then, an initial similarity matrix is ​​generated. , The dimension is , of which Line number Column elements are .

[0026] The initial similarity matrix is ​​sparsified, retaining node pairs with similarity exceeding a preset threshold to generate a sparse complementarity metric matrix. The preset similarity threshold is denoted as... , The value is set based on the number of edge nodes and historical experience. In this embodiment... .based on For the initial similarity matrix Filter elements: For elements in ,like If the value is true, then retain the element's value; otherwise, discard it. Set to zero. The matrix obtained after filtering is a sparse complementary metric matrix. , The non-zero elements retained in the table represent highly complementary information between corresponding node pairs, while zero elements represent low complementarity of information.

[0027] When generating dynamic fusion weights, softmax normalization is performed on each row of the sparse complementarity metric matrix to generate a relative complementarity weight vector between each edge node and other nodes. For the sparse complementarity metric matrix... The The line contains The element, denoted as the nth element. Line number Column elements are For the first Perform softmax normalization on the row to generate relatively complementary weight vectors. ,in The calculation method is as follows: Before performing softmax normalization, if If it is zero, then The value is equal to 1, but to avoid zero-value nodes from participating in the weighting, all zero elements can be replaced with a sufficiently small negative number before calculation, so that the corresponding weights after softmax approach zero. In this embodiment, The zero element is replaced with before being passed to softmax. Then calculate .

[0028] The relative complementary weight vector is element-wise multiplied with the corresponding node's low-rank factor matrix to obtain the node-level dynamic fusion weight vector. During element-wise multiplication, the relative complementary weight vector... The length is And low-rank factor matrix China and other countries The dimensions of the corresponding low-rank factor matrices may differ. The generation process of the dynamically fused weight vector is as follows: for the ... Each edge node is weighted. Multiply by the first The low-rank factor matrix of the edge nodes Generate a weighted factor matrix from all elements in the set; for all... Perform this operation and superimpose the weighted factor matrices at their corresponding element positions to obtain the result that is the same as the first... The fusion vector with the same dimension as the low-rank factor matrix of the edge nodes is the fusion vector of the th edge node. The node-level dynamic fusion weight vector of each edge node, denoted as... , The length is equal to The total number of elements.

[0029] In some embodiments, to maintain dimensional consistency, the low-rank factor matrices of all edge nodes are first reshaped to have the same dimensions, i.e., the number of rows and columns are unified by padding with zeros or truncation. In this embodiment, a uniform upper limit on the number of rows is set. Number of columns Fill or truncate each low-rank factor matrix to The standardized low-rank factor matrix is ​​obtained. Then, the node-level dynamic fusion weight vector for: in: Indicates the matrix Expand it into a one-dimensional vector by row.

[0030] The dynamic fusion weight vector is extended along the modal dimension to generate a dynamic fusion weight matrix with the same dimensions as the low-rank factor matrix. For the ... Each edge node will dynamically fuse the weight vector. Rearranged with The dynamic fusion weight matrix is ​​obtained from a matrix with the same dimensions. , The dimension is The arrangement is as follows: The former each element as The first line, the next one The first element becomes the second row, and so on, until the row is filled. All rows of the given data. This results in the dynamic fusion weight matrix. With the The low-rank factor matrix of the edge nodes The dimensions are completely identical.

[0031] Example 3: In specific implementation, please refer to Figure 3 When using dynamic fusion weights to weight and reorganize the low-rank factor matrix, each edge node already possesses a dynamic fusion weight matrix and a low-rank factor matrix with the same dimension as that dynamic fusion weight matrix. The total number of edge nodes is denoted as... The total number of data modalities is denoted as For the first The edge node at the ... The low-rank factor matrix under each mode is denoted as . The corresponding dynamic fusion weight matrix is ​​denoted as ,in , . and All dimensions are , The number of rows in the matrix. is the number of columns in the matrix.

[0032] The low-rank factor matrix of each edge node is multiplied by the corresponding dynamic fusion weight matrix using the Hadamard product operation to generate a weighted factor matrix. The Hadamard product is an element-wise matrix multiplication. The edge node at the ... Weighting factor matrix under each mode Calculated using the following formula: in, This represents the Hadamard product operator. The Middle Line number The elements of the column are equal to Elements in the same position and The product of elements at the same position in the middle. For row index, , For column indexes, Weighting factor matrix Dimensions and and Exactly the same.

[0033] Within the same modality, tensor superposition is performed on the weighted factor matrices of all edge nodes to generate the local fusion factor matrix for that modality. For the ... Each modality represents a weighted factor matrix of all edge nodes. Summation is performed based on the corresponding element positions. Since the dimensions of the weighting factor matrices for different edge nodes in the same modality may differ, all weighting factor matrices are first unified to the same dimension before performing the superposition summation. The method used is: determine the... Maximum number of rows in all weighting factor matrices for each modality and maximum number of columns ,in , For insufficient dimensions The weighted factor matrix is ​​padded with zero elements on its right and bottom sides to expand its dimensions to [missing information]. The standardized weighted factor matrix is ​​obtained. Then for the first Perform element-wise summation on all standardized weighted factor matrices within the modality to generate the nth modality. Local fusion factor matrix of each modality : in, The dimension is , The Middle Line number The column element is all corresponding The sum of elements at the same position in the middle.

[0034] After getting all After obtaining the local fusion factor matrices for each modality, the local fusion factor matrices of each modality are recombined using the inverse Kronecker product operation to obtain the local fusion tensor. The inverse Kronecker product operation recombination means that the local fusion factor matrices of each modality, which were originally matrix slices obtained by expanding the same data tensor along different modes during the tensor decomposition stage, are now reconstructed into a complete three-dimensional tensor through the inverse process. During recombination, the local fusion factor matrices of the first modality are... The local fusion factor matrix of the first mode is regarded as a matrix expanded from the data tensor along the first mode, and the first mode is treated as a matrix expanded from the data tensor along the first mode. The local fusion factor matrix of each modality is considered as a matrix expanded along the second modality, and so on. For each modality, there exists a corresponding expansion index arrangement rule. According to this rule, elements are extracted from the local fusion factor matrix and filled into the corresponding positions in the target tensor. The target tensor is the local fusion tensor, denoted as . Its three dimensions are respectively the first modal dimensions. Second mode size and third modal dimensions . No. Local fusion factor matrix of each modality The dimensions must satisfy , ;No. Local fusion factor matrix of each modality The dimensions must satisfy , ;No. Local fusion factor matrix of each modality The dimensions must satisfy , If the number of modes exceeds three orders, it is expanded to a higher-order tensor, with the same principle. In actual execution, the local fusion factor matrix of any mode is selected as the main matrix, and an inverse expansion operation is performed according to the expansion rules of that mode. The local fusion factor matrices of the other modes are used to verify and correct the element values ​​at corresponding positions. Finally, tensor recombination is completed through multiple inverse expansions to generate a locally fused tensor. .

[0035] Example 4: In specific implementation, please refer to Figure 4When cross-node tensor completion is performed through the central node, the locally fused tensors of each edge node are transmitted to the central node. The total number of edge nodes is denoted as . , No. The local fusion tensor of each edge node is denoted as... , , It is a three-dimensional tensor with dimensions of ,in The first modal dimension, For the second modal size, This represents the third modal size. The central node concatenates all local fused tensors according to their edge node indices. The concatenation operation adds a new dimension beyond the original three dimensions, corresponding to the edge node indices. The concatenated cross-node tensor block is denoted as... , The size is The edge node index order is consistent with the edge node number, that is... In the newly added dimension Each slice is .

[0036] The cross-node tensor block contains empty positions due to sensor failures, communication interruptions, etc., and the element values ​​at these empty positions are invalid. Before inputting the cross-node tensor block into the variational autoencoder, the empty positions are temporarily filled with zero values, and a shape similar to... Identical binary void mask tensor , An element with a value of 1 indicates that there is valid data at the corresponding position, and an element with a value of 0 indicates that the corresponding position is empty.

[0037] When generating the global latent space representation by encoding latent variables of cross-node tensor blocks using a pre-trained variational autoencoder, batch normalization is first performed on the cross-node tensor blocks to generate normalized tensor blocks. The batch normalization process is performed along the node dimensions of the cross-node tensor blocks. For each element sequence defined by a fixed spatial location and feature index, calculate the sequence in... The mean at each node and variance Then, a normalized translation and scaling operation is performed on the sequence to generate a normalized tensor block. The formula for batch normalization is: ,in and For learnable scaling and offset parameters, To prevent division by zero of small constants, the value is . .

[0038] The normalized tensor block is input into the encoder network of the variational autoencoder. The encoder network of the variational autoencoder consists of three concatenated convolutional modules. The first convolutional module contains a convolutional kernel with a size of [missing information]. A 3D convolutional layer with a stride of 1. The first output has 32 channels, followed by a batch normalization layer and a ReLU activation function; the second convolutional module contains a convolutional kernel with a size of [missing information]. A 3D convolutional layer with a stride of 1. The output has 64 channels, followed by a batch normalization layer and a ReLU activation function; the third convolutional module contains a convolutional kernel with a size of [missing information]. A 3D convolutional layer with a stride of 1. The output has 128 channels, followed by batch normalization layers and ReLU activation functions. After three convolutional layers and spatial downsampling, the spatial dimension of the output feature map is compressed to a fraction of its original size. The feature map is flattened into a one-dimensional vector, and then mapped through a fully connected layer to obtain the mean vector of the latent variables. Sum of logarithmic variance vector The dimensions of the mean vector and the log-variance vector are both 1. , In this embodiment, the latent space dimension is preset. The value is set to 128, which is based on the conclusion that the reconstruction error tends to stabilize when the latent space dimension is 128 in the tensor completion task ablation experiment.

[0039] Random noise is sampled from the log-variance vector, and then added to the mean vector to generate a global latent space representation. The standard deviation vector is first calculated from the log-variance vector. The calculation method is as follows Random noise vector From the dimension multivariate standard normal distribution Obtained by mid-sampling. Global latent space representation. Obtained through the following formula: in, Let be the global latent space representation vector, with dimension . ; Let be the mean vector output by the encoder network, with dimension . ; The standard deviation vector is obtained by using the log-variance vector. according to The calculated dimension is ; Let be a random noise vector sampled from a multivariate standard normal distribution, with dimension . ; This represents the element-wise multiplication operation of vectors.

[0040] The variational autoencoder's pre-training process uses a complete historical cross-node tensor block dataset. The training data is composed of concatenated local fusion tensors from all fully acquired edge nodes, excluding missing positions. During training, for each complete tensor block, a binary mask is randomly generated to simulate missing positions. These missing positions are then set to zero, and the mask is recorded, forming an input tensor block with missing positions. The variational autoencoder's reconstruction loss is the mean squared error between the input tensor block and the decoder's output reconstructed tensor block at the valid positions indicated by the mask. The KL divergence loss is the KL divergence between the latent variable distribution and the standard normal distribution. The total loss function is a weighted sum of the reconstruction loss and the KL divergence loss, with the weight coefficient for the KL divergence loss set to 0.1. The Adam optimizer is used, with a learning rate of 0.0002, a batch size of 16, and training stopped after 200 epochs. After pre-training, the parameters of the encoder and decoder networks are fixed.

[0041] When decoding tensor elements with missing dimensions based on global latent space representation, the global latent space representation... The decoder network is an input variational autoencoder. The decoder network structure is symmetrical to the encoder network, starting with a fully connected layer... The mapping is performed as a vector with the same dimension as the flattened feature map of the last layer of the encoder, and then the vector is reshaped into a 3D feature map. The reshaped feature map is then passed through three deconvolution modules to gradually restore spatial resolution. The first deconvolution module contains a convolution kernel with a size of [missing information]. A 3D deconvolution layer with a stride of 1. The output has 64 channels, followed by a batch normalization layer and a ReLU activation function; the second deconvolution module contains a convolution kernel with a size of A 3D deconvolution layer with a stride of 1. The output has 32 channels, followed by a batch normalization layer and a ReLU activation function; the third deconvolution module contains a convolution kernel with a size of A 3D deconvolution layer with a stride of 1. The output channel is 1, followed by a sigmoid activation function to restrict the output to a reasonable data range. The final reconstructed tensor block output by the decoder is denoted as... , The size is the same as the input cross-node tensor block, which is... .

[0042] Extract the sub-tensors from the reconstructed tensor block that correspond to the missing positions in the original cross-node tensor block. Utilize the previously generated binary missing mask tensor. ,Sure The set of coordinates where the element with a value of 0 is used to apply this set of coordinates. ,extract The element values ​​at the corresponding coordinates constitute the subtensor. Finally, the subtensor Fill to the original cross-node tensor block The vacant positions will be reserved. The original valid data with an element value of 1 remains unchanged, and a completed cross-node tensor block is generated. The completed cross-node tensor block is the complete fused data.

[0043] Example 5: In practical implementation, a data fusion processing system based on edge computing includes a data acquisition module, a spatiotemporal alignment module, a tensor decomposition module, a weight calculation module, a local fusion module, and a global fusion module. The data acquisition module is deployed at each edge node to collect heterogeneous data streams from sensor arrays deployed at the edge nodes. The sensor arrays consist of various types of sensors, including temperature sensors, vibration sensors, and image sensors. Each sensor outputs data components with different data formats and sampling frequencies, collectively forming a heterogeneous data stream. At each edge node, the data acquisition module aggregates the raw signals from its respective sensor, encapsulates each data component into a data packet with sensor identifiers and timestamps, and transmits it to the spatiotemporal alignment module via the communication bus within the edge node.

[0044] The spatiotemporal alignment module receives heterogeneous data streams from various edge nodes and performs spatiotemporal alignment processing on these streams to generate a data tensor with a unified spatiotemporal dimension. Internally, the spatiotemporal alignment module includes a time synchronization unit and a spatial normalization unit. The time synchronization unit extracts the timestamp information of each data component in the heterogeneous data stream and performs linear interpolation resampling on the data components based on the timestamp information to generate a time-synchronized multi-source data sequence. The linear interpolation resampling is implemented as follows: the time synchronization unit maintains a global reference clock and generates a unified time series with a fixed time interval; for each data component, based on the sampled value of the data component at the original timestamp, a linear interpolation function is used to calculate the interpolation result at each moment in the unified time series, forming the resampled data sequence. The spatial normalization unit obtains the spatial coordinate parameters of each sensor in the sensor array, including the sensor's three-dimensional position coordinates and attitude angle in a preset installation coordinate system. The spatial normalization unit performs a coordinate system uniform transformation on the multi-source data sequence based on the spatial coordinate parameters to generate a spatially normalized data tensor. When the coordinate system is transformed, the spatial normalization unit maps the data from the local coordinate system of each sensor to the global reference coordinate system through rotation matrix and translation vector, and performs grid-discretion processing on the spatial position. The discretized spatial index, temporal index and feature index are combined into the three dimensions of the three-dimensional data tensor, thereby outputting a data tensor with unified spatiotemporal dimensions.

[0045] The tensor decomposition module is deployed at each edge node. It receives the data tensor output by the spatiotemporal alignment module and decomposes the data tensor into multiple low-rank factor matrices, each corresponding to a data mode. The tensor decomposition module internally includes a multidimensional expansion unit, an alternating least squares optimization unit, and a factor combination unit. The multidimensional expansion unit performs multidimensional expansion operations on the data tensor, generating matrix slices expanded along each mode. For a third-order data tensor, the multidimensional expansion unit fixes two mode indices other than the current mode, arranges the data corresponding to all indices of the current mode into a column or row of a matrix, generating temporal mode matrix slices expanded along the temporal mode, spatial mode matrix slices expanded along the spatial mode, and eigenmode matrix slices expanded along the eigenmode. The alternating least squares optimization unit is pre-configured with an alternating least squares optimizer, which calls a non-negative matrix factorization algorithm to decompose each matrix slice. The alternating least squares optimizer receives a matrix slice, initializes the basis matrix and coefficient matrix, and solves for the other matrix using non-negative least squares by alternately fixing one of the basis matrix and coefficient matrix. This process is iterated until convergence, and the basis matrix and coefficient matrix corresponding to each mode are output. The factor combination unit combines the basis matrix and coefficient matrix of each mode into a low-rank factor matrix. The combination method is to multiply the basis matrix and coefficient matrix, and the product is used as the low-rank factor matrix of that mode. The rank of the low-rank factor matrix is ​​recorded. The value of the rank is equal to the preset decomposition rank parameter, which is determined by the proportion of singular value cumulative energy of the matrix slice.

[0046] The weight calculation module is deployed at each edge node or center node to calculate the information complementarity measure between edge nodes based on the low-rank factor matrix, generating dynamic fusion weights. The weight calculation module internally includes a feature extraction unit, a similarity calculation unit, a sparsity processing unit, and a weight generation unit. The feature extraction unit extracts the row vectors of the low-rank factor matrix corresponding to each edge node as node feature representations, treating each row vector as a feature vector, and summing them to form the feature vector set for that edge node. The similarity calculation unit calculates the cosine similarity between the feature representations of each pair of edge nodes, generating an initial similarity matrix. For the feature vector sets of two edge nodes, the similarity calculation unit calculates the cosine similarity of all row vector pairs in the two sets and takes the average, using this as the similarity value for the two edge nodes, filling the corresponding positions in the initial similarity matrix. The sparsity processing unit performs sparsification on the initial similarity matrix, retaining node pairs with similarity exceeding a preset threshold, generating a sparse complementarity measure matrix. The preset threshold is set based on the statistical quantiles of the similarity distribution, for example, using the median of similarity as the threshold. The weight generation unit includes a softmax normalization subunit and a weight matrix expansion subunit. The softmax normalization subunit performs softmax normalization on each row of the sparse complementarity metric matrix, generating a relative complementary weight vector between each edge node and other nodes. The weight matrix expansion subunit performs element-wise multiplication of the relative complementary weight vector with the corresponding node's low-rank factor matrix to obtain a node-level dynamic fusion weight vector. Then, the dynamic fusion weight vector is expanded along the modal dimension and reshaped using the same dimensional arrangement rules as the low-rank factor matrix to generate a dynamic fusion weight matrix with the same dimensions as the low-rank factor matrix.

[0047] The local fusion module is deployed at each edge node to reorganize the low-rank factor matrix using dynamic fusion weights, obtaining the local fusion tensor of the edge node. Internally, the local fusion module includes a Hadamard product unit, a modal superposition unit, and a Kronecker inverse operation reorganization unit. The Hadamard product unit performs a Hadamard product operation on the low-rank factor matrix of each edge node and the corresponding dynamic fusion weight matrix, multiplying corresponding elements in two matrices of the same shape to generate a weighted factor matrix. The modal superposition unit performs tensor superposition and summation on the weighted factor matrices of all edge nodes within the same modality, generating the local fusion factor matrix for that modality. Before superposition and summation, the modal superposition unit expands the weighted factor matrices of different edge nodes in that modality to a uniform maximum number of rows and columns using zero-padding to ensure dimensionality consistency before element-wise addition. The Kronecker inverse operation reorganization unit performs the Kronecker product inverse operation on the local fusion factor matrices of each modality to obtain the local fusion tensor. During the Kronecker product inverse operation recombination, the Kronecker inverse operation recombination unit, following the inverse rules of tensor expansion, sequentially extracts the corresponding slices from the local fusion factor matrix of each modality and fills them into the corresponding positions of the target three-dimensional tensor, ultimately forming a local fusion tensor.

[0048] The global fusion module is deployed at the central node to aggregate the local fusion tensors from each edge node to the central node. The central node then performs cross-node tensor completion to generate complete fused data. The global fusion module internally includes a tensor stitching unit, a variational autoencoder encoding unit, a variational autoencoder decoding unit, and a gap-filling unit. The tensor stitching unit receives the local fusion tensors uploaded from all edge nodes and stitches them together into cross-node tensor blocks at the central node according to the edge node indices. Specifically, it adds a dimension based on the edge node indices to the original 3D tensors, forming a 4D cross-node tensor block. The variational autoencoder encoding unit internally includes a batch normalization subunit and an encoder network. The batch normalization subunit performs batch normalization on the cross-node tensor blocks, calculating the mean and variance along the node dimensions and then normalizing and scaling to generate normalized tensor blocks. The encoder network consists of multiple 3D convolutional layers and downsampling layers, performing convolution and downsampling operations on the normalized tensor blocks, outputting the mean vector and log-variance vector of the latent variables. The variational autoencoder (VAE) encoding unit samples random noise from the log-variance vector and adds the random noise to the mean vector to generate a global latent space representation. The VAE decoding unit contains a decoder network composed of multiple layers of 3D deconvolutional layers and upsampling layers. The global latent space representation is input into the decoder network, and reconstructed tensor blocks are generated through multiple deconvolutional and upsampling operations. The gap-filling unit extracts sub-tensors from the reconstructed tensor blocks corresponding to the gap positions in the original cross-node tensor blocks and fills the gap positions with these sub-tensors, generating a completed cross-node tensor block as the complete fused data. The gap-filling unit determines the coordinates of the gap positions based on the gap mask tensor, which is generated by the data acquisition module when missing sensor data is detected and uploaded to the central node along with the local fused tensor.

[0049] In some embodiments, the encoder network in the variational autoencoder encoding unit and the decoder network in the variational autoencoder decoding unit adopt a symmetrical encoding / decoding structure, wherein the convolution kernel size of the encoder network is [missing information]. The downsampling step size is The number of output channels for each level is 32, 64, and 128, respectively; the deconvolution kernel size of the decoder network is... The upsampling step size is The number of channels in the hierarchical output is 64, 32, and 1, respectively. The latent space dimension is set to 128, a value determined based on the verification of the relationship between the reconstruction error and the latent space dimension of the variational autoencoder on the tensor completion task.

[0050] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A data fusion processing method based on edge computing, characterized in that, The method includes: Acquire heterogeneous data streams from sensor arrays deployed at edge nodes; Perform spatiotemporal alignment processing on the heterogeneous data stream to generate a data tensor with a unified spatiotemporal dimension; The data tensor is decomposed into multiple low-rank factor matrices, each of which corresponds to a data mode. Based on the low-rank factor matrix, calculate the information complementarity measure between edge nodes and generate dynamic fusion weights; The low-rank factor matrix is ​​weighted and reorganized using the dynamic fusion weights to obtain the local fusion tensor of the edge nodes; The local fusion tensors of each edge node are aggregated to the central node, and cross-node tensor completion is performed through the central node to generate complete fusion data.

2. The data fusion processing method based on edge computing according to claim 1, characterized in that, The spatiotemporal alignment process performed on the heterogeneous data stream specifically includes: Extract the timestamp information of each data component in the heterogeneous data stream, and perform linear interpolation resampling on the data components based on the timestamp information to generate a time-synchronized multi-source data sequence; Obtain the spatial coordinate parameters of each sensor in the sensor array, and perform a coordinate system transformation on the multi-source data sequence based on the spatial coordinate parameters to generate a spatially normalized data tensor.

3. The data fusion processing method based on edge computing according to claim 1, characterized in that, The step of decomposing the data tensor into multiple low-rank factor matrices specifically includes: Perform a multidimensional expansion operation on the data tensor to generate matrix slices expanded along each mode; The pre-configured alternating least squares optimizer is invoked to perform non-negative matrix decomposition on each of the matrix slices, generating the basis matrix and coefficient matrix corresponding to each mode; The basis matrix and coefficient matrix of each mode are combined into the low-rank factor matrix, and the rank estimate of each low-rank factor matrix is ​​recorded.

4. The data fusion processing method based on edge computing according to claim 1, characterized in that, The calculation of the information complementarity metric between edge nodes based on the low-rank factor matrix specifically includes: Extract the row vectors of the low-rank factor matrix corresponding to each edge node as the node feature representation; Calculate the cosine similarity between the feature representations of each pair of edge nodes to generate an initial similarity matrix; The initial similarity matrix is ​​sparsified, and node pairs with similarity exceeding a preset threshold are retained to generate a sparse complementary metric matrix.

5. The data fusion processing method based on edge computing according to claim 4, characterized in that, The generation of dynamic fusion weights specifically includes: Softmax normalization is performed on each row of the sparse complementary metric matrix to generate a relative complementary weight vector between each edge node and other nodes; The relative complementary weight vector is multiplied element-wise with the low-rank factor matrix of the corresponding node to obtain the node-level dynamic fusion weight vector. The dynamic fusion weight vector is extended along the modal dimension to generate a dynamic fusion weight matrix with the same dimension as the low-rank factor matrix.

6. The data fusion processing method based on edge computing according to claim 1, characterized in that, The step of using the dynamic fusion weights to weight and reorganize the low-rank factor matrix specifically includes: Perform a Hadamard product operation on the low-rank factor matrix of each edge node and the corresponding dynamic fusion weight matrix to generate a weighted factor matrix. Tensor superposition is performed on the weighted factor matrices of all edge nodes within the same mode to generate the local fusion factor matrix of that mode; The local fusion factor matrices of each modality are recombined by performing the Kronecker product inverse operation to obtain the local fusion tensor.

7. The data fusion processing method based on edge computing according to claim 1, characterized in that, The process of performing cross-node tensor completion through the central node specifically includes: The local fusion tensors of each edge node are concatenated into a cross-node tensor block at the central node according to the edge node index; The cross-node tensor blocks are latent variable encoded using a pre-trained variational autoencoder to generate a global latent space representation; Based on the tensor elements of the missing dimensions in the global latent space representation decoding output, the missing positions in the cross-node tensor block are filled to generate the complete fused data.

8. The data fusion processing method based on edge computing according to claim 7, characterized in that, The process of encoding latent variables in the cross-node tensor block using a pre-trained variational autoencoder specifically includes: Batch normalization is performed on the cross-node tensor blocks to generate normalized tensor blocks; The normalized tensor block is input into the encoder network of the variational autoencoder, and the mean vector and log-variance vector of the latent variables are output through multi-layer convolution and downsampling operations. Random noise is sampled from the logarithmic variance vector, and the random noise is added to the mean vector to generate the global latent space representation.

9. The data fusion processing method based on edge computing according to claim 7, characterized in that, The tensor elements for decoding the missing dimensions based on the global latent space representation specifically include: The global latent space representation is input into the decoder network of the variational autoencoder, and a reconstructed tensor block is generated through multi-layer deconvolution and upsampling operations; Extract the sub-tensors from the reconstructed tensor block that correspond to the missing positions in the original cross-node tensor block; The sub-tensors are filled into the missing positions to generate a completed cross-node tensor block as the complete fused data.

10. A data fusion processing system based on edge computing, used to implement the data fusion processing method based on edge computing as described in any one of claims 1 to 9, characterized in that, The system includes: The data acquisition module is used to acquire heterogeneous data streams from sensor arrays deployed at edge nodes; The spatiotemporal alignment module is used to perform spatiotemporal alignment processing on the heterogeneous data stream to generate a data tensor with a unified spatiotemporal dimension. The tensor decomposition module is used to decompose the data tensor into multiple low-rank factor matrices, each low-rank factor matrix corresponding to a data mode. The weight calculation module is used to calculate the information complementarity measure between edge nodes based on the low-rank factor matrix and generate dynamic fusion weights. The local fusion module is used to perform weighted recombination of the low-rank factor matrix using the dynamic fusion weights to obtain the local fusion tensor of the edge nodes; The global fusion module is used to aggregate the local fusion tensors of each edge node to the central node, and then perform cross-node tensor completion through the central node to generate complete fusion data.

Images