A multi-source data fusion method and system based on bias recognition embedding

Abstract

The application discloses a multi-source data fusion method and system based on bias recognition embedding, relates to the technical field of artificial intelligence, and comprises the following steps: adding noise through an ICEEMDAN algorithm, decomposing by using EMD, performing significance estimation and separating signal extraction through a DSC model and an HEPSO optimization function, performing dimension reduction by using a self-encoder and performing constraint by using an optical flow field, performing multi-scale decomposition by using Caputo-Katugampola fractional derivative, calculating an adaptive diffusion kernel through a shift Chebyshev polynomial, and solving by using a convolution quotient domain, and optimizing by using fractional order gradient features and enhanced energy functionals. The application improves the precision and robustness of multi-source data fusion by combining bias recognition embedding and multi-scale fractional order decomposition, combining optical flow field constraint and sparse coding, and realizes efficient fusion, denoising processing and feature optimization of data.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a multi-source data fusion method and system based on bias recognition and embedding. Background Art

[0002] As one of the core technologies in the field of modern information processing, multi-source data fusion technology has been widely used in computer vision, natural language processing, signal processing and multimodal data analysis in recent years. This technology aims to extract comprehensive features and improve the robustness and accuracy of data representation by integrating data from different sources and different modalities. Traditional methods are usually based on statistical analysis, signal decomposition or machine learning models, such as PCA, wavelet transform, EMD and Bi-LSTM. With the rise of fractional calculus and sparse coding technology, researchers have begun to explore processing methods for nonlinear and non-stationary signals to improve the accuracy and robustness of data fusion. Through technologies such as calculating significant features through optical flow fields and combining optimization algorithms for signal separation, the feature extraction ability of multimodal data has been significantly improved.

[0003] However, the existing technology still has shortcomings. The existing methods fail to introduce the convolution quotient domain and Hilbert space for reconstruction and optimization at the numerical solution level, which makes it difficult to meet the task requirements of high precision and strong robustness. Summary of the Invention

[0004] In view of the above existing problems, the inventors proposed the present invention.

[0005] Therefore, the present invention provides a multi-source data fusion method and system based on bias recognition embedding, which solves the problem that existing methods fail to introduce convolution quotient domain and Hilbert space for reconstruction and optimization at the numerical solution level, and are difficult to meet the task requirements of high precision and strong robustness.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a multi-source data fusion method based on bias identification embedding, which comprises:

[0008] Multimodal data is collected and preprocessed using an API to obtain the original dataset and high- and low-frequency subbands. Noise is added to the original dataset using the ICEEMDAN algorithm, decomposed using EMD, and filtered to obtain the denoised IMF. Data reconstruction is then performed to extract features from the multimodal data. This data is then enhanced using Bi-LSTM and CNN, respectively. Optical flow is calculated based on the reconstructed denoised data. Saliency estimation and separation signal extraction are performed using the DSC model and HEPSO optimization function. A global feature vector is generated, dimensionality reduction is performed using an autoencoder, and optical flow is used for constraint to obtain the final representation.

[0009] Based on high- and low-frequency subbands, multi-scale decomposition is performed using Caputo-Katugampola fractional derivatives to generate enhanced saliency maps. The final representation is combined to calculate the adaptive diffusion kernel by shifting Chebyshev polynomials and solving it using the convolution quotient field to obtain the adjusted high-frequency subbands. Multi-scale wavelet decomposition is performed using Daubechies wavelets, the Gini index is calculated, and weighted subband coefficients are obtained by screening. The final salient detail layer is obtained by inverse wavelet transform. The initial gradient field is calculated and optimized using fractional gradient features and enhanced energy functionals. It is converted to integer order through Hilbert space expansion and adjusted using MSB analysis. The background detail layer is extracted in combination with the ITTI model, and denoised by wavelet decomposition using fractional heat equations and smoothing to obtain the final fusion layer.

[0010] Based on the final fusion layer, sparse coding is used to generate sparse feature vectors to obtain the final data fusion representation.

[0011] As a preferred solution of the multi-source data fusion method based on bias identification and embedding of the present invention, the method of adding noise to the original data set by the ICEEMDAN algorithm to obtain the final representation includes:

[0012] Based on the original dataset, white noise is added using the ICEEMDAN algorithm and decomposed using EMD to generate the final IMF and residual. The denoised IMF is obtained by filtering and denoising using IWT, and the reconstructed denoised data is generated. The enhanced sequence features, enhanced image features, and initial optical flow field are extracted, and the dimensions are aligned using linear transformation. The initial feature vector is obtained by feature concatenation.

[0013] Based on the image data in the reconstructed denoised data, the initial saliency estimate is obtained through the DSC model, the optimization objective function is constructed, the initial saliency map is output, the candidate separation matrix is ​​obtained through blind source separation, the HFPSO optimization function is constructed, the optimal separation matrix is ​​output, and the separation signal is obtained through ICA;

[0014] Using the initial optical flow field, the initial saliency map is updated by inverse deformation and combined with the separation signal to obtain a dynamic saliency map. The multi-head attention output and the bias-enhanced feature vector are combined to generate a global feature vector. The autoencoder is used for dimensionality reduction and the initial optical flow field is used for constraint to obtain an optical flow constrained feature vector. The feature vector is smoothed by bilinear interpolation to obtain a smoothed feature vector, which is mapped using linear projection and normalized by the L2 norm to obtain the final representation.

[0015] As a preferred solution of the multi-source data fusion method based on bias identification and embedding of the present invention, wherein: based on the high and low frequency sub-bands, multi-scale decomposition is performed using Caputo-Katugampola fractional derivatives, and the final significant detail layer is obtained by inverse wavelet transform, including:

[0016] Based on high-frequency subbands and low-frequency subbands, combined with smooth eigenvectors, Caputo-Katugampola fractional-order derivatives are used for multi-scale decomposition, and Caputo-Katugampola sequence derivatives are used to generate enhanced saliency maps. The adaptive diffusion kernel is calculated by shifting Chebyshev polynomials, and the fractional-order PDE is obtained by Mikusiński operation calculus, and solved by convolution quotient field. The solution of the fractional-order PDE is approximated by shifting Chebyshev polynomials to obtain the fused low-frequency subbands and high-frequency subbands. The biased enhanced eigenvector is used for adjustment, and Daubechies wavelet is used as the basis function. Multi-scale subbands are generated by wavelet decomposition to obtain subband coefficients, and the Gini index is calculated. The weighted subband coefficients are obtained by screening, and the inverse wavelet decomposition transform is performed. The final representation is combined for adjustment to obtain the final significant detail layer.

[0017] As a preferred solution of the multi-source data fusion method based on bias identification embedding of the present invention, wherein: the initial gradient field is calculated and optimized using fractional gradient features and enhanced energy functionals to obtain the final fusion layer, including:

[0018] Based on the final salient detail layer, the initial gradient field is obtained by differential approximation. The Caputo-Katugampola fractional derivatives are used to generate fractional gradient features. Combined with the final salient detail layer, an enhanced energy functional is constructed. The fractional gradient features are used for optimization. The fractional gradient flow is converted to integer order using Hilbert space expansion. The output is the optimized enhanced salient detail layer, which is adjusted using MSB analysis to obtain the adjusted salient detail layer. The background detail layer is extracted using the ITTI model, smoothed using the fractional heat equation, and denoised through wavelet decomposition to obtain the final fused detail layer.

[0019] As a preferred solution of the multi-source data fusion method based on bias identification embedding of the present invention, wherein: based on the final fusion layer, sparse coding is used to generate a sparse feature vector, including:

[0020] Based on the final fused detail layer, enhanced saliency map and final representation, an initial feature vector is generated by spatial weighting, which is updated using incremental learning to obtain an updated feature vector, and sparse coding is used to generate a sparse feature vector.

[0021] As a preferred solution of the multi-source data fusion method based on bias identification and embedding of the present invention, the step of obtaining the final data fusion representation includes:

[0022] Based on the sparse feature vector, the global feature vector is obtained by aggregation, and combined with the final fusion detail layer to obtain the final fusion representation.

[0023] As a preferred solution of the multi-source data fusion method based on bias identification and embedding of the present invention, wherein: the multimodal data is collected and preprocessed using the API interface to obtain the original data set and high and low frequency sub-bands, including:

[0024] Use the API interface to collect multimodal data, including images, time series, and text data, and perform preprocessing to obtain low-frequency subbands, high-frequency subbands, original data sets, and bias-enhanced feature vectors.

[0025] In a second aspect, the present invention provides a multi-source data fusion system based on bias identification and embedding, comprising:

[0026] The processing enhancement module is used to collect multimodal data using the API interface for preprocessing, obtaining the original data set and high- and low-frequency subbands. The original data set is noised using the ICEEMDAN algorithm, decomposed using EMD, and filtered to obtain the denoised IMF. The data is reconstructed and features of the multimodal data are extracted. The data is enhanced using Bi-LSTM and CNN, respectively. The optical flow field is calculated based on the reconstructed denoised data. The DSC model and HEPSO optimization function are used for saliency estimation and separation signal extraction, and a global feature vector is generated. The autoencoder is used for dimensionality reduction and the optical flow field is used for constraint to obtain the final representation.

[0027] Decomposition and fusion module, used to perform multi-scale decomposition based on high- and low-frequency subbands using Caputo-Katugampola fractional derivatives to generate enhanced saliency maps. The adaptive diffusion kernel is calculated by shifting Chebyshev polynomials in combination with the final representation, and solved using the convolution quotient field to obtain the adjusted high-frequency subbands. Multi-scale wavelet decomposition is performed using Daubechies wavelets, the Gini index is calculated, and weighted subband coefficients are obtained by screening. The final salient detail layer is obtained by inverse wavelet transform. The initial gradient field is calculated, optimized using fractional gradient features and enhanced energy functionals, and converted to integer order through Hilbert space expansion. Adjustment is performed using MSB analysis, and the background detail layer is extracted in combination with the ITTI model. Fractional heat equations and smoothing are used to denoise the final fusion layer through wavelet decomposition.

[0028] The sparse representation module is used to generate sparse feature vectors based on the final fusion layer using sparse coding to obtain the final data fusion representation.

[0029] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the computer program, when executed by the processor, implements any step of the bias-identification-embedded multi-source data fusion method according to the first aspect of the present application.

[0030] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements any step of the bias-identification-embedded multi-source data fusion method according to the first aspect of the present application.

[0031] The present application has the following beneficial effects: the present application adds noise through the ICEEMDAN algorithm, decomposes using EMD, estimates significance and extracts signals through the DSC model and HEPSO optimization function, reduces dimensions using a self-encoder and constrains using an optical flow field, performs multi-scale decomposition using Caputo-Katugampola fractional derivative, calculates an adaptive diffusion kernel through a shift Chebyshev polynomial, and solves using a convolution quotient domain, and optimizes using fractional gradient features and enhanced energy function; the accuracy and robustness of multi-source data fusion are improved, efficient data fusion, denoising processing and feature optimization are achieved. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0033] Figure 1 A flowchart of a bias-identification-embedded multi-source data fusion method according to embodiment 1;

[0034] Figure 2 A schematic diagram of a bias-identification-embedded multi-source data fusion system according to embodiment 1. DETAILED DESCRIPTION

[0035] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0036] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the scope of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0037] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it constitute a separate or selective embodiment that is mutually exclusive with other embodiments.

[0038] Example 1, with reference to Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a multi-source data fusion method based on bias identification embedding, including the following steps:

[0039] S1. Use the API interface to collect multimodal data for preprocessing, obtaining the original dataset and high- and low-frequency subbands. Add noise to the original dataset using the ICEEMDAN algorithm, decompose it using EMD, filter out the denoised IMF, and reconstruct the data. Extract the features of the multimodal data, enhance it using Bi-LSTM and CNN, calculate the optical flow field based on the reconstructed denoised data, perform saliency estimation and separation signal extraction using the DSC model and HEPSO optimization function, generate a global feature vector, use an autoencoder for dimensionality reduction, and use the optical flow field for constraint to obtain the final representation.

[0040] Specifically, use the API interface to collect multimodal data for preprocessing to obtain the original data set and high and low frequency sub-bands, including:

[0041] Use API interfaces to collect multimodal data, including images, time series, and text data, and perform preprocessing;

[0042] The preprocessing includes grayscale normalization and resolution alignment of the image data, noise removal by using a bilateral filter, decomposition using a Laplacian pyramid to obtain low-frequency subbands and high-frequency subbands, and applying a SOMP algorithm to the low-frequency subbands to generate a low-dimensional feature vector;

[0043] Perform mean normalization on the time series data, use sliding average filtering to remove noise, extract frequency domain features through Fourier transform, and generate time series feature vectors;

[0044] Use NLP to segment text data, remove stop words, and extract stems, and use the BERT model to generate semantic embedding vectors;

[0045] Extract the original feature vectors in the high-frequency subband, combine them with the low-dimensional feature vectors, time series feature vectors, and semantic embedding vectors, map them through linear transformation, and use L2 regularization for scale alignment to generate a structured feature set;

[0046] Based on the structured feature set, the Hellinger distance is combined with the Jaccard matrix to calculate the bias distance between features. The formula is:

[0047] ,

[0048] in, is the bias distance, and is the eigenvector after Jaccard matrix transformation, For the feature subsets, is the number of feature subsets;

[0049] Build a GAN, including input layer, hidden layer, bias feature output layer, and classification output layer;

[0050] Use ImageNet, IoT sensor, and Twitter datasets for preprocessing to obtain training structured feature sets for GAN training;

[0051] Input the structured feature set into GAN and output a low-dimensional bias feature vector;

[0052] Concatenate the low-dimensional bias feature vector and the bias distance to obtain the bias-enhanced feature vector;

[0053] Extract the normalized image data and time series data, combine them with the segmented text data, and splice them into the original dataset.

[0054] By using the API interface to collect multimodal data and perform differentiated preprocessing, the system ensures its breadth and flexibility in the raw data acquisition stage. SOMP is used to compress the low-frequency subband features, which improves the data compression ratio while maintaining the image structural information. Frequency features are extracted through Fourier transform, which can effectively capture the periodicity and mutation in the time signal, which is conducive to modeling the behavioral patterns and dynamic characteristics of sensor data. BERT provides context-sensitive semantic vectors, which helps to maintain semantic continuity and conceptual consistency during cross-modal fusion. Linear transformation and L2 regularization are used to unify the scale of multi-source features, which can effectively prevent the dominant modality bias caused by the difference in numerical dimensions between modalities. By splicing low-dimensional bias vectors and bias distances, the double-layer bias features between individuals and groups are fused, further improving the integrity and expressiveness of the system in bias identification and response modeling.

[0055] Furthermore, the ICEEMDAN algorithm is used to add noise to the original dataset to obtain the final representation, including:

[0056] Based on the original data set, white noise is added through the ICEEMDAN algorithm, and the local IMF is obtained by EMD decomposition. The final IMF and residual are generated through iterative averaging (based on the early stopping method).

[0057] Based on the final IMF, the final IMF threshold is set by empirical rules, and the IMF threshold greater than the final IMF threshold is selected. The IWT is used for denoising to obtain the denoised IMF.

[0058] The IMF threshold below the final value is filtered out and set as the low-frequency IMF. The denoised IMF is combined for reconstruction to obtain the reconstructed denoised data. The formula is:

[0059] ,

[0060] in, To reconstruct the denoised data, To denoise IMF, is the denoised IMF number, is the residual, is the low-frequency IMF, is the final IMF number, is the index of the iteration number;

[0061] Based on the reconstructed denoised data, the data entropy is calculated using the Shannon entropy formula, and the time series data and text data are extracted. The Bi-LSTM is used to extract sequence features to obtain enhanced sequence features.

[0062] Based on the reconstructed denoised data, the image features are extracted and the Sandglass Block CNN is used to extract the image features to obtain enhanced image features.

[0063] Based on the reconstructed denoised data, Transforme is used to obtain the initial optical flow field;

[0064] Based on enhanced sequence features, enhanced image features and initial optical flow field, linear transformation is used to align dimensions and the features are concatenated into an initial feature vector.

[0065] Extract image data from the reconstructed denoised data, construct the data matrix (based on the covariance matrix), the weight matrix (based on the Gaussian similarity kernel), and the smoothing constraint matrix (based on the graph Laplacian matrix);

[0066] Based on the image data in the reconstructed denoised data, the initial saliency estimate is obtained through the DSC model. The optimization objective function is constructed by combining the three matrices. The iterative solution is performed through gradient descent. When the maximum number of iterations is reached, the initial saliency map is output. The formula is:

[0067] ,

[0068] Among them, Y is the initial saliency map, is the saliency map vector, is the data matrix, is the weight matrix, is the regularization parameter, is the identity matrix (obtained based on the normalization of the data matrix), is the normalization factor, is the initial significance estimate, is the smooth constraint matrix, is transposed;

[0069] Based on the image data in the reconstructed denoised data, the candidate separation matrix is ​​obtained by blind source separation and defined as particles. The HFPSO optimization function is constructed and the HFPSO optimization function value is calculated. The global optimal position is obtained by position update and velocity update (based on the fixed maximum iteration method setting), the optimal separation matrix is ​​output, and the separation signal is obtained by ICA. The formula is:

[0070] ,

[0071] in, is the optimal separation matrix, is the candidate separation matrix The optimization function, is the number of signals, To separate the signal The probability density function of (based on negative entropy estimation), is the candidate separation matrix The determinant of is the signal index;

[0072] Using the initial optical flow field, the initial saliency map is updated through reverse deformation to obtain a temporary saliency map. Combined with the separation signal, a linear layer is used for mapping, and the temporary saliency map is enhanced through the softmax function to obtain a dynamic saliency map.

[0073] Based on the initial feature vector, the query vector is obtained through linear projection;

[0074] Use bias to enhance the feature vector, separate the signal and the initial optical flow field to adjust the query vector. The formula is:

[0075] ,

[0076] in, To adjust the query vector, is the query vector, Biased characteristics The linear change of For light flow projection matrix (based on linear projection), T is an adaptive bias guide light field (the bias enhanced feature vector is mapped through a fully connected layer to obtain a bias weight, and the initial flow light field is adjusted by weighted fusion using the bias weight to obtain)

[0077] Based on the adjustment query vector, the multi-head attention output is calculated by the softmax function;

[0078] Based on the multi-head attention output, the dynamic saliency map and the bias enhanced feature vector, an alignment feature vector is generated, and the formula is:

[0079] ,

[0080] Wherein, P is an alignment feature vector, P is a dynamic saliency map, is a multi-attention head output, is layer normalization;

[0081] Based on the bias enhanced feature vector, the dynamic weight vector is calculated by the softmax function;

[0082] Based on the dynamic weight vector and the alignment feature vector, the isomorphic aggregation feature vector is obtained by weighted calculation combined with the linear change result of the bias feature, and is spliced into a global feature vector;

[0083] The global feature vector is reduced using the autoencoder, and the initial flow field is used for constraint to obtain the flow constraint feature vector, and the formula is:

[0084] ,

[0085] Wherein, QW is a flow constraint feature vector, is an autoencoder dimension reduction result, is an initial flow field, is the flow constraint weight of the sample , and is the number of samples;

[0086] The displacement component in the initial flow field is extracted, the flow constraint feature vector is smoothed by bilinear interpolation, and the smoothed feature vector is obtained;

[0087] Based on the smoothed feature vector, linear projection is used for mapping, and L2 norm is used for normalization processing to obtain the final representation.

[0088] By adding noise and decomposing the signal through the ICEEMDAN algorithm, the time-frequency resolution of the signal can be effectively improved, noise interference can be reduced, and the signal characteristics of the data can be enhanced. Through EMD decomposition, the original signal can be decomposed into components of different frequencies, which is convenient for processing high-frequency noise and low-frequency signals separately. In the processing of time series data, Bi-LSTM can capture the before and after information of the sequence, thereby extracting more temporal features. Through specially designed convolution blocks, local and global features in the image can be effectively extracted, and the image's expressiveness can be enhanced. It is suitable for tasks such as image classification, target detection, and image segmentation. Through saliency detection, it can be obtained from the image. The most informative areas are extracted from the image. HFPSO combines the global search capability and local search strategy of the particle swarm optimization algorithm, enabling it to quickly converge to the global optimal solution when dealing with complex optimization problems. Optical flow field technology can reflect the spatiotemporal characteristics of image changes by calculating the motion information of objects in the image. In the process of updating the saliency map and generating the dynamic saliency map, the combination of optical flow field makes the image features more dynamic and accurate. By constructing the global feature vector, a more comprehensive representation is obtained. By combining the multi-head attention mechanism with the dynamic saliency map, the feature vector can be adjusted more accurately, improving the model's performance when processing complex data.

[0089] S2. Based on the high and low frequency sub-bands, multi-scale decomposition is performed using Caputo-Katugampola fractional derivatives to generate an enhanced saliency map. The adaptive diffusion kernel is calculated by shifting the Chebyshev polynomials in combination with the final representation, and the solution is performed using the convolution quotient field to obtain the adjusted high frequency sub-band. Multi-scale wavelet decomposition is performed using the Daubechies wavelet, the Gini index is calculated, and the weighted sub-band coefficients are obtained by screening. The final significant detail layer is obtained by inverse wavelet transform. The initial gradient field is calculated, optimized using fractional gradient features and enhanced energy functionals, and converted to integer order through Hilbert space expansion. It is adjusted using MSB analysis, and the background detail layer is extracted in combination with the ITTI model. The fractional heat equation and smoothing are used, and denoising is performed through wavelet decomposition to obtain the final fusion layer.

[0090] Specifically, based on the high and low frequency sub-bands, the Caputo-Katugampola fractional order derivatives are used for multi-scale decomposition, and the inverse wavelet transform is performed to obtain the final significant detail layer, including:

[0091] Based on the high-frequency sub-band and the low-frequency sub-band, combined with the smooth eigenvector, the Caputo-Katugampola fractional derivative is used to perform multi-scale decomposition to obtain the pixel value function values ​​of the low-frequency sub-band and the high-frequency sub-band;

[0092] Based on the dynamic saliency map and the initial light field, using Caputo-Katugampola sequence derivative, the enhanced saliency map is generated;

[0093] Based on the enhanced saliency map, the bias feature enhancement vector and the final representation, the adaptive diffusion kernel is calculated by shifting Chebyshev polynomial, and the formula is:

[0094] ,

[0095] Wherein, is the adaptive diffusion kernel, is the order of the shifting Chebyshev polynomial, is the order of the approximation coefficient (calculated based on orthogonal projection), is the enhanced state quantity of the enhanced saliency map (based on fractional derivative), is the gradient threshold, is the bias enhancement feature vector, is the final representation, is the regularization parameter, is the order of the shifting Chebyshev polynomial.

[0096] Based on the pixel value function value of the adaptive diffusion kernel, the low frequency subband and the high frequency subband, the fractional PDE is obtained by Mikusiński operation calculus, the Laplace transform is used to convert into algebraic operation in frequency domain into convolution quotient domain, and the finite difference method is used for discretization, and the Hilbert space expansion is used to convert into integer order form, and the Gaver-Stehfest algorithm is used for inverse transformation to obtain the solution of the fractional PDE, and the formula is:

[0097] ,

[0098] ,

[0099] Wherein, is the pixel value function value of the low frequency subband, is the pixel value function value of the high frequency subband, is the divergence operator, is the gradient operator, is the fractional derivative, is the reconstruction denoising data weight, is the low frequency subband in the reconstruction denoising data, is the high frequency subband in the reconstruction denoising data.

[0100] ​The solution of the fractional PDE is approximated using the shifted Chebyshev polynomial to obtain the fused low-frequency subband and high-frequency subband, and the formula is as follows:

[0101] ,

[0102] ,

[0103] wherein, is the fused low-frequency subband, is the fused high-frequency subband, and are approximation coefficients of order , is the approximation order, is the bias constraint weight, and are gradient operators;

[0104] The fused low-frequency subband and high-frequency subband are adjusted using the bias-enhanced feature vector to generate the adjusted low-frequency subband and high-frequency subband, and the formula is as follows:

[0105] ,

[0106] ,

[0107] wherein, and are the adjusted low-frequency subband and high-frequency subband, respectively;

[0108] Based on the adjusted high-frequency subband, the Daubechies wavelet is used as the base function to generate multi-scale subbands through wavelet decomposition to obtain subband coefficients, and the formula is as follows:

[0109] ,

[0110] wherein, is the subband coefficient, is the Daubechies wavelet base function;

[0111] Based on the subband coefficients, the Gini index is calculated, and the formula is as follows:

[0112] ,

[0113] wherein, is the Gini index, is the total number of subband coefficients, is the th element of the subband coefficient , is the th element of the subband coefficient elements;

[0114] Sort the Gini index in descending order, filter the sub-band coefficients that are greater than or equal to the Gini index threshold (set based on the quantile method), calculate the weighting factor of the sub-band coefficient by the initial flow field and the enhanced saliency map, and obtain the weighted sub-band coefficient. The formula is:

[0115] ,

[0116] in, is the weighted subband coefficient, is the initial streamer field, is the normalization parameter;

[0117] The weighted sub-band coefficients are subjected to inverse wavelet decomposition and adjusted in combination with the final representation to obtain the final significant detail layer, which is formulated as follows:

[0118]

[0119] in, For the final salient detail layer, is the result of inverse transform of wavelet decomposition, To adjust the weight.

[0120] Through the multi-scale decomposition of fractional derivatives, richer hierarchical features can be extracted from the signal, especially for the separation of high-frequency and low-frequency sub-bands, which can finely capture the local fluctuations and global trends of the signal. By generating enhanced saliency maps based on Caputo-Katugampola sequence derivatives and combining dynamic features such as optical flow fields, it is possible to efficiently extract salient areas in images or signals, and on this basis improve the accuracy of image or signal processing. The combination of adaptive diffusion kernel calculation and fractional-order PDE solution can efficiently fuse and optimize the pixel values ​​of low-frequency and high-frequency sub-bands, and solve the PDE through Mikusiński operation calculus. It can handle non-integer order differential problems, effectively reduce the impact of noise, and improve the resolution and accuracy of data. By calculating the Gini index to screen the sub-band coefficients, it can effectively identify the most representative and effective feature sub-bands in multi-source data fusion. The inverse wavelet transform is combined with the final representation for adjustment, which can accurately reconstruct the detail layer of the signal after denoising. Through the multi-scale decomposition of the Daubechies wavelet basis function, it can efficiently extract small changes and important features in the signal, further improving the signal quality and detail expression of the data. Sparse coding generates sparse feature vectors, which can further compress the dimension of the data and improve the discriminability of the fused data representation.

[0121] Furthermore, the initial gradient field is calculated and optimized using fractional gradient features and enhanced energy functionals to obtain the final fusion layer, including:

[0122] Based on the final significant detail layer, the spatial gradient is calculated by differential approximation and combined with the initial flow field to obtain the initial gradient field. The formula is:

[0123] ,

[0124] in, is the initial gradient field, is the spatial gradient of the final salient detail layer;

[0125] Based on the initial gradient field, the Caputo-Katugampola fractional derivative is used to generate the fractional gradient feature. The formula is:

[0126] ,

[0127] in, is the fractional gradient feature, is the Caputo-Katugampola fractional derivative operator (calculated based on the Caputo-Katugampola fractional derivative);

[0128] Based on the fractional gradient features and the final significant detail layer, the enhanced energy functional is constructed, and the formula is:

[0129] ,

[0130] in, To enhance the energy functional, For the final salient detail layer The spatial gradient norm of is the Sonine nucleus, is the fractional gradient norm, is the weight, is the integral differential element;

[0131] Based on the enhanced energy functional, the fractional gradient feature is used for optimization to obtain the enhanced significant detail layer. The formula is:

[0132] ,

[0133] in, To enhance the salient detail layer, To enhance the gradient of the energy functional;

[0134] Based on the enhanced salient detail layer, the fractional order gradient flow is converted into integer order using Hilbert space expansion, and the output is the optimized enhanced salient detail layer. The formula is:

[0135] ,

[0136] in, Enhance the salient detail layer for optimization;

[0137] Based on the optimization and enhancement of the salient detail layer, the MSB analysis is used to perform adjustments to obtain the adjusted salient detail layer;

[0138] Based on the adjustment of the significant detail layer, the background detail layer is extracted by the ITTI model, smoothed by the fractional-order heat equation, and denoised by wavelet decomposition to obtain the final fused detail layer. The formula is:

[0139] ,

[0140] in, To smooth the background detail layer, To finally fuse the detail layer, is the fusion weight (calculated by the softmax function based on the enhanced saliency map), is the weight of the high-frequency subband in the reconstructed denoised data.

[0141] Calculating the initial gradient field based on the spatial gradient of the final salient detail layer provides an effective foundation for subsequent fractional gradient feature generation. Using Caputo-Katugampola fractional derivatives to calculate gradient features helps capture subtle changes in the image. By constructing an enhanced energy functional, more local detail features can be effectively introduced into the salient detail layer of the image. Optimizing the fractional gradient features based on the enhanced energy functional can further enhance the salient details in the image, especially under complex backgrounds or low-contrast conditions. The optimization process can effectively improve the quality of details, making the image clearer and sharper. Converting the fractional gradient flow to integer gradients through Hilbert space expansion can simplify computational complexity while preserving key detail features. MSB analysis can remove noise from the image and adjust the salient detail layer to make the main features of the image more prominent. The ITTI model simulates the human eye's attention mechanism and effectively extracts the background detail layer in the image. Smoothing the background detail layer using a fractional heat equation and denoising it through wavelet decomposition helps improve the overall image quality.

[0142] S3. Based on the final fusion layer, sparse coding is used to generate sparse feature vectors to obtain the final data fusion representation;

[0143] Specifically, based on the final fused detail layer, enhanced saliency map and final representation, an initial feature vector is generated through spatial weighting;

[0144] Based on the initial feature vector, incremental learning is used to update the updated feature vector, and the formula is:

[0145] ,

[0146] ,

[0147] in, For time Time data The local eigenvector of For data subdomains of 、 The coordinates are and coordinates The differential increment of (based on the enhanced saliency map), To update the feature vector, is the learning rate, For data The significance weight of

[0148] Based on the updated feature vector, sparse coding is used to generate a sparse feature vector.

[0149] Incremental learning enables the model to be updated instantly when processing new data, reducing the complexity and computational cost of retraining, and capable of processing large-scale data sets. It is particularly suitable for rapidly changing dynamic environments. Sparse coding generates sparse feature vectors, which can remove redundant information while retaining the most representative and effective features. It not only reduces the overhead of data storage and computing, but also improves the efficiency of subsequent processing. By simplifying the representation of data, eliminating noise interference, and focusing key features on a small number of important basic elements, the quality of data fusion and feature representation is improved.

[0150] Furthermore, the final data fusion representation is obtained, including:

[0151] Based on the sparse feature vector, the global feature vector is obtained by aggregation, and the formula is:

[0152] ,

[0153] ,

[0154] in, For time Global eigenvector, For layer Aggregation weights (based on the final fused detail layer), time layer The sparse feature vector of For data Enhanced saliency map of

[0155] Based on the global feature vector and combined with the final fusion detail layer, the final fusion representation is obtained, and the formula is:

[0156] ,

[0157] ,

[0158] in, For the final fusion representation, For time Normalized projection of the global eigenvector, For layer The final fused detail layer.

[0159] By generating global feature vectors based on aggregation based on sparse feature vectors, it can effectively aggregate information from different time periods or modalities, and improve the understanding and integration capabilities of global features. The normalized projection method based on global feature vectors further enhances the discriminability and differentiation capabilities of features, especially in complex and dynamic data scenarios. It can maintain high-quality information expression and support accurate decision-making and prediction, making it widely used in multiple fields such as target detection, behavior analysis, and pattern recognition.

[0160] This embodiment also provides a multi-source data fusion system based on bias identification and embedding, including:

[0161] The processing enhancement module is used to collect multimodal data using the API interface for preprocessing, obtaining the original data set and high- and low-frequency subbands. The original data set is noised using the ICEEMDAN algorithm, decomposed using EMD, and filtered to obtain the denoised IMF. The data is reconstructed and features of the multimodal data are extracted. The data is enhanced using Bi-LSTM and CNN, respectively. The optical flow field is calculated based on the reconstructed denoised data. The DSC model and HEPSO optimization function are used for saliency estimation and separation signal extraction, and a global feature vector is generated. The autoencoder is used for dimensionality reduction and the optical flow field is used for constraint to obtain the final representation.

[0162] Decomposition and fusion module, used to perform multi-scale decomposition based on high- and low-frequency subbands using Caputo-Katugampola fractional derivatives to generate enhanced saliency maps. The adaptive diffusion kernel is calculated by shifting Chebyshev polynomials in combination with the final representation, and solved using the convolution quotient field to obtain the adjusted high-frequency subbands. Multi-scale wavelet decomposition is performed using Daubechies wavelets, the Gini index is calculated, and weighted subband coefficients are obtained by screening. The final salient detail layer is obtained by inverse wavelet transform. The initial gradient field is calculated, optimized using fractional gradient features and enhanced energy functionals, and converted to integer order through Hilbert space expansion. Adjustment is performed using MSB analysis, and the background detail layer is extracted in combination with the ITTI model. Fractional heat equations and smoothing are used to denoise the final fusion layer through wavelet decomposition.

[0163] The sparse representation module is used to generate sparse feature vectors based on the final fusion layer using sparse coding to obtain the final data fusion representation.

[0164] This embodiment also provides a computer device, which is suitable for the multi-source data fusion method based on bias identification and embedding, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the multi-source data fusion method based on bias identification and embedding proposed in the above embodiment.

[0165] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad on the computer device housing, or an external keyboard, touchpad, or mouse.

[0166] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-source data fusion method based on bias identification and embedding as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0167] In summary, the present invention adds noise through the ICEEMDAN algorithm and uses EMD for decomposition, performs saliency estimation and separation signal extraction through the DSC model and HEPSO optimization function, uses autoencoders for dimensionality reduction and uses optical flow fields for constraints, uses Caputo-Katugampola fractional-order derivatives for multi-scale decomposition, calculates the adaptive diffusion kernel through shifted Chebyshev polynomials and uses convolution quotient fields for solution, and uses fractional-order gradient features and enhanced energy functionals for optimization; improves the accuracy and robustness of multi-source data fusion, and realizes efficient data fusion, denoising and feature optimization.

[0168] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A multi-source data fusion method based on bias identification embedding, characterized by: include, Multimodal data is collected and preprocessed using an API to obtain the original dataset and high- and low-frequency subbands. Noise is added to the original dataset using the ICEEMDAN algorithm, decomposed using EMD, and filtered to obtain the denoised IMF. Data reconstruction is then performed to extract features from the multimodal data. This data is then enhanced using Bi-LSTM and CNN, respectively. Optical flow is calculated based on the reconstructed denoised data. Saliency estimation and separation signal extraction are performed using the DSC model and HEPSO optimization function. A global feature vector is generated, dimensionality reduction is performed using an autoencoder, and optical flow is used for constraint to obtain the final representation. Based on high- and low-frequency subbands, multi-scale decomposition is performed using Caputo-Katugampola fractional derivatives to generate enhanced saliency maps. The final representation is combined to calculate the adaptive diffusion kernel by shifting Chebyshev polynomials and solving it using the convolution quotient field to obtain the adjusted high-frequency subbands. Multi-scale wavelet decomposition is performed using Daubechies wavelets, the Gini index is calculated, and weighted subband coefficients are obtained by screening. The final salient detail layer is obtained by inverse wavelet transform. The initial gradient field is calculated and optimized using fractional gradient features and enhanced energy functionals. It is converted to integer order through Hilbert space expansion and adjusted using MSB analysis. The background detail layer is extracted in combination with the ITTI model, and denoised by wavelet decomposition using fractional heat equations and smoothing to obtain the final fusion layer. Based on the final fusion layer, sparse coding is used to generate Sparse feature vector, obtain the final data fusion representation; The preprocessing includes grayscale normalization and resolution alignment of the image data, noise removal by using a bilateral filter, decomposition using a Laplacian pyramid to obtain low-frequency subbands and high-frequency subbands, and applying a SOMP algorithm to the low-frequency subbands to generate a low-dimensional feature vector; Perform mean normalization on the time series data, use sliding average filtering to remove noise, extract frequency domain features through Fourier transform, and generate time series feature vectors; Use NLP to segment text data, remove stop words, and extract stems, and use the BERT model to generate semantic embedding vectors; Extract the original feature vectors in the high-frequency subband, combine them with the low-dimensional feature vectors, time series feature vectors, and semantic embedding vectors, map them through linear transformation, and use L2 regularization for scale alignment to generate a structured feature set; Based on the structured feature set, the Hellinger distance is combined with the Jaccard matrix to calculate the bias distance between features. The formula is: ， in, is the bias distance, and is the eigenvector after Jaccard matrix transformation, For the feature subsets, is the number of feature subsets; Build a GAN, including input layer, hidden layer, bias feature output layer, and classification output layer; Use ImageNet, IoT sensor, and Twitter datasets for preprocessing to obtain training structured feature sets for GAN training; Input the structured feature set into GAN and output a low-dimensional bias feature vector; Concatenate the low-dimensional bias feature vector and the bias distance to obtain the bias-enhanced feature vector; Extract the normalized image data and time series data, combine them with the segmented text data, and splice them into the original dataset.

2. The multi-source data fusion method based on bias identification and embedding according to claim 1, characterized in that: The ICEEMDAN algorithm is used to add noise to the original dataset to obtain the final representation, including: Based on the original dataset, white noise is added using the ICEEMDAN algorithm and decomposed using EMD to generate the final IMF and residual. The denoised IMF is obtained by filtering and denoising using IWT, and the reconstructed denoised data is generated. The enhanced sequence features, enhanced image features, and initial optical flow field are extracted, and the dimensions are aligned using linear transformation. The initial feature vector is obtained by feature concatenation. Based on the image data in the reconstructed denoised data, the initial saliency estimate is obtained through the DSC model, the optimization objective function is constructed, the initial saliency map is output, the candidate separation matrix is ​​obtained through blind source separation, the HFPSO optimization function is constructed, the optimal separation matrix is ​​output, and the separation signal is obtained through ICA; Using the initial optical flow field, the initial saliency map is updated by inverse deformation and combined with the separation signal to obtain a dynamic saliency map. The multi-head attention output and the bias-enhanced feature vector are combined to generate a global feature vector. The autoencoder is used for dimensionality reduction and the initial optical flow field is used for constraint to obtain an optical flow constrained feature vector. The feature vector is smoothed by bilinear interpolation to obtain a smoothed feature vector, which is mapped using linear projection and normalized by the L2 norm to obtain the final representation.

3. The multi-source data fusion method based on bias identification and embedding according to claim 2, characterized in that: The method uses Caputo-Katugampola fractional derivatives to perform multi-scale decomposition based on high and low frequency sub-bands, and performs inverse wavelet transform to obtain the final significant detail layer, including: Based on high-frequency subbands and low-frequency subbands, combined with smooth eigenvectors, Caputo-Katugampola fractional-order derivatives are used for multi-scale decomposition, and Caputo-Katugampola sequence derivatives are used to generate enhanced saliency maps. The adaptive diffusion kernel is calculated by shifting Chebyshev polynomials, and the fractional-order PDE is obtained by Mikusiński operation calculus, and solved by convolution quotient field. The solution of the fractional-order PDE is approximated by shifting Chebyshev polynomials to obtain the fused low-frequency subbands and high-frequency subbands. The biased enhanced eigenvector is used for adjustment, and Daubechies wavelet is used as the basis function. Multi-scale subbands are generated by wavelet decomposition to obtain subband coefficients, and the Gini index is calculated. The weighted subband coefficients are obtained by screening, and the inverse wavelet decomposition transform is performed. The final representation is combined for adjustment to obtain the final significant detail layer.

4. The multi-source data fusion method based on bias identification and embedding according to claim 3, characterized in that: The initial gradient field is calculated and optimized using fractional gradient features and enhanced energy functionals to obtain the final fusion layer, including: Based on the final salient detail layer, the initial gradient field is obtained by differential approximation. The Caputo-Katugampola fractional derivatives are used to generate fractional gradient features. Combined with the final salient detail layer, an enhanced energy functional is constructed. The fractional gradient features are used for optimization. The fractional gradient flow is converted to integer order using Hilbert space expansion. The output is the optimized enhanced salient detail layer, which is adjusted using MSB analysis to obtain the adjusted salient detail layer. The background detail layer is extracted using the ITTI model, smoothed using the fractional heat equation, and denoised through wavelet decomposition to obtain the final fused detail layer.

5. The multi-source data fusion method based on bias identification and embedding according to claim 4, characterized in that: Based on the final fusion layer, sparse coding is used to generate Sparse feature vectors, including: Based on the final fused detail layer, enhanced saliency map and final representation, an initial feature vector is generated by spatial weighting, which is updated using incremental learning to obtain an updated feature vector, and sparse coding is used to generate a sparse feature vector.

6. The multi-source data fusion method based on bias identification and embedding according to claim 5, characterized in that: The final data fusion representation is obtained, including: Based on the sparse feature vector, the global feature vector is obtained by aggregation, and combined with the final fusion detail layer to obtain the final fusion representation.

7. The multi-source data fusion method based on bias identification and embedding according to claim 6, characterized in that: The API interface is used to collect multimodal data for preprocessing to obtain the original data set and high and low frequency sub-bands, including: Use the API interface to collect multimodal data, including images, time series, and text data, and perform preprocessing to obtain low-frequency subbands, high-frequency subbands, original data sets, and bias-enhanced feature vectors.

8. A multi-source data fusion system based on bias identification and embedding, based on the multi-source data fusion method based on bias identification and embedding according to any one of claims 1 to 7, characterized in that: include, The processing enhancement module is used to collect multimodal data using the API interface for preprocessing, obtaining the original data set and high- and low-frequency subbands. The original data set is noised using the ICEEMDAN algorithm, decomposed using EMD, and filtered to obtain the denoised IMF. The data is reconstructed and features of the multimodal data are extracted. The data is enhanced using Bi-LSTM and CNN, respectively. The optical flow field is calculated based on the reconstructed denoised data. The DSC model and HEPSO optimization function are used for saliency estimation and separation signal extraction, and a global feature vector is generated. The autoencoder is used for dimensionality reduction and the optical flow field is used for constraint to obtain the final representation. Decomposition and fusion module, used to perform multi-scale decomposition based on high- and low-frequency subbands using Caputo-Katugampola fractional derivatives to generate enhanced saliency maps. The adaptive diffusion kernel is calculated by shifting Chebyshev polynomials in combination with the final representation, and solved using the convolution quotient field to obtain the adjusted high-frequency subbands. Multi-scale wavelet decomposition is performed using Daubechies wavelets, the Gini index is calculated, and weighted subband coefficients are obtained by screening. The final salient detail layer is obtained by inverse wavelet transform. The initial gradient field is calculated, optimized using fractional gradient features and enhanced energy functionals, and converted to integer order through Hilbert space expansion. Adjustment is performed using MSB analysis, and the background detail layer is extracted in combination with the ITTI model. Fractional heat equations and smoothing are used to denoise the final fusion layer through wavelet decomposition. The sparse representation module is used to generate sparse feature vectors based on the final fusion layer using sparse coding to obtain the final data fusion representation.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the multi-source data fusion method based on bias identification and embedding according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-source data fusion method based on bias identification and embedding according to any one of claims 1 to 7 are implemented.

Images