A method for fusion and reconstruction of optical fiber signals of multi-source heterogeneous data

Abstract

The application discloses a kind of multi-source heterogeneous data's optical fiber signal fusion and reconstruction method.The method is by optimizing the point spread function characteristics of optical fiber signal mapping physical model, then, introduce self-attention mechanism, drive the fusion of multi-source heterogeneous data, extract texture and structure information from multi-granularity PSF feature, through PSF fusion and adaptive weight, the expression ability of feature after fusion is improved.Through the decomposition of multi-source heterogeneous data and noise suppression integration into an optimization framework, multi-source heterogeneous data fusion enhancement is realized.Through serial decomposition, the structure layer of multi-source heterogeneous data is estimated, Gaussian regularization is applied for optimization, and finally the structure layer of multi-source heterogeneous data is merged to reconstruct.The application realizes the learning and fusion of multi-granularity PSF feature by introducing PSF fusion multi-granularity feature learning module, effectively captures the multi-scale information of heterogeneous data.Adopting double update and model optimization strategy, the reconstruction of multi-source heterogeneous data is realized by optimizing model parameters.

Description

Technical Field

[0001] This invention belongs to the field of intelligent evaluation and analysis technology, specifically, it relates to a method for fiber optic signal fusion and reconstruction of multi-source heterogeneous data. Background Technology

[0002] At present, the field of multi-source heterogeneous data fusion in optical fiber signals faces a series of challenges. Regarding information extraction from optical fiber signals, the physical models used often yield low-quality key data, limiting the accuracy of subsequent applications and analyses. Especially in complex scenarios, the fusion process struggles to capture all details, leading to increasingly significant distortion and information loss in multi-source heterogeneous data fusion. Simultaneously, the complexity of the computational process places high demands on hardware and computing resources. On the other hand, multi-source heterogeneous data fusion faces the challenge of data heterogeneity; different data sources have diverse representations, distributions, and characteristics, and effectively fusing them requires addressing the problems arising from this heterogeneity. Furthermore, the information richness provided by different data sources may vary, posing a challenge to balance different information during fusion. The data may contain noise, inconsistencies, and errors, requiring solutions for accurate cleaning and correction during the fusion process. Additionally, since the data involves different spatiotemporal scales, modeling spatiotemporal relationships to extract more accurate information is also a crucial issue.

[0003] Therefore, it is necessary to achieve more accurate and high-quality fusion and reconstruction of multi-source heterogeneous data from optical fiber signals. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method for fiber optic signal fusion and reconstruction of multi-source heterogeneous data.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A method for fiber optic signal fusion and reconstruction of multi-source heterogeneous data includes: physical model optimization of multi-source heterogeneous data, self-attention-driven multi-source heterogeneous data fusion, and enhanced multi-source heterogeneous data reconstruction.

[0007] 1) Physical model optimization for multi-source heterogeneous data

[0008] S1: Using PSF as the physical mechanism model of fiber optic signals, a weighted point spread function is superimposed on multi-source heterogeneous data. The image generated from the multi-source heterogeneous data is then represented as:

[0009] ,

[0010] in, The plane function value represents the object, i.e., the fiber optic signal to be measured; M represents the image magnification factor, used to adjust the position of the light signal on the image; and PSF represents the point spread function. This represents the numerical value of a point in an image, i.e., multi-source heterogeneous data. This indicates the position of the point in the image coordinate system;

[0011] S2: Use a weighted sparse coding model to filter out Gaussian noise in fiber optic signals;

[0012] S3: Decompose the multi-source heterogeneous data, extract the texture layer and structure layer features, and apply Gaussian regularization for optimization to generate reconstructed data of the multi-source heterogeneous data;

[0013] 2) Self-attention driven multi-source heterogeneous data fusion

[0014] S4: Introduce a self-attention-based PSF fusion multi-granularity feature learning module to extract PSF fusion multi-granularity features from multi-source heterogeneous data;

[0015] S5: Design a PSF fusion method based on maximizing output to fuse features of different granularities and generate fused features;

[0016] 3) Enhance the reconstruction of multi-source heterogeneous data

[0017] S6: Estimate the fusion features to generate fusion feature estimates;

[0018] S7: Connect the fused features generated from two adjacent iterations along the channel, and then pass them through two parameterized channels. The convolutional layers are subjected to convolutional processing to achieve dual updates;

[0019] S8: Connect the fused feature estimates generated from two adjacent iterations along the channels, and parameterize them as follows: The two convolutional layers are convolutionally processed, and the convolution result is weighted and added to the original fusion feature estimate to update the original domain;

[0020] S9: Optimize the model tensor initialized by the point spread function (PSF) obtained by measuring point fiber signals using the original dual network model.

[0021] Furthermore, S2 specifically includes:

[0022] Determine using the following formula Sparse coding representation:

[0023] ,

[0024] in, , and These represent the operations of the 1-norm and 2-norm, respectively. , express sparse basis vectors of dimension, This represents the dimension of the sparse basis vectors. The dimension of the sparse coding is represented. Represents the set of real numbers. This represents the trade-off parameter used to balance error and sparsity, where, Through training sample set To determine this, we need to solve the following objective function minimization problem:

[0025] .

[0026] Further, S3 includes:

[0027] nuclear width Gaussian kernel function and kernel width Gaussian kernel function They are respectively introduced into and In, among them, This indicates that the Gaussian kernel function was applied. The inverse process of the total variation regularization term is used to extract... Fine-grained texture layer features in the middle. This indicates that the Gaussian kernel function was applied. The inverse process of the subsequent multi-dictionary low-rank constraint regularization term is used to extract... Large-scale structural layer features in;

[0028] Determine the LRGV regularization term ,

[0029] in,

[0030]

[0031]

[0032] express gradient, This represents the sum of the weighted spread functions. This represents the convolution operation. This indicates element-wise multiplication. Represents a weighted matrix;

[0033] The reconstructed data of multi-source heterogeneous data is obtained through the following objective function.

[0034]

[0035] in, Variables representing reconstructed data, Let represent the regularization parameters. The first term of the objective function is the Frobenius criterion fidelity term, and the second term is the LRGV regularization term. This represents the F-norm.

[0036] Further, S4 includes:

[0037] Determine the feature map of any data block in multi-source heterogeneous data;

[0038] The content vector of the arbitrary data block is determined using a self-attention mechanism. ,

[0039] in,

[0040] ,

[0041] ,

[0042] ,

[0043] ,

[0044] ,

[0045] These are the attention weight coefficients after softmax processing. This is a matrix-vector structure processed by the tanh activation function. and For a learnable parameter matrix, and For learnable bias values, and This represents a function that performs mathematical processing on the feature map. Represents data block The corresponding feature map, Represents data block The corresponding feature map.

[0046] Further, S5 includes:

[0047] Let the PSF fusion of multi-granularity features be denoted as and divided into Groups, each group contains One feature channel;

[0048] Within each group, The inputs of each feature channel are subjected to Maxout operation to obtain the Maxout features, denoted as . , , ;

[0049] The Maxout features within each group are average-pooled and concatenated to obtain the final PSF features, denoted as . ;

[0050] The final PSF features, texture layer features, and structure layer features are weighted and averaged to obtain the fused features. ,Right now

[0051] ,

[0052] in, and Represents feature weights, This represents the features of the texture layer and the structure layer.

[0053] Further, S6 includes:

[0054] By solving the regularization optimization problem, the fusion features are estimated, which can be expressed by the formula:

[0055]

[0056] in, This represents circular convolution, where G indicates that the result is cropped to the sensor size. Indicates the measurement of fiber optic signals. The parameter representing the degree of regularization is related to the data fidelity term. It is a regularization function. Indicates fusion features, This represents the estimated value of the fusion feature.

[0057] Further, S9 includes:

[0058] Assumption The parameters of the forward model of the original dual network model are replaced with circular convolutions to replace the forward operators in the forward model. and adjacency operator ,Right now

[0059] ,

[0060] ,

[0061] in, , , representing the main variable and the dual variable, respectively. Indicates fiber optic signal measurement, This indicates multi-source heterogeneous data measurement, where P indicates zero-padding at twice the size of the image sensor. Indicates circular cross-correlation;

[0062] Initialize using PSF ,Right now ,

[0063] ,

[0064] ,

[0065] Learn n convolutional kernels, each kernel corresponding to a main variable, a dual variable, and a parameter. ,Right now

[0066] ,

[0067] ,

[0068] ,

[0069] Update using the following formula:

[0070] ,

[0071] ,

[0072] in, and This represents a small convolutional neural network during the iterative process. After parameterization and iteration, then... Selected as The best estimate.

[0073] The beneficial effects of this invention are:

[0074] This method first addresses the problem of extracting key data from fiber optic signals by using the point spread function (PSF) to overcome the resolution and quality limitations of traditional methods, thereby obtaining clearer and more accurate data results. Secondly, it addresses the problem of multi-source heterogeneous data fusion by constructing effective data fusion models and strategies to effectively integrate information from different data sources, thus providing more comprehensive and richer data information.

[0075] This invention integrates multi-granularity fiber signal point spread function (PSF) feature fusion with a self-attention mechanism to achieve efficient fusion of multi-source heterogeneous data. By introducing a multi-granularity learning module for PSF feature fusion, this method can learn multi-granularity PSF features based on uniform grid sampling. Furthermore, through the self-attention mechanism, PSF features of different granularities are driven to fuse, automatically capturing the correlations between features of different granularities. This innovative combination of multi-granularity PSF feature fusion and the self-attention mechanism enables the model to more accurately extract and express the key structural information of multi-source heterogeneous data.

[0076] This invention also employs an integrated strategy of dual update and model optimization, unifying multi-source heterogeneous data decomposition and noise suppression into a single optimization framework. Through dual update, the method fuses variables from the measurement domain and the original domain, and uses convolutional layers for updating, achieving progressive optimization of multi-source heterogeneous data. Simultaneously, a model optimization strategy is introduced to iteratively optimize the parameters of the forward model and the adjoint model, thereby better controlling the quality of the reconstructed multi-source heterogeneous data.

[0077] Furthermore, this method can also handle issues such as noise and spatiotemporal scale inconsistencies between data, further improving the quality of the fusion results. Through its application in the field of multi-source heterogeneous data fusion of optical fiber signals, this innovative method is expected to provide a new approach for achieving more accurate and comprehensive optical fiber signal data fusion tasks, with broad application prospects and far-reaching impact.

[0078] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0079] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0080] Figure 1 This is a schematic flowchart of a fiber optic signal fusion and reconstruction method for multi-source heterogeneous data;

[0081] Figure 2 This is a schematic diagram of the optimization of the physical model for multi-source heterogeneous data;

[0082] Figure 3 This is a schematic diagram of the self-attention fusion module;

[0083] Figure 4 This is a schematic diagram of an optical fiber signal reconstruction model. Detailed Implementation

[0084] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the preferred embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0085] Figure 1 This is a schematic flowchart illustrating a method for fiber optic signal fusion and reconstruction of multi-source heterogeneous data. (For example...) Figure 1 As shown, the method includes physical model optimization of multi-source heterogeneous data, self-attention-driven multi-source heterogeneous data fusion, and enhanced multi-source heterogeneous data reconstruction.

[0086] 1) Physical model optimization for multi-source heterogeneous data

[0087] Please refer to the following description for details. Figure 2 .

[0088] S1: Using the PSF as the physical mechanism model for fiber optic signals, it can be used to guide the fusion of multi-source heterogeneous data. The point spread function can be independent of its position in the object plane; in this case, it is called shift invariance. By superimposing a weighted point spread function onto the multi-source heterogeneous data, the image generated from the multi-source heterogeneous data is represented as:

[0089] ,

[0090] in, The plane function value represents the object, i.e., the fiber optic signal to be measured; M represents the image magnification factor, used to adjust the position of the light signal on the image; and PSF represents the point spread function. This represents the numerical value of a point in an image, i.e., multi-source heterogeneous data. This indicates the position of the point in the image coordinate system. The values ​​of these points are usually calculated by image processing algorithms and are used to represent the characteristics of the image at different locations.

[0091] Due to the inherent limited resolution of multi-source heterogeneous data systems, the measured PSF is not without uncertainty. In a Gaussian-distributed transmission system, the PSF can be modeled using the following formula:

[0092] ,

[0093] in, The factor depends on the cutoff rate and the irradiance level. It is the numerical aperture. It's the speed of light. It is the photon frequency of the imaging beam. Indicates the intensity of the reference beam. It is an adjustment factor. It is the corresponding The radial position on the plane starting from the center.

[0094] S2: Use a weighted sparse coding model to filter out Gaussian noise in fiber optic signals;

[0095] If known ,in, express sparse basis vectors of dimension, The dimension of the sparse basis vectors represents the length of each sparse basis vector or the number of features. Sparse coding The dimension, i.e., the length of the sparse coding vector or the number of features it represents. Let the set of real numbers be represented. Then, for the input signal... Sparse coding This can be solved by the following method. The norm minimization problem yields the following:

[0096] ,

[0097] Among them, the input signal This refers to the value after the PSF has been calculated, then the data has been divided into blocks and vectorized. and These represent the operations of the 1-norm and 2-norm, respectively. This represents the trade-off parameter used to balance error (the first term in the above equation) and sparsity (the second term in the above equation). The formula is essentially a linear regression problem known as Lasso, which can be solved using the LARS algorithm. After solving the formula, the input signal can be obtained. The sparse representation is as follows:

[0098] .

[0099] Through the above input signals As can be seen from the sparse representation, if given Then, the input signal can be approximately represented by a sparse representation. Wherein, It can be obtained through training sample set To determine this, we need to solve the following objective function minimization problem:

[0100] .

[0101] Given heterogeneous input data from multiple sources, it can be divided into a set of data blocks that overlap 50% with each other, assuming that the bytes of the heterogeneous data have the same amount of overlap. For example, if there are N data blocks, then there are a total of 2N-1 overlapping data. The data block size = total data volume / (2N - 1). Here, the total data volume is the total amount of data to be divided, and N is the number of data blocks. The data blocks are sorted sequentially from the top left corner to the bottom right corner. It is the first One data block. Vectorization, the process of which is as follows, involves flattening image pixels into a one-dimensional array in multi-source heterogeneous data. For example, if an image is 100x100 pixels, it is converted into a vector containing 10,000 elements. Audio data can be vectorized using Mel-frequency cepstral coefficients, and text data can be converted into word feature vectors using term frequency-inverse document frequency, etc., to obtain the input signal in the formula. .

[0102] S3: Decompose the multi-source heterogeneous data, extract texture and structure layer features, and apply Gaussian regularization for optimization to generate reconstructed data of the multi-source heterogeneous data.

[0103] First, the structural layer of the heterogeneous data is estimated. This structural layer contains shared structural features from various data sources, and a low-rank prior is applied to eliminate noise. Finally, a weighted synthesis is performed between the fine-texture layer and the structural layer as a merging of the structural layers. For multi-source heterogeneous data, the structural layer contains shared structural features from various data sources, such as common targets or the overall shape of the scene. By separating the structural layer from other data features, multi-source heterogeneous data can be better understood and analyzed, leading to more accurate data fusion. Therefore, when processing multi-source heterogeneous data, the structural layer can refer to the shared structural information in the data to better understand and utilize its characteristics.

[0104] nuclear width Gaussian kernel function and kernel width Gaussian kernel function They are respectively introduced into and In, among them, This indicates that the Gaussian kernel function was applied. The inverse process of the total variation regularization term is used to extract... Fine-grained texture layer features in the middle. This indicates that the Gaussian kernel function was applied. The inverse process of the subsequent multi-dictionary low-rank constraint regularization term is used to extract... Large-scale structural layer features in;

[0105] Next, determine the LRGV regularization term. ,

[0106] in,

[0107]

[0108] express The gradient indicates local features in an image, such as edges and textures. This refers to the input data itself. The relationship between the two is... It is about data information The results obtained by applying the gradient operator are used to extract features. This represents the sum of the weighted spread functions to describe the characteristics at different locations. This represents the convolution operation. This indicates element-wise multiplication. This represents a weighted matrix.

[0109] By combining the two terms, the final result, LRGV, is obtained. Observations show that fine texture data is extracted into the texture layer, while edges are better preserved in the structure layer. Therefore, LRGV-based regularization can better perform edge-preserving filtering tasks and obtain accurate fine texture data and structure layers.

[0110] The reconstructed data of multi-source heterogeneous data is obtained through the following objective function.

[0111]

[0112] in, Variables representing reconstructed data, The regularization parameter is represented by the first term of the objective function, which is the Frobenius criterion fidelity term, guaranteeing... and The overall similarity between them; the second term is the LRGV regularization term. This represents the F-norm.

[0113] pass Figure 2 As described above, using a weighted sparse coding model to process Gaussian noise in fiber optic signals helps filter out noise and extract key features. Multi-source heterogeneous data is decomposed, and Gaussian regularization is applied to optimize the data. The structural layers of the data are estimated, including shared structural features, to eliminate noise. The fine texture layer and the structural layer are weighted and synthesized to obtain the final data reconstruction. This step preserves edge information and accurately extracts fine texture data.

[0114] 2) Self-attention driven multi-source heterogeneous data fusion

[0115] Please refer to the following description for details. Figure 3 .

[0116] S4: Introduce a PSF fusion multi-granularity feature learning module based on self-attention. This module can learn multi-granularity features on the basis of uniform grid sampling, that is, extract PSF fusion multi-granularity features from multi-source heterogeneous data.

[0117] This step specifically includes:

[0118] Determine the feature map of any data block in multi-source heterogeneous data;

[0119] The content vector of the arbitrary data block is determined using a self-attention mechanism. ,

[0120] in,

[0121] ,

[0122] ,

[0123] ,

[0124] ,

[0125] ,

[0126] These are the attention weights after softmax processing, used to capture feature maps. and The similarity of the regions they represent This is a matrix-vector structure processed by the tanh activation function. and For a learnable parameter matrix, and For learnable bias values, and This represents a function that performs mathematical processing on the feature map (such as linear transformation, similarity calculation, etc.). Represents data block The corresponding feature map, Represents data block The corresponding feature map.

[0127] S5: Design a PSF-based multi-granularity feature fusion method that maximizes output. This method fuses features of different granularities to generate fused features. The aim of this step is to fuse features from different granularities to further extract comprehensive and high-quality feature representations. To reduce feature dimensionality, a PSF feature hierarchical fusion method is employed.

[0128] Let the PSF fusion of multi-granularity features be denoted as and divided into Groups, each group contains One feature channel;

[0129] Within each group, The inputs of each feature channel are subjected to Maxout operation to obtain the Maxout features, denoted as . , , ;

[0130] The Maxout features within each group are average-pooled and concatenated to obtain the final PSF features, denoted as . ;

[0131] Feature hierarchical fusion combines feature information from different levels to obtain a more comprehensive and richer feature representation. Weighted averaging is used for feature hierarchical fusion, which involves weighting the final PSF features and features from the texture and structure layers to obtain the fused features. ,Right now

[0132] ,

[0133] in, and Represents feature weights, This represents the features of the texture layer and the structure layer.

[0134] 3) Enhance the reconstruction of multi-source heterogeneous data

[0135] Please refer to the following description for details. Figure 4 .

[0136] S6: The input consisting of fused fiber optic signal measurements is batch-processed in a certain hierarchy, and the fused features are estimated to generate fused feature estimates.

[0137] During training and inference, the model preprocesses the multi-source heterogeneous input data. This preprocessed input will then be used as input to the model for subsequent double update and original update steps.

[0138] The measurement result of fiber optic signal b is the result of a linear transformation A on a point in scene x, and includes some additional noise. :

[0139] ,

[0140] Here, b and x are both vectors. Each column of A corresponds to a linear transformation of a point in the scene, also known as a PSF. Storing the PSF of every point in memory is a very demanding task. Compared to storing all PSFs, A can be approximated as a sheared convolution with the PSF measured along the optical axis using an aperture.

[0141] ,

[0142] here, Let represent circular convolution, and G denote cropping the result to the size of the imaging sensor. When reconstructing multi-source heterogeneous data using the convolutional feedforward model, a single experimentally measured PSF is typically used. On-axis PSF is usually measured by illuminating a point source along the optical axis of the existing system. Assuming b is the result of convolution with the experimentally measured PSF, the fused features are estimated by solving a regularization optimization problem, expressed by the formula:

[0143]

[0144] in, Indicates the measurement of fiber optic signals. The parameter representing the degree of regularization is related to the data fidelity term. It is a regularization function used to penalize impossible solutions when noise is present. Indicates fusion features, This represents the estimated value of the fusion feature.

[0145] S7: Connect the fused features generated from two adjacent iterations along the channel, and then pass them through two parameterized channels. The convolutional layers are convolutionally processed to achieve dual updates. Specifically, the variables in the measurement domain are compared with the previous variables (i.e., ... , Connect it to b). Update the connected tensor Mi parameterizedly through two convolutional layers. These two convolutional layers share parameters. The convolutional layer performs convolution operations on the input tensor based on its spatial structure and channel information, and generates new feature maps.

[0146] , 'b' and 'b' are variables in the measurement domain, representing the input after fiber optic signal measurement fusion. These variables are concatenated along the channel dimension during the dual update step. The two parameterized convolutional layers are used for processing.

[0147] S8: Connect the fused feature estimates generated from two adjacent iterations along the channels, and parameterize them as follows: The two convolutional layers are convolutionally processed, and the convolution result is weighted and added to the original fused feature estimate to update the original domain.

[0148] Specifically, the variables in the measurement domain , Connect them along the channel dimension. The result of the connection is that these variables are placed in the same tensor, forming a new tensor. The shape of this tensor is H×W×(2C), where H represents the height, W represents the width, and C represents the number of channels. This join operation connects the current original domain variables... And the previous primitive field variables They are placed in the same tensor.

[0149] By two parameters The convolutional layer connects the tensors Perform a convolution operation. These two convolutional layers share parameters. In other words, they use the same parameters for convolution operations. During the update of the original domain, the convolutional layers adjust the input tensor based on its spatial structure and channel information. Perform a convolution operation to generate a new feature map.

[0150] The updated original domain is obtained by weighted summing of the output of the convolutional layer with the original variables. This weighted summation can also be implemented using convolution operations, where the convolution kernel is used to learn the weights. Updating the original domain updates the variables in the original domain. and It is fused with the feature map output from the convolutional layer to produce a more accurate update result.

[0151] and It is also a variable in the measurement domain, through Two parameterized convolutional layers are used for the original update step.

[0152] Combination Figure 4 UNet is a classic deep learning network architecture used for semantic segmentation tasks. It consists of an encoder and a decoder, and includes symmetrical convolutional layers and upsampling layers. Deep neural networks are primarily convolutional neural networks, and UNet is a neural network architecture whose feature concatenation method helps the network better preserve high-level semantic information and improves reconstruction accuracy.

[0153] S9: Utilizing the primal dual network model (i.e. Figure 4 The fiber optic signal reconstruction model with multi-source heterogeneous data depth calibration optimizes the model tensor initialized by the point spread function (PSF) obtained by point fiber optic signal measurement.

[0154] A forward model is a model that maps the input domain to the output domain. In this scenario, it is a model that maps input fiber optic signal measurements to the reconstruction of output multi-source heterogeneous data. These parameters represent the forward model and are used to control the model's mapping process. These parameters determine how the model converts input fiber optic signal measurements into output model tensors.

[0155] The adjoint model is the transpose or inverse of the forward model. It is used to calculate the derivative of the loss function with respect to the parameters of the forward model. Here, the parameters of the adjoint model are also... The role of the adjoint model is to guide the update process of model parameters by calculating gradients, in order to minimize the loss function.

[0156] Replace the forward operator in the forward model with circular convolution. and adjacency operator ,Right now

[0157] ,

[0158] ,

[0159] in, , , representing the main variable and the dual variable, respectively. Indicates fiber optic signal measurement, This indicates multi-source heterogeneous data measurement, where P indicates zero-padding at twice the size of the image sensor. This indicates a circular cross-correlation.

[0160] Allows optimization of PSF during training. Initialize using PSF. ,Right now Allows the network to modify the fiber optic signal model PSF during training:

[0161] ,

[0162] ,

[0163] The PSF can be optimized during training. n convolutional kernels are learned, each kernel corresponding to one main variable, one dual variable, and one parameter. ,Right now

[0164] ,

[0165] ,

[0166] ,

[0167] Each main variable and dual variable , All of them are related to the core they have learned. Perform convolution or cross correlation.

[0168] The learned algorithm variants are updated using the following formula:

[0169] ,

[0170] ,

[0171] in, and This represents a small convolutional neural network during the iterative process. After parameterization and iteration, then... Selected as The best estimate.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for fiber optic signal fusion and reconstruction of multi-source heterogeneous data, characterized in that, include: Physical model optimization for multi-source heterogeneous data, self-attention-driven multi-source heterogeneous data fusion, and enhanced multi-source heterogeneous data reconstruction; 1) Physical model optimization for multi-source heterogeneous data S1: Using PSF as the physical mechanism model of fiber optic signals, a weighted point spread function is superimposed on multi-source heterogeneous data, then the coordinate points... The corresponding multi-source heterogeneous data is represented as follows: ， in, The plane function value represents the object, i.e., the fiber optic signal to be measured; M represents the image magnification factor, used to adjust the position of the light signal on the image; and PSF represents the point spread function. This represents the numerical value of a point in an image, i.e., multi-source heterogeneous data. This indicates the position of the point in the image coordinate system; S2: Use a weighted sparse coding model to filter out Gaussian noise in fiber optic signals, specifically including: Determine using the following formula Sparse coding representation: ， Among them, the input signal Sparse coding refers to the value obtained after calculating the PSF, dividing the data into blocks, and vectorizing the result. , and These represent the operations of the 1-norm and 2-norm, respectively. , express sparse basis vectors of dimension, This represents the dimension of the sparse basis vectors. The dimension of the sparse coding is represented. Represents the set of real numbers. This represents the trade-off parameter used to balance error and sparsity, where, Through training sample set To determine this, we need to solve the following objective function minimization problem: ； S3: Decompose the multi-source heterogeneous data, extract the texture layer and structure layer features, and apply Gaussian regularization for optimization to generate reconstructed data of the multi-source heterogeneous data; 2) Self-attention driven multi-source heterogeneous data fusion S4: Introduce a self-attention-based PSF fusion multi-granularity feature learning module to extract PSF fusion multi-granularity features from multi-source heterogeneous data; S5: Design a PSF fusion method based on maximizing output to fuse features of different granularities and generate fused features; 3) Enhance the reconstruction of multi-source heterogeneous data S6: Estimate the fusion features to generate fusion feature estimates; S7: Connect the fused features generated from two adjacent iterations along the channel, and then pass them through two parameterized channels. The convolutional layers are subjected to convolutional processing to achieve dual updates; S8: Connect the fused feature estimates generated from two adjacent iterations along the channels, and parameterize them as follows: The two convolutional layers are convolutionally processed, and the convolution result is weighted and added to the original fusion feature estimate to update the original domain; S9: Optimize the model tensor initialized by the point spread function (PSF) obtained by measuring point fiber signals using the original dual network model.

2. The method for fiber optic signal fusion and reconstruction of multi-source heterogeneous data according to claim 1, characterized in that, S3 includes: With a kernel width of Gaussian kernel function and kernel width is Gaussian kernel function They are respectively introduced into and In, among them, This indicates that the Gaussian kernel function was applied. The inverse process of the total variation regularization term is used to extract... Fine-grained texture layer features in the middle. This indicates that the Gaussian kernel function was applied. The inverse process of the subsequent multi-dictionary low-rank constraint regularization term is used to extract... Large-scale structural layer features in; Determine the LRGV regularization term ,in, express gradient, This represents the sum of the weighted spread functions. This represents the convolution operation. This indicates element-wise multiplication. Represents a weighted matrix; The reconstructed data of multi-source heterogeneous data is obtained through the following objective function. : in, Variables representing reconstructed data, Let represent the regularization parameters. The first term of the objective function is the Frobenius criterion fidelity term, and the second term is the LRGV regularization term. This represents the F-norm.

3. The method for fiber optic signal fusion and reconstruction of multi-source heterogeneous data according to claim 1, characterized in that, S4 includes: Determine the feature map of any data block in multi-source heterogeneous data; The content vector of the arbitrary data block is determined using a self-attention mechanism. , in, ， ， ， ， ， These are the attention weight coefficients after softmax processing. This is a matrix-vector structure processed by the tanh activation function. and For a learnable parameter matrix, and For learnable bias values, and This represents a function that performs mathematical processing on the feature map. Represents data block The corresponding feature map, Represents data block The corresponding feature map.

4. The fiber optic signal fusion and reconstruction method for multi-source heterogeneous data according to claim 1, characterized in that, S5 includes: Let the PSF fusion of multi-granularity features be denoted as and divided into Groups, each group contains One feature channel; Within each group, The inputs of each feature channel are subjected to Maxout operation to obtain the Maxout features, denoted as . , , ; The Maxout features within each group are average-pooled and concatenated to obtain the final PSF features, denoted as . ; The final PSF features, texture layer features, and structure layer features are weighted and averaged to obtain the fused features. ,Right now ， in, and Represents feature weights, This represents the features of the texture layer and the structure layer.

5. The fiber optic signal fusion and reconstruction method for multi-source heterogeneous data according to claim 1, characterized in that, S6 includes: By solving the regularization optimization problem, the fusion features are estimated, which can be expressed by the formula: in, This represents circular convolution, where G indicates that the result is cropped to the sensor size. Indicates the measurement of fiber optic signals. The parameter representing the degree of regularization is related to the data fidelity term. It is a regularization function. Indicates fusion features, This represents the estimated value of the fusion feature.

6. The method for fiber optic signal fusion and reconstruction of multi-source heterogeneous data according to claim 5, characterized in that, S9 includes: Assumption The parameters of the forward model of the original dual network model are replaced with circular convolutions to replace the forward operators in the forward model. and adjacency operator ,Right now ， ， in, , , representing the main variable and the dual variable, respectively. Indicates fiber optic signal measurement, This indicates multi-source heterogeneous data measurement, where P indicates zero-padding at twice the size of the image sensor. Indicates circular cross-correlation; Initialize using PSF ,Right now , ， ， Learn n convolutional kernels, each kernel corresponding to a main variable, a dual variable, and a parameter. ,Right now ， ， ， Update using the following formula: ， ， in, and This represents a small convolutional neural network during the iterative process. After parameterization and iteration, then... Selected as The best estimate.

Images