Distributed multi-source heterogeneous information fusion method based on Transform architecture

Abstract

According to the distributed multi-source heterogeneous information fusion method based on the Transform architecture, a multi-level distributed feature fusion framework is provided, secondary feature extraction fusion is achieved by adopting an end-to-end self-attention mechanism encoder architecture based on heterogeneous source information primary features, the calculation difficulty is reduced, and balance between the calculation cost and the time efficiency is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of signal processing, and in particular relates to a distributed multi-source heterogeneous information fusion method based on a Transformer architecture. Background Art

[0002] With the development of Internet of Things technology, the types of monitoring information collection are becoming more and more abundant, including time series information, text information, image information, etc. When directly using source information for fault diagnosis, command decision-making, image analysis and understanding, and unmanned driving, since only single information analysis can be used separately and then comprehensive judgment can be made, the accuracy of the results will inevitably be affected. Therefore, the multi-source heterogeneous information fusion method has become a basic problem in the field of signal processing and the focus of many industry scholars, forming a large number of research results and widely used. Information fusion includes three levels: data layer fusion, feature layer fusion, and decision layer fusion. It can be divided into three typical methods: direct operation on data source, based on object statistical features and probability model, and based on rule reasoning. They all aim to form a unified expression form of multi-source heterogeneous information.

[0003] At present, feature layer fusion is the main research hotspot of multi-source heterogeneous information fusion, but it still lacks the ability to effectively fuse low-quality multimodal data in the real world. This is because: ①Unknown environmental factors, sensor failures, and signal loss during transmission may introduce noise and reduce the reliability of the fusion method. ②Incomplete information collected reduces the reliability of the fusion method. ③The heterogeneous encoding properties and information quality inconsistency between information lead to excessive reliance on certain information by the fusion method, which damages the reliability of the method. ④The application environment is complex and changeable. Different samples, different time and space, and the quality of heterogeneous information have dynamic characteristics, which brings challenges to the fusion method. In order to solve the above problems, neural networks, fuzzy theory, and DS evidence theory are applied to feature layer fusion. Neural networks acquire knowledge through specific learning algorithms and perform fusion and re-learning according to the obtained uncertainty reasoning mechanism. This method has a large amount of calculation and is difficult to establish rules. Fuzzy theory is a multi-valued logic that specifies a real number from 0 to 1 to represent the degree of truth, which also requires a lot of calculations. DS evidence theory is based on prior knowledge to distinguish between unknown and uncertainty, but prior knowledge is not easy to obtain and has a small scope of application. Therefore, under the condition of poor quality of source information, it is crucial to study information fusion methods with low computational difficulty and cost.

[0004] In addition, most of the existing multi-source heterogeneous information fusion methods are synchronous fusion, that is, all collected information is used as the input of the model at the same time, which will inevitably have high requirements on computing power, and the timeliness of the calculation will also be affected. Especially in large-scale monitoring systems, this contradiction is quite prominent. Therefore, under certain computing power conditions, while not sacrificing fusion accuracy, improving computing efficiency is a problem that multi-source heterogeneous information fusion methods need to solve. Summary of the invention

[0005] The purpose of the present invention is to solve the problems in the prior art and propose a distributed multi-source heterogeneous information fusion method based on the Transformer architecture. The method is applicable to all information fusion application scenarios and is general.

[0006] The present invention is implemented by the following technical scheme. The present invention proposes a distributed multi-source heterogeneous information fusion method based on Transformer architecture. The specific steps of the method are:

[0007] Step 1: Build a source information type attribute recognition model to determine the type attribute of the source information and classify and arrange the source information according to type and collection time;

[0008] Step 2: Construct a data layer feature extraction combination model to extract the intermediate feature vectors of the source information according to the type attributes output in step 1;

[0009] Step 3: Perform preprocessing including outlier processing, normalization and denoising on the intermediate feature vector outputted from step 2;

[0010] Step 4: Build a Transformer architecture that integrates the attention mechanism, extract features from the output of step 2 again and embed them into a unified feature space to achieve unified expression of feature vectors.

[0011] Furthermore, the step 1 is specifically as follows:

[0012] Step 1, extracting the transport layer and application layer contents of the data packet as key information for identification;

[0013] Step 1 and 2, extracting statistical features of high-frequency continuous substring sets;

[0014] Step 13, extracting statistical features of a set of consecutive identical character strings;

[0015] Step 14: Use the classification CART decision tree to calculate the Gini gain of the statistical feature according to the feature vector, select the minimum value as the partition feature, find the classification category with the minimum Gini index from the partition feature as the optimal partition point, and output the type attribute of the key information.

[0016] Furthermore, in steps one and two, the extracted information is segmented using a sliding window of length n: assuming n = {1, 2, ..., m}, where m is a positive integer, the information is segmented into m substring sets, the frequencies corresponding to repeated elements in the substring sets are counted, and different frequency k values ​​are taken to obtain different high-frequency continuous substring sets; in order to accurately distinguish different types of information content, the information entropy between different substrings is extracted as the characteristic value of the key information.

[0017] Furthermore, in step 1-3, the frequency of consecutive identical character substrings, the number of consecutive identical character substrings, the maximum length and the average length of consecutive identical character substrings are calculated respectively; together with the information entropy between different substrings, they constitute the characteristic vector of the key information.

[0018] Furthermore, in step 2, wavelet transform is used to extract statistical features of time series information. Assuming that the time series information in a certain sampling period is x(t), when a wavelet basis function After displacement, the inner product of α and x(t) is taken at different scales, and its wavelet transform is:

[0019]

[0020]

[0021] Where X(w) and They are x(t) and Fourier transform of

[0022] Assume that the original signal is decomposed into n layers, n is a positive integer, and the high-frequency parts of the 1st to nth layers are extracted; when a mutation point is found in the original information, different wavelet basis functions are used to project the decomposed time series onto different scales, and m discontinuity points are found from them, m is a positive integer; k trend items, k is a positive integer and l random items, l is a positive integer, which constitute the characteristic vector of the time series;

[0023] The CNN-GoogLeNet model is used to extract features from image information. The CNN-GoogLeNet model consists of an input layer, a feature extraction layer, and a fully connected layer. The feature extraction layer includes a convolutional layer and a pooling layer. Local features are extracted in the convolutional layer. The formula is as follows:

[0024]

[0025] Where l represents the number of layers, represents the convolution kernel, represents the receptive field of the input layer, Represents a bias for each output image; the convolution kernel is multiplied by the local receptive field, and then the sum is added to the bias. The initial value of is 0; after the input feature map is pooled, its number remains unchanged, and the side length becomes 1 / n of the original, where n is the size of the pooling; the calculation formula of the pooling layer is as follows:

[0026]

[0027] in represents the pooling layer, represents the weight coefficient, Represents bias; sum all values ​​in the pooling domain and take the average value as the feature value of the downsampled feature map;

[0028] For text information, mutual information is used for feature extraction; after the text information is segmented and duplicated, a bag of words is established as a candidate feature vector for feature selection; mutual information is used as the feature selection evaluation function, and the formula is as follows:

[0029] H(X,Y)=H(X)+H(Y / X)=H(Y)+H(X / Y) (5)

[0030] H(X)-H(X / Y)=H(Y)-H(Y / X) (6)

[0031] The mutual information of the difference random variables X and Y in formula (6) is as follows:

[0032] I(X,Y)=H(X)-H(X / Y)=H(Y)-H(Y / X) (7)

[0033] Where I(X,Y)) represents the amount of information provided by knowing one of the variables X or Y to eliminate the uncertainty of the other variable Y or X;

[0034] Mutual information is used to select features from word bags as feature selection for text information. The formula is as follows:

[0035]

[0036] Where P(C,t) represents the probability that the text containing the feature word t in the text set belongs to class C, P(C) represents the probability of the text of class C appearing in the text set, and P(t) represents the probability of the text containing the feature word t appearing. It can be seen from formula (8) that if t is irrelevant to class C, the mutual information value is 0.

[0037] The additive fusion algorithm is used to realize model combination for the time series information, image information, and text information feature extraction models, so as to achieve the synchronous extraction of data layer features.

[0038] Furthermore, the specific way to achieve model combination is as follows: the additive fusion algorithm is a nonlinear tree model. The input information is matched with model 1, model 2, and model 3 according to the different type attribute values ​​of the source information. The matching model extracts the features and forms a feature vector output at the output end. The model matching formula is as follows:

[0039]

[0040] Where P i is the matching probability of the i-th model, |Model i| is the amount of information matched by model i, |U| is the total amount of all source information;

[0041] The attribute distance is calculated using the neighborhood distance method. The calculation formula is as follows:

[0042] D ij =|attr ij -attr kj | (10)

[0043] Assume u k To match source information, U i (1≤i≤n) is the i-th source information, attr kj (1≤j≤m) is U i The value of attribute j, attr ij (1≤j≤m) is u k The attribute value of D ij It represents the attribute distance between source information k and i on attribute j. The greater the difference between the source information and the model attribute value, the greater the attribute distance.

[0044] Furthermore, in step 3, the deviation standardization method is used for normalization processing, and the calculation formula is as follows:

[0045]

[0046] where x j is the normalized value, j is a positive integer, x i is the original value, i is a positive integer, x min is the minimum value in the original array, x max is the maximum value in the original array;

[0047] The standard deviation method is used to process outliers and remove values ​​greater than the set threshold. The calculation formula is as follows:

[0048]

[0049] Where σ represents the population standard deviation, x i represents the i-th value, i is a positive integer, represents the array mean, n represents the length of the array, and if a data point is greater than 3σ from the mean, it will be considered an outlier and removed;

[0050] The moving mean filtering algorithm is used to remove noise. When a sequence x[n] includes N values, n is the index of the sequence, a positive integer, a window of a certain length M is slid on the sequence, and the average value of the window value is calculated for filtering. The calculation formula is as follows:

[0051]

[0052] Where y[k] is the filtered output, x[k] represents the value with index k in the sequence, and 1 / M is the normalization factor.

[0053] Furthermore, in step 4, the key information (A, B, C) is first extracted from the original data, and then a feature sequence (a1, a2, ..., a n ,b1,b2,…,b m ,c1,c2,…,c k ), the attention Head calculation method is used to find the temporal correlation of A, B, and C, and the weights are assigned accordingly. The weighted fusion is performed based on the Transformer module. Through repeated cross-extraction, the feature data is averaged pooled and fully connected layer aggregated. Finally, various multi-source heterogeneous information is represented as a unified feature vector, recorded as (x1, x2, …, x n ).

[0054] Compared with the prior art, the present invention has the following advantages:

[0055] The present invention provides a new solution to the problem of multi-source heterogeneous information fusion. Compared with the existing inventions, the innovations are as follows: a multi-level distributed feature fusion framework is proposed, based on the primary features of heterogeneous source information, an end-to-end self-attention mechanism encoder architecture is used to realize secondary feature extraction and fusion, which reduces the computational difficulty and achieves a balance between computational cost and time efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flow chart of the distributed multi-source heterogeneous information fusion method based on the Transformer architecture of the present invention;

[0057] Figure 2 Schematic diagram of wavelet decomposition of the present invention;

[0058] Figure 3 It is a schematic diagram of the pooling operation of the present invention;

[0059] Figure 4 is a schematic diagram of the additive fusion algorithm of the present invention;

[0060] Figure 5 This is a schematic diagram of distributed multi-source heterogeneous information fusion based on the Transformer architecture of the present invention. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] See also Figure 1-Figure 5 , the present invention proposes a distributed multi-source heterogeneous information fusion method based on Transformer architecture, Figure 1 The flowchart of the distributed source heterogeneous information fusion method based on the Transformer architecture provided by the present invention is shown, and the specific steps of the method are:

[0063] Step 1: Build a source information type attribute recognition model to determine the type attribute of the source information, such as numerical sequence, picture, text, etc., and classify and arrange the source information according to type and collection time;

[0064] The step 1 is specifically as follows:

[0065] Step 1, extracting the transport layer and application layer contents of the data packet as key information for identification;

[0066] Step 1 and 2, extract the statistical features of the high-frequency continuous substring set; use a sliding window of length n to segment the extracted information: assuming n = {1, 2, ..., m}, where m is a positive integer, segment the information into m substring sets, count the frequencies corresponding to repeated elements in the substring sets, and take different frequency k values ​​to obtain different high-frequency continuous substring sets; in order to accurately distinguish different types of information content, extract the information entropy between different substrings as the characteristic value of key information;

[0067] Step 13, extracting statistical features of the set of consecutive identical character strings; calculating the frequency of consecutive identical character substrings, the number of consecutive identical character substrings, the maximum length and average length of consecutive identical character substrings; and forming a feature vector of key information together with the information entropy between different substrings;

[0068] Step 14: Use the classification CART decision tree to calculate the Gini gain of the statistical feature according to the feature vector, select the minimum value as the partition feature, find the classification category with the minimum Gini index from the partition feature as the optimal partition point, and output the type attribute of the key information.

[0069] Step 2: Construct a data layer feature extraction combination model to extract the intermediate feature vectors of the source information according to the type attributes output in step 1;

[0070] In step 2, wavelet transform is used to extract statistical features of time series information. Assuming that the time series information in a certain sampling period is x(t), when a wavelet basis function After displacement, the inner product of α and x(t) is taken at different scales, and its wavelet transform is:

[0071]

[0072]

[0073] Where X(w) and They are x(t) and The Fourier transform of Figure 2 As shown, S represents the original sequence, A1, A2, A3 represent the high-frequency parts of the 1st to 3rd layers, D1, D2, F3 represent the low-frequency parts of the 1st to 3rd layers, and the wavelet decomposition method continuously decomposes the time series into high-frequency and low-frequency parts.

[0074] Assume that the original signal is decomposed into n layers, n is a positive integer, and the high-frequency parts of the 1st to nth layers are extracted; when a mutation point is found in the original information, different wavelet basis functions are used to project the decomposed time series onto different scales, and m discontinuity points are found from them, m is a positive integer; k trend items, k is a positive integer and l random items, l is a positive integer, which constitute the characteristic vector of the time series;

[0075] The CNN-GoogLeNet model is used to extract features from image information. The CNN-GoogLeNet model consists of an input layer, a feature extraction layer, and a fully connected layer. The feature extraction layer includes a convolution layer and a pooling layer. In the convolution layer, each neuron of the feature map obtained by the input layer is convolved with the local receptive field of the previous layer and convolved with a learnable kernel to extract local features. The formula is as follows:

[0076]

[0077] Where l represents the number of layers, represents the convolution kernel, represents the receptive field of the input layer, Represents a bias for each output image; the previous layer of the convolution layer can be the initial image, or it can be a feature map generated by the convolution layer or the downsampling layer. The convolution kernel is multiplied by the local receptive field, and then the sum is added to the bias. The parameters inside the convolution kernel are trainable, and the bias The initial value of is 0; after the input feature map is pooled, its number remains unchanged, and the side length becomes 1 / n of the original, where n is the size of the pooling; the calculation formula of the pooling layer is as follows:

[0078]

[0079] in represents the pooling layer, represents the weight coefficient, Represents the bias; sum all the values ​​in the pooling domain and take the average value as the feature value of the downsampled feature map, such as Figure 3 As shown;

[0080] For text information, mutual information is used for feature extraction; after the text information is segmented and duplicated, a bag of words is established as a candidate feature vector for feature selection; mutual information is used as the feature selection evaluation function, and the formula is as follows:

[0081] H(X,Y)=H(X)+H(Y / X)=H(Y)+H(X / Y) (5)

[0082] H(X)-H(X / Y)=H(Y)-H(Y / X) (6)

[0083] The mutual information of the difference random variables X and Y in formula (6) is as follows:

[0084] I(X,Y)=H(X)-H(X / Y)=H(Y)-H(Y / X) (7)

[0085] Where I(X,Y)) represents the amount of information provided by knowing one of the variables X or Y to eliminate the uncertainty of the other variable Y or X;

[0086] Mutual information is used to select features from word bags as feature selection for text information. The formula is as follows:

[0087]

[0088] Where P(C,t) represents the probability that the text containing the feature word t in the text set belongs to class C, P(C) represents the probability of the text of class C appearing in the text set, and P(t) represents the probability of the text containing the feature word t appearing. It can be seen from formula (8) that if t is irrelevant to class C, the mutual information value is 0.

[0089] The additive fusion algorithm is used to realize model combination for the time series information, image information, and text information feature extraction models, so as to achieve the synchronous extraction of data layer features.

[0090] like Figure 4 As shown in the figure, the specific way to realize model combination is as follows: the additive fusion algorithm is a nonlinear tree model. The input information is matched with model 1, model 2, and model 3 according to the different type attribute values ​​of the source information. The matching model extracts the features and forms a feature vector output at the output end. The model matching formula is as follows:

[0091]

[0092] Where P i is the matching probability of the i-th model, |Model i | is the amount of information matched by model i, |U| is the total amount of all source information;

[0093] The attribute distance is calculated using the neighborhood distance method. The calculation formula is as follows:

[0094] D ij =|attr ij -attr kj | (10)

[0095] Assume u k To match source information, U i (1≤i≤n) is the i-th source information, attr kj (1≤j≤m) is U i The value of attribute j, attr ij (1≤j≤m) is u k The attribute value of D ij It represents the attribute distance between source information k and i on attribute j. The greater the difference between the source information and the model attribute value, the greater the attribute distance.

[0096] Step 3: Perform preprocessing including outlier processing, normalization and denoising on the intermediate feature vector outputted from step 2;

[0097] In step 3, the deviation standardization method is used for normalization processing, and the calculation formula is as follows:

[0098]

[0099] where x j is the normalized value, j is a positive integer, x i is the original value, i is a positive integer, x min is the minimum value in the original array, x max is the maximum value in the original array;

[0100] The standard deviation method is used to process outliers and remove values ​​greater than the set threshold. The calculation formula is as follows:

[0101]

[0102] Where σ represents the population standard deviation, x i represents the i-th value, i is a positive integer, represents the array mean, n represents the length of the array, and if a data point is greater than 3σ from the mean, it will be considered an outlier and removed;

[0103] The moving mean filtering algorithm is used to remove noise. When a sequence x[n] includes N values, n is the index of the sequence, a positive integer, a window of a certain length M is slid on the sequence, and the average value of the window value is calculated for filtering. The calculation formula is as follows:

[0104]

[0105] Where y[k] is the filtered output, x[k] represents the value with index k in the sequence, and 1 / M is the normalization factor.

[0106] Step 4: Build a Transformer architecture that integrates the attention mechanism, extract features from the output of step 2 again and embed them into a unified feature space to achieve unified expression of feature vectors.

[0107] In step 4, we first extract key information (A, B, C) from the original data, and then use statistical methods to extract a feature sequence (a1, a2, ..., a n ,b1,b2,…,b m ,c1,c2,…,c k ), the attention Head calculation method is used to find the temporal correlation of A, B, and C, and the weights are assigned accordingly. The weighted fusion is performed based on the Transformer module. Through repeated cross-extraction, the feature data is averaged pooled and fully connected layer aggregated. Finally, various multi-source heterogeneous information is represented as a unified feature vector, recorded as (x1, x2, …, x n ).

[0108] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A distributed multi-source heterogeneous information fusion method based on Transformer architecture, characterized by: The specific steps of the method are: Step 1: Build a source information type attribute recognition model to determine the type attribute of the source information and classify and arrange the source information according to type and collection time; Step 2: Construct a data layer feature extraction combination model to extract the intermediate feature vectors of the source information according to the type attributes output in step 1; Step 3: Perform preprocessing including outlier processing, normalization and denoising on the intermediate feature vector outputted from step 2; Step 4: Build a Transformer architecture that integrates the attention mechanism, extract features from the output of step 2 again and embed them into a unified feature space to achieve unified expression of feature vectors.

2. The method according to claim 1, characterized in that: The step 1 is specifically as follows: Step 1, extracting the transport layer and application layer contents of the data packet as key information for identification; Step 1 and 2, extracting statistical features of high-frequency continuous substring sets; Step 13, extracting statistical features of a set of consecutive identical character strings; Step 14: Use the classification CART decision tree to calculate the Gini gain of the statistical feature according to the feature vector, select the minimum value as the partition feature, find the classification category with the minimum Gini index from the partition feature as the optimal partition point, and output the type attribute of the key information.

3. The method according to claim 2, characterized in that: In steps one and two, the extracted information is segmented using a sliding window of length n: assuming n = {1, 2, ..., m}, where m is a positive integer, the information is segmented into m substring sets, the frequencies corresponding to repeated elements in the substring sets are counted, and different frequency k values ​​are taken to obtain different high-frequency continuous substring sets; in order to accurately distinguish different types of information content, the information entropy between different substrings is extracted as the characteristic value of the key information.

4. The method according to claim 2, characterized in that: In step 1-3, the frequency of consecutive identical character substrings, the number of consecutive identical character substrings, the maximum length and the average length of consecutive identical character substrings are calculated respectively; together with the information entropy between different substrings, they constitute the characteristic vector of the key information.

5. The method according to claim 2, characterized in that: In step 2, wavelet transform is used to extract statistical features of time series information. Assuming that the time series information in a certain sampling period is x(t), when a wavelet basis function After displacement, the inner product of α and x(t) is taken at different scales, and its wavelet transform is: Where X(w) and They are x(t) and Fourier transform of Assume that the original signal is decomposed into n layers, n is a positive integer, and the high-frequency parts of the 1st to nth layers are extracted; when a mutation point is found in the original information, different wavelet basis functions are used to project the decomposed time series onto different scales, and m discontinuity points are found from them, m is a positive integer; k trend items, k is a positive integer and l random items, l is a positive integer, which constitute the characteristic vector of the time series; The CNN-GoogLeNet model is used to extract features from image information. The CNN-GoogLeNet model consists of an input layer, a feature extraction layer, and a fully connected layer. The feature extraction layer includes a convolutional layer and a pooling layer. Local features are extracted in the convolutional layer. The formula is as follows: Where l represents the number of layers, represents the convolution kernel, represents the receptive field of the input layer, Represents a bias for each output image; the convolution kernel is multiplied by the local receptive field, and then the sum is added to the bias. The initial value of is 0; after the input feature map is pooled, its number remains unchanged, and the side length becomes 1 / n of the original, where n is the size of the pooling; the calculation formula of the pooling layer is as follows: Where down(x j l-1 ) represents the pooling layer, represents the weight coefficient, Represents bias; sum all values ​​in the pooling domain and take the average value as the feature value of the downsampled feature map; For text information, mutual information is used for feature extraction; after the text information is segmented and duplicated, a bag of words is established as a candidate feature vector for feature selection; mutual information is used as the feature selection evaluation function, and the formula is as follows: H(X,Y)=H(X)+H(Y / X)=H(Y)+H(X / Y) (5) H(X)-H(X / Y)=H(Y)-H(Y / X) (6) The mutual information of the difference random variables X and Y in formula (6) is as follows: I(X,Y)=H(X)-H(X / Y)=H(Y)-H(Y / X) (7) Where I(X,Y)) represents the amount of information provided by knowing one of the variables X or Y to eliminate the uncertainty of the other variable Y or X; Mutual information is used to select features from word bags as feature selection for text information. The formula is as follows: Where P(C,t) represents the probability that the text containing the feature word t in the text set belongs to class C, P(C) represents the probability of the text of class C appearing in the text set, and P(t) represents the probability of the text containing the feature word t appearing. It can be seen from formula (8) that if t is irrelevant to class C, the mutual information value is 0. The additive fusion algorithm is used to realize model combination for the time series information, image information, and text information feature extraction models, so as to achieve the synchronous extraction of data layer features.

6. The method according to claim 5, characterized in that: The specific way to realize the model combination is as follows: the additive fusion algorithm is a nonlinear tree model. The input information is matched with model 1, model 2, and model 3 according to the different type attribute values ​​of the source information. The matching model extracts the features and forms a feature vector output at the output end. The model matching formula is as follows: Where P i is the matching probability of the i-th model, |Model i | is the amount of information matched by model i, |U| is the total amount of all source information; The attribute distance is calculated using the neighborhood distance method. The calculation formula is as follows: D ij =|attr ij -attr kj | (10) Assume u k To match source information, U i (1≤i≤n) is the i-th source information, attr kj (1≤j≤m) is U i The value of attribute j, attr ij (1≤j≤m) is u k The attribute value of D ij It represents the attribute distance between source information k and i on attribute j. The greater the difference between the source information and the model attribute value, the greater the attribute distance.

7. The method according to claim 2, characterized in that: In step 3, the deviation standardization method is used for normalization processing, and the calculation formula is as follows: where x j is the normalized value, j is a positive integer, x i is the original value, i is a positive integer, x min is the minimum value in the original array, x max is the maximum value in the original array; The standard deviation method is used to process outliers and remove values ​​greater than the set threshold. The calculation formula is as follows: Where σ represents the population standard deviation, x i represents the i-th value, i is a positive integer, represents the array mean, n represents the length of the array, and if a data point is greater than 3σ from the mean, it will be considered an outlier and removed; The moving mean filtering algorithm is used to remove noise. When a sequence x[n] includes N values, n is the index of the sequence, a positive integer, a window of a certain length M is slid on the sequence, and the average value of the window value is calculated for filtering. The calculation formula is as follows: Where y[k] is the filtered output, x[k] represents the value with index k in the sequence, and 1 / M is the normalization factor.

8. The method according to claim 2, characterized in that: In step 4, we first extract key information (A, B, C) from the original data, and then use statistical methods to extract a feature sequence (a1, a2, ..., a n ,b1,b2,…,b m ,c1,c2,…,c k ), the attention Head calculation method is used to find the temporal correlation of A, B, and C, and the weights are assigned accordingly. The weighted fusion is performed based on the Transformer module. Through repeated cross-extraction, the feature data is averaged pooled and fully connected layer aggregated. Finally, various multi-source heterogeneous information is represented as a unified feature vector, recorded as (x1, x2, …, x n ).

Images