Transform-based data fusion method and system

Abstract

The invention discloses a Transform-based data fusion method and system, and relates to the technical field of data fusion, and the method comprises the steps: collecting multi-modal data, carrying out the data preprocessing of the multi-modal data, and obtaining the preprocessed data; the preprocessed data comprises independent statements and structured features; carrying out vectorization processing on the independent statements by adopting an Embedding technology to obtain a text vector matrix; extracting text features in the text vector matrix based on an LST M recurrent neural network; constructing a training data set according to text features in the extracted text vector matrix and a plurality of structured features in the preprocessed data; and according to the training data set, based on a Transform technology, training the fusion neural network model to obtain a trained fusion neural network model. The invention provides a new thought and method for processing and analyzing the multi-modal data.

Description

Technical Field

[0001] The present application relates to the technical field of data fusion, and in particular to a Transformer-based data fusion method and system. Background Art

[0002] With the rapid development of information technology and the diversification of data acquisition methods, deep learning technology has been widely used in the field of data analysis. Traditional data analysis methods mainly rely on the processing of structured data (such as numerical data, tabular data, etc.), which often show certain limitations when facing data complexity and diversity. Especially when processing unstructured data such as text, traditional methods are difficult to fully explore the rich information contained therein. In order to solve these problems, new data fusion methods and deep learning models have emerged. In multimodal data analysis, how to effectively fuse information from different data sources (such as text data and numerical data) has become an important research topic. Summary of the invention

[0003] The purpose of this application is to provide a Transformer-based data fusion method and system, which can use Embedding technology to map different types of data into a unified representation space, use LSTM to capture the sequence dependencies of the data, extract the global features of the data through Transformer, and integrate data from different modalities through a fusion mechanism.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a Transformer-based data fusion method, comprising:

[0006] Collecting multimodal data; the multimodal data includes structured data and unstructured text data;

[0007] Performing data preprocessing on the multimodal data to obtain preprocessed data; the preprocessed data includes a plurality of independent sentences and a plurality of structured features;

[0008] Using Embedding technology, we vectorize the independent sentences in the preprocessed data to obtain a text vector matrix;

[0009] Extracting text features from the text vector matrix based on an LSTM recurrent neural network;

[0010] Constructing a training data set according to the text features extracted from the text vector matrix and a plurality of structural features in the preprocessed data;

[0011] According to the training data set, based on the Transformer technology, the fusion neural network model is trained to obtain a trained fusion neural network model.

[0012] In a second aspect, the present application provides a Transformer-based data fusion system, comprising:

[0013] A data collection module, used to collect multimodal data; the multimodal data includes structured data and unstructured text data;

[0014] A preprocessing module, used for performing data preprocessing on the multimodal data to obtain preprocessed data; the preprocessed data includes a plurality of independent sentences and a plurality of structured features;

[0015] The processing module is used to use the Embedding technology to vectorize the independent sentences in the preprocessed data to obtain a text vector matrix;

[0016] An extraction module, used for extracting text features in the text vector matrix based on an LSTM recurrent neural network;

[0017] A construction module, used to construct a training data set according to the text features extracted from the text vector matrix and a plurality of structural features in the preprocessed data;

[0018] The training module is used to train the fusion neural network model according to the training data set based on the Transformer technology to obtain a trained fusion neural network model.

[0019] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0020] The present application provides a data fusion method and system based on Transformer. After collecting multimodal data, it is necessary to perform data preprocessing steps on it to generate a preprocessed data set. The data set contains independent sentences and structured features. Subsequently, the Embedding technology is applied to convert the independent sentences into vector form to form a text vector matrix. The text features are extracted from the text vector matrix using a long short-term memory (LSTM) recurrent neural network. Based on the extracted text features and several structured features in the preprocessed data, a composite data set for training is constructed. By using Transformer technology, the fusion neural network model is trained to finally obtain a trained fusion neural network model. The present application provides innovative ideas and methods for the field of multimodal data processing and analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A flowchart of a Transformer-based data fusion method provided in one embodiment of the present application;

[0023] Figure 2 A structural diagram of an LSTM repeating unit provided in an embodiment of the present application.

[0024] Figure 3 A flowchart of an Embedding-LSTM-Transformer fusion technology provided in one embodiment of the present application;

[0025] Figure 4 A schematic diagram of functional modules of a Transformer-based data fusion system provided in one embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0028] Embodiment 1

[0029] like Figure 1 As shown, this embodiment provides a Transformer-based data fusion method, including:

[0030] Step 101: Collect multimodal data; the multimodal data includes structured data and unstructured text data;

[0031] Step 102: performing data preprocessing on the multimodal data to obtain preprocessed data; the preprocessed data includes a plurality of independent sentences and a plurality of structured features;

[0032] Step 103: Using the Embedding technology, vectorize the independent sentences in the preprocessed data to obtain a text vector matrix;

[0033] Step 104: extracting text features in the text vector matrix based on the LSTM recurrent neural network;

[0034] Step 105: constructing a training data set according to the text features extracted from the text vector matrix and a plurality of structural features in the preprocessed data;

[0035] Step 106: According to the training data set, based on the Transformer technology, the fusion neural network model is trained to obtain a trained fusion neural network model.

[0036] In some embodiments, when executing step 101, the specific steps may be as follows:

[0037] Collect multimodal data, including structured data (such as tabular data) and unstructured text data (such as documents, reports, etc.), as the raw data for the experiment.

[0038] Specifically, the acquired multimodal data is mainly used to predict the target state at a certain point in the future based on the multimodal data during the data use process. When making predictions, the labels are divided into two categories: Category 1 indicates the presence of a specific risk or abnormal state, which is marked as 1; Category 0 indicates the absence of a risk or abnormal state, which is marked as 0.

[0039] When executing step 102, the specific steps may be as follows:

[0040] like Figure 3 As shown, the multimodal data is cleaned and screened to obtain multimodal data after removing data samples with special properties.

[0041] The random forest algorithm is used to evaluate each feature of the structured data in the multimodality, and features with evaluation scores greater than a set threshold score are screened as structured features.

[0042] A natural language processing tool is used to extract text content from the unstructured text data of the structured data in the multimodality.

[0043] Based on the sentence symbols, the extracted text content is divided into sentences to obtain several independent sentences.

[0044] Specifically, when preprocessing data, for structured data, the random forest algorithm is used to evaluate the importance of data by combining labels with feature data. The specific steps include: merging the target label with the relevant data, and dividing them into training set and test set in a ratio of 7:3, and using the cross-validation method to fit the model. The importance score of each feature for model prediction is obtained through the random forest model attribute feature_imp ortances_ in the Scikit-learn library, and the feature with a higher score J is selected as the most influential feature.

[0045] For text data, a specific extraction method is used to obtain relevant content from designated parts (such as chapter titles such as "Discussion and Analysis"), and the required text data is extracted through information such as text identifiers, subtitles, and page numbers.

[0046] When executing step 103, the specific steps may be as follows:

[0047] Perform word segmentation on each independent sentence in the preprocessed data to obtain a word sequence;

[0048] For each word in the word sequence, using the Embedding technology, each word is mapped to a high-dimensional vector space to obtain a word vector;

[0049] Map all word vectors to a vector of a set dimension to generate a text vector matrix.

[0050] Specifically, Figure 3 As shown, this embodiment performs sentence embedding processing on text data based on the Embedding model. The specific steps are as follows:

[0051] 3.1) Word segmentation: In step 102, after the text information is split into separate sentences, each sentence is segmented. For example, "system optimization performance improvement" is split into "system", "optimization", "performance", and "improvement". By converting a continuous text sequence into a discrete word sequence, it provides a basis for subsequent word embedding.

[0052] 3.2) Word Embedding: For each word after segmentation in 3.1, use word embedding technology to deeply mine its semantic and grammatical information, map each word into a high-dimensional vector space, and convert it into a high-dimensional word vector for subsequent model training.

[0053] 3.3) Sequence encoding: Encode the result of word embedding, and map the word vectors of all input sequences into a vector of fixed dimension. In this embodiment, the fixed dimension is selected as 1024. In this embodiment, the Transformer model is used for deep sequence encoding in the embedding pre-training model. The Transformer model adopts the Multi-HeadAttention mechanism and Feedforward Neural Network, which can globally encode and interpret the input ordered sequence. Finally, through this method, the encoding structure of the management discussion and analysis part is: n*1024 document vector matrix, V={v1, v2,..., vn}, and v1 is represented as the vector of each sentence listed in v1={dim1, dim2,..., dim1024}.

[0054] When executing step 104, the specific steps may be as follows:

[0055] Through step 103, each sentence in the document is embedded to generate a vector matrix about the text. The text vector matrix is ​​feature extracted based on the LSTM (Long Short-Term Memory) recurrent neural network, which is a special recurrent neural network that overcomes the gradient vanishing and gradient exploding problems of traditional RNN when processing long sequence data by introducing a gate mechanism. LSTM is particularly suitable for processing and predicting time-based sequence data, such as text, time series prediction, speech recognition, etc.

[0056] Among them, Figure 2 The LSTM recurrent neural network model proposed by Hochreiter is shown, and the LSTM recurrent neural network includes a forget gate, an input gate, and an output gate.

[0057] Forget Gate: Based on the input of the current time step and the hidden state of the previous time step, the output is a value between 0 and 1, which is used to determine which information needs to be removed from the unit state. The specific formula is:

[0058] f t =σ(W f ·[h t-1 ,x t ]+b f ).

[0059] In the formula, f t : The output of the forget gate, ranging from 0 to 1, is used to decide which information to retain (close to 1) or forget (close to 0).

[0060] h t-1 : The hidden state of the previous time step. xt : Input for the current time step.

[0061] W f and b f : Weight and bias of the forget gate.

[0062] σ: Sigmoid activation function.

[0063] Input Gate: The input gate determines what new information will be written into the cell state. It includes two steps: calculating the activation value i of the input gate t and candidate cell states The formula is as follows:

[0064] i t =σ(W i ·[h t-1 ,x t ]+b i ).

[0065]

[0066] Among them, i t : The activation value of the input gate, which is used to determine what new information will be written into the cell state. New candidate cell states, used to generate new cell states. tanh: Hyperbolic tangent function, used to scale candidate states to between (-1,1)(-1,1)(-1,1), W i is the input gate weight matrix, used for linear transformation, with a shape of (h, h+x), b i is the bias vector, with shape (h, 1); W c is the candidate cell state weight matrix, with a shape of (h, h+x), used to convert [h t-1 , x t ) is linearly combined, b c is a bias vector with shape (h, 1). Then, the cell state C t Updated by the following formula:

[0067]

[0068] In the formula, C t : The cell state at the current time step. C t-1 : The cell state of the previous time step. Output Gate: The output gate determines the hidden state of the output and outputs according to the updated cell state. The specific formula is:

[0069] o t =σ(W o ·[h t-1 ,x t]+b o ).

[0070] h t =o t *tanh(C t ).

[0071] In the formula, o t is the activation value of the output gate; h t is the hidden state of the current time step; W o is the output gate weight matrix, with a shape of (h, h+x), used for linear transformation, b o is a bias vector with shape (h, 1).

[0072] Among them, based on the text information sentence vector matrix generated in step 103, the LSTM recurrent neural network is used to extract important feature information in the text, and the extracted 1024-dimensional features are reduced in dimension. The specific operations are as follows:

[0073] 1) In the process of extracting important features of the text information sentence vector matrix, the data set v = {v1, v2, v3.....vi} is first grouped and converted into a 3D array format before entering the LSTM: (number of samples, max_rows, number of features (lstm_units)). Among them, the number of samples is the total number of training samples, the average value of max_rows determined through multiple experiments is 255, and the number of features (dimension) remains unchanged at 1024.

[0074] Perform label processing on the target column and convert the original label into a binary classification label, for example, convert category A to 1 and category B to 0. The converted label data forms a 2D array with a shape of (number of samples, number of labels) for easy use in supervised learning models.

[0075] 2) After the data processing is completed, the training data is fitted and trained. The LSTM model training process mainly includes the following contents:

[0076] The main input parameters of the LSTM model include: input_shape (including parameters batch_size=32, max_rows, lstm_units=X.shape[-1] represents the size of the last dimension of tensor X (the tensor representing the initial data)), lstm_units=128, dense_units=128, dropout_rate=0.5, l2_reg=0.001, learning_rate=0.0001, output_dim=1.

[0077] Input layer (InputLayer): defines the input shape of the model. The input input_shape is three-dimensional, with a shape of (batch_size, max_rows, lstm_units), i.e. the number of time steps and the number of features. The output has the same shape as the input, where batch_size = 32, max_rows = 255, lstm_units = 1024.

[0078] LSTM Layer: Processes time series data and extracts the relationship between time steps.

[0079] The LSTM unit processes the data of each time step through the internal memory unit and gating mechanism (input gate, forget gate, output gate). Input: 3D tensor from the input layer: (batch_size, max_rows, lstm_units). By setting return_sequences = True, LSTM outputs the state of each time step and generates a 3D tensor (batch_size, max_rows, lstm_units).

[0080] Batch Normalization Layer: Normalizes the output tensors (batch_size, max_rows, lstm_units) from the LSTM layer and outputs normalized tensors of the same shape to stabilize the training process and improve the generalization ability of the model.

[0081] LeakyReLU activation function (LeakyReLUActivationFunction): Apply the LeakyReLU activation function to the elements of the BatchNormalization layer, and output the same shape, but after the tensor is processed by the LeakyReLU activation function, the model can handle more complex patterns by introducing nonlinearity. The negative part will be multiplied by a small slope (alpha = 0.1) to prevent neurons from "dying".

[0082] Dropout layer (DropoutLayer): Dropout is applied to the tensor processed by LeakyReLU. During training, each neuron is temporarily removed with a certain probability (dropout_rate=0.5), thereby preventing the model from over-relying on certain specific neurons and enhancing the generalization ability of the model. The output is a tensor that has been lost, but the shape of the tensor remains unchanged.

[0083] FlattenLayer: Flattens the three-dimensional tensor (batch_size, max_rows, lst m_units) from the dropout layer into one dimension for use in the fully connected layer, and outputs a one-dimensional tensor (batch_size, max_rows*lstm_units).

[0084] Fully connected layer (DenseLayer): Perform linear transformation on the flattened vector (batch_size, max_rows*lstm_units) to generate a new feature representation, that is, a one-dimensional tensor (batch_size, dense_units).

[0085] Output Layer: The tensor data generated by the fully connected layer is processed through the dropout layer to generate a one-dimensional tensor (batch_size, output_dim), where output_dim = 128 is the final output dimension of the model.

[0086] In some embodiments, when executing step 105-step 106, the specific steps may be as follows:

[0087] Fusion technology is mainly used for the integration of multimodal data, by fusing data from different sources (such as structured data and unstructured text data) to improve the model performance in downstream tasks (such as classification, prediction). Unlike hard fusion, fusion allows data to maintain a certain degree of independence during fusion, avoiding information loss or oversimplification, while being able to combine the advantages of multiple data types.

[0088] In multimodal data processing scenarios, this technology can better utilize the complementarity of different data sources. For example, structured data can provide precise numerical information, while text data can provide context or qualitative analysis. By fusing and combining this information, the prediction accuracy of the model can be improved.

[0089] In the specific operation, after the text information is feature extracted and reduced to 128 dimensions through the LSTM recurrent neural network, it is fused with the structured features extracted from the structured data in step 102. Finally, the fused data is input into the Transformer model for training to optimize the model performance.

[0090] Among them, the specific operations of the fusion neural network model are as follows:

[0091] 1) Linear layer mapping of features: First, the original structured data features and text features are mapped to the same hidden dimension through linear layers, set to hidden_dim = 128. Specifically, this step is to multiply the input features by a weight matrix and add a bias term, and then apply the ReLU activation function. Through this process, the structured features and text features are mapped to a unified dimension, ensuring that they can be effectively spliced ​​and processed in subsequent feature fusion.

[0092] 2) Transformer encoder layer: The mapped structured features and text features are further extracted through their respective Transformer encoder layers. In each Transformer encoder layer, the input features (d_model = hidden_dim) are first weighted and summed through a multi-head attention mechanism (nhead = 4) to extract important features. Then, these features are processed more deeply by a feedforward neural network (dim_feedforward = hidden_dim*4, that is, the hidden layer dimension of the feedforward neural network is set to 4 times the input feature dimension) to further refine the information. Finally, residual connections and LayerNorm operations are used to ensure the stability of the model and the training effect. After being processed by the Transformer encoder layer, higher-level and important features are extracted from the structured features and text features.

[0093] 3) Fusion layer: In the fusion layer, the structured features and text features extracted by the Transformer encoder are first concatenated in the feature dimension. To further enhance the expressiveness of the model, the concatenated features are passed through a fusion linear layer and the ReLU activation function is applied for nonlinear transformation. The fusion linear layer maps the concatenated features to a unified hidden dimension, thereby improving the model's ability to learn multimodal data.

[0094] 4) Output layer: Map the fused structured features and text features to the target category space and generate the final classification results. Through a linear layer, the fused features are converted into scores for each category. These scores are processed by the Softmax activation function to generate the probability distribution of the category. For binary classification problems, the output probability represents the possibility of positive and negative classes.

[0095] In some embodiments, during Transformer fusion and binary classification, the main training parameters are batch_size=32, num_epochs=20, and learning_rate=0.001.

[0096] In addition, this application also provides an application result of a Transformer-based data fusion method, as follows:

[0097] This algorithm is applied to the financial crisis early warning system for listed companies, which covers the financial data and annual report data of listed companies. Table 1 shows the fitting data results of this algorithm. After comparing the results, it is found that the performance of this algorithm in AUC, ACC, Recall value and other aspects is significantly better than the other two groups of algorithms.

[0098] Table 1 Comparison of algorithm results

[0099]

[0100] Embodiment 2

[0101] like Figure 4 As shown, this embodiment provides a Transformer-based data fusion system, including:

[0102] A data collection module 401 is used to collect multimodal data; the multimodal data includes structured data and unstructured text data;

[0103] A preprocessing module 402 is used to perform data preprocessing on the multimodal data to obtain preprocessed data; the preprocessed data includes a plurality of independent sentences and a plurality of structured features;

[0104] The processing module 403 is used to use the embedding technology to vectorize the independent sentences in the preprocessed data to obtain a text vector matrix;

[0105] An extraction module 404 is used to extract text features in the text vector matrix based on an LSTM recurrent neural network;

[0106] A construction module 405 is used to construct a training data set according to the text features extracted from the text vector matrix and a plurality of structural features in the preprocessed data;

[0107] The training module 406 is used to train the fusion neural network model according to the training data set based on the Transformer technology to obtain a trained fusion neural network model.

[0108] In summary, this application has the following technical effects:

[0109] This application proposes a fusion method that uses Embedding technology to map different types of data to a unified representation space, uses LSTM to capture the sequence dependencies of the data, extracts the global features of the data through Transformer, and integrates data from different modalities through a fusion mechanism. This application can fully mine important feature information in structured and unstructured data, and can also provide more accurate and reliable results in tasks such as classification prediction. Therefore, the fusion technology provided by this application provides new ideas and methods for the processing and analysis of multimodal data, and has broad application prospects.

[0110] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A Transformer-based data fusion method, characterized in that: The data fusion method based on Transformer includes: Collecting multimodal data; the multimodal data includes structured data and unstructured text data; Performing data preprocessing on the multimodal data to obtain preprocessed data; the preprocessed data includes a plurality of independent sentences and a plurality of structured features; Using Embedding technology, we vectorize the independent sentences in the preprocessed data to obtain a text vector matrix; Extracting text features from the text vector matrix based on an LSTM recurrent neural network; Constructing a training data set according to the text features extracted from the text vector matrix and a plurality of structural features in the preprocessed data; According to the training data set, based on the Transformer technology, the fusion neural network model is trained to obtain a trained fusion neural network model.

2. According to the Transformer-based data fusion method of claim 1, it is characterized in that: Performing data preprocessing on the multimodal data to obtain preprocessed data specifically includes: Clean and filter the multimodal data to obtain multimodal data after removing data samples with special properties; Using a random forest algorithm, each feature of the structured data in the multimodal data is evaluated, and features with evaluation scores greater than a set threshold score are selected as structured features; Using natural language processing tools to extract text content from unstructured text data of structured data in the multimodal data; Based on the sentence symbols, the extracted text content is divided into sentences to obtain several independent sentences.

3. The Transformer-based data fusion method according to claim 1, characterized in that: Using Embedding technology, we vectorize the independent sentences in the preprocessed data to obtain a text vector matrix, which includes: Perform word segmentation on each independent sentence in the preprocessed data to obtain a word sequence; For each word in the word sequence, using the Embedding technology, each word is mapped to a high-dimensional vector space to obtain a word vector; Map all word vectors to a vector of a set dimension to generate a text vector matrix.

4. The Transformer-based data fusion method according to claim 3, characterized in that: The set dimension is 1024 dimensions.

5. The Transformer-based data fusion method according to claim 1, characterized in that: The LSTM recurrent neural network includes a forget gate, an input gate and an output gate.

6. The Transformer-based data fusion method according to claim 5, characterized in that: The formula expression of the forget gate is: f t =σ(W f ·[h t-1 ,x t ]+b f ); Among them, f t is the output of the forget gate, ranging from 0 to 1; h t-1 is the hidden state of the previous time step; x t is the input of the current time step; W f and b f are the weight and bias of the forget gate respectively; σ is the Sigmoid activation function.

7. The Transformer-based data fusion method according to claim 6, characterized in that: The formula expression of the input gate is: i t =σ(W i ·[h t-1 ,x t ]+b i ); Among them, i t is the activation value of the input gate; is the new candidate cell state; tanh is the hyperbolic tangent function, W i is the input gate weight matrix, used for linear transformation, with a shape of (h, h+x), b i is the bias vector, with shape (h, 1); W c is the candidate cell state weight matrix, with a shape of (h, h+x), used to convert [h t-1 , x t ) is linearly combined, b c is a bias vector with shape (h, 1).

8. The Transformer-based data fusion method according to claim 5, characterized in that: The formula expression of the output gate is: the t =σ(W o ·[h t-1 ,x t ]+b o ); h t =o t *tanh(C t ); Among them, t is the activation value of the output gate; h t is the hidden state of the current time step; W o is the output gate weight matrix, with a shape of (h, h+x), used for linear transformation, b o is a bias vector with shape (h, 1).

9. The Transformer-based data fusion method according to claim 1, characterized in that: The fusion neural network model includes a Transformer encoder layer, a fusion layer and an output layer; The Transformer encoder layer is used to extract features of the mapped structural features and text features based on the Transformer technology; The fusion layer is used to splice the structural features and text features extracted by the Transformer encoder in the feature dimension to obtain fused structural features and text features; The output layer is used to map the fused structural features and text features to the target category space.

10. A Transformer-based data fusion system, characterized in that: include: A data collection module, used to collect multimodal data; the multimodal data includes structured data and unstructured text data; A preprocessing module, used for performing data preprocessing on the multimodal data to obtain preprocessed data; the preprocessed data includes a plurality of independent sentences and a plurality of structured features; The processing module is used to use the Embedding technology to vectorize the independent sentences in the preprocessed data to obtain a text vector matrix; An extraction module, used for extracting text features in the text vector matrix based on an LSTM recurrent neural network; A construction module, used to construct a training data set according to the text features extracted from the text vector matrix and a plurality of structural features in the preprocessed data; The training module is used to train the fusion neural network model according to the training data set based on the Transformer technology to obtain a trained fusion neural network model.

Images