Deep neural network feature extraction method suitable for long text

Abstract

The invention relates to a deep neural network feature extraction method suitable for a long text, and the method comprises the steps: obtaining original text data, extracting keywords from the original text data, and generating a word dictionary comprising the keywords; extracting feature information of the original text data, and performing normalization processing to generate a data feature tensor; a pre-training language model is used as a backbone network, and context semantic features of the original text data are extracted; and in combination with the data feature tensor and the context semantic features, constructing a deep neural network model to extract long text features. The method has the beneficial effects that text key information extraction is carried out by combining a sorting algorithm and text feature fusion, the long-distance dependency relationship is efficiently processed by utilizing an improved deep neural network model, and accurate extraction of long text key features is realized by comprehensively considering multiple dimensions of semantics and structures.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a deep neural network feature extraction method suitable for long texts. Background Technology

[0002] In today's digital age, text data is exploding in size. Natural Language Processing (NLP), as a key technology bridging human language and computer understanding, plays an indispensable role in numerous fields such as information retrieval, intelligent customer service, and machine translation. Its importance and wide-ranging applications are self-evident. Among the many tasks in NLP, processing long texts remains a highly challenging task. Due to the rich semantic information and complex structure contained in long texts, the accurate extraction of their key features plays a decisive role in improving the performance of tasks such as text classification, sentiment analysis, and text summarization.

[0003] However, existing natural language processing methods face significant challenges when dealing with long texts. Traditional long text feature extraction methods primarily focus on rule-based approaches or optimizations for specific tasks. These methods largely rely on extensive manual annotation or simple statistical methods, making them ill-suited to the diverse content and complex semantic structures of long texts. For example, traditional methods may extract information solely based on basic features such as word frequency and part-of-speech, neglecting the semantic connections between words and the dependencies within the context. Furthermore, as the richness and complexity of natural language expressions continue to increase, relying solely on static features for feature extraction proves insufficient. Especially when processing long texts containing complex semantics and long-distance dependencies, existing methods struggle to effectively capture and utilize deep semantic information, resulting in low-quality feature extraction and limiting their effectiveness and widespread adoption in practical applications. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a deep neural network feature extraction method suitable for long texts.

[0005] Firstly, a deep neural network feature extraction method suitable for long texts is provided, including:

[0006] S1. Obtain raw text data, extract keywords from the raw text data, and generate a word dictionary including the keywords;

[0007] S2. Extract the feature information of the original text data, perform normalization processing, and generate data feature tensors;

[0008] S3. Use a pre-trained language model as the backbone network to extract the contextual semantic features of the original text data.

[0009] S4. Combine data feature tensors with contextual semantic features to construct a deep neural network model to extract features from long texts.

[0010] Preferably, S1 includes:

[0011] S101. Obtain the raw text data, extract keywords from the raw text data, and calculate the importance score of each keyword;

[0012] S102. Sort all keywords according to the importance scores;

[0013] S103. Based on the sorting results, select the top N keywords and assign a unique code to each keyword to generate the word dictionary, where N is a preset positive integer.

[0014] Preferably, S2 includes:

[0015] S201. Collect feature information of the original text data, wherein the feature information includes at least one of the following: publication year, original text length, and number of keywords;

[0016] S202. Perform logarithmic transformation and linear normalization on each feature information to map it to the interval [0,1].

[0017] Preferably, S3 includes:

[0018] S301. Obtain the language model and its weights pre-trained on a large corpus;

[0019] S302. Remove the final task output layer of the pre-trained language model, use the word dictionary obtained in S1 to convert the original text into a dictionary vector, input it into the pre-trained model and obtain contextual semantic features.

[0020] Preferably, S4 includes:

[0021] S401. The data feature tensor is fused with the context semantic features to generate a fused feature;

[0022] S402. Input the fused features into a deep neural network model, perform nonlinear transformation and dimensionality reduction, and output the final long text feature representation.

[0023] As a preferred option, the fusion in S401 is a splicing operation.

[0024] As a preferred option, it also includes:

[0025] S5. Use a long text dataset to perform end-to-end supervised training on the deep neural network model built from S1 to S4 to optimize the model parameters.

[0026] In a second aspect, a deep neural network feature extraction system suitable for long texts is provided for performing any of the methods described in the first aspect, including:

[0027] The acquisition module is used to acquire raw text data, extract keywords from the raw text data, and generate a word dictionary including the keywords;

[0028] The first extraction module is used to extract the feature information of the original text data, perform normalization processing, and generate a data feature tensor.

[0029] The second extraction module is used to extract the contextual semantic features of the original text data by using a pre-trained language model as the backbone network.

[0030] The module is used to construct a deep neural network model to extract features from long texts using the word dictionary, data feature tensor, and backbone network.

[0031] Thirdly, a computer storage medium is provided, wherein a computer program is stored therein; when the computer program is run on a computer, the computer causes the computer to perform any of the methods described in the first aspect.

[0032] Fourthly, an electronic device is provided, comprising:

[0033] Memory, used to store computer programs;

[0034] A processor for executing the computer program to implement the method as described in any of the first aspects.

[0035] The beneficial effects of this invention are as follows: This invention proposes an innovative deep neural network feature extraction method suitable for long texts. By combining ranking algorithms with text feature fusion to extract key information from the text, and utilizing an improved deep neural network model to efficiently handle long-distance dependencies, it comprehensively considers semantic and structural dimensions to achieve accurate extraction of key features from long texts. This method not only significantly improves the efficiency and accuracy of long text processing tasks, but also lays a solid data foundation for subsequent natural language processing applications, demonstrating broad application prospects and research value in many fields that rely on long text analysis. Attached Figure Description

[0036] Figure 1 A flowchart of a deep neural network feature extraction method suitable for long texts provided by the present invention;

[0037] Figure 2 The flowchart for constructing a dictionary of text keywords provided by this invention;

[0038] Figure 3The flowchart for constructing data feature tensors provided by this invention;

[0039] Figure 4 The flowchart for constructing a pre-trained Fast-former model provided by this invention;

[0040] Figure 5 The flowchart provided by this invention illustrates the construction of a deep neural network model to extract features from long text.

[0041] Figure 6 This is a model architecture diagram provided for the present invention. Detailed Implementation

[0042] The present invention will be further described below with reference to embodiments. The description of the embodiments below is only for the purpose of helping to understand the present invention. It should be noted that those skilled in the art can make several modifications to the present invention without departing from the principle of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0043] Example 1:

[0044] To address the problems of existing technologies, Embodiment 1 of this application provides a deep neural network feature extraction method suitable for long texts. By constructing a keyword dictionary, structured data features, and integrating them with an efficient Fast-former backbone network, a deep and efficient extraction of semantic information from long texts is achieved. This method effectively solves the problems of low model processing efficiency and easy loss of key information caused by long text sequences, significantly improves the representation quality of text features, and provides better underlying technical support for downstream tasks such as text classification, information retrieval, and intelligent recommendation. It has significant practical application value and promising prospects for widespread application.

[0045] Specifically, such as Figure 1 and Figure 6 As shown, the method includes:

[0046] S1. Obtain the original text data, extract keywords from the original text data, and generate a word dictionary including the keywords.

[0047] like Figure 2 As shown, S1 includes:

[0048] S101. Obtain the original text data, extract keywords from the original text data, and calculate the importance score of each keyword.

[0049] S101 employs the Text-Rank algorithm. Specifically, the text data is first segmented, stop words are removed, and a word graph is constructed. In this word graph, each word is treated as a node. If two words co-occur within a preset window size, an edge is established between them. Subsequently, the score of each node is iteratively calculated, based on the scores of its neighboring nodes and the weights of the edges, until convergence. Finally, the converged score of each word is its Text-Rank score, which reflects the importance of that word in the entire text.

[0050] For example, suppose we have a text about "artificial intelligence" containing the words "deep learning," "model," "training," and "data." When the window size K=2, the sentence fragment "...deep learning model training..." will generate edges (deep learning, model) and (model, training). In the constructed graph, the "model" node may be connected to multiple nodes such as "deep learning," "training," and "data." Through iterative calculation, core terms like "model" and "deep learning" will ultimately receive higher Text-Rank scores, while stop words (which have been removed) or secondary words like "a" and "of" will receive lower scores.

[0051] S102. Sort all keywords according to the importance score.

[0052] Specifically, a list is created containing all the keywords obtained in S101 and their corresponding Text-Rank scores. An efficient sorting algorithm, such as quicksort, is then used to sort this list in descending order of Text-Rank scores. This sorted list places the most important keywords (i.e., those with the highest scores) at the front, facilitating subsequent filtering steps.

[0053] For example, suppose the three extracted keywords and their scores are: [('Artificial Intelligence', 0.025), ('Algorithm', 0.015), ('Neural Network', 0.032)]. After sorting in descending order, the list becomes: [('Neural Network', 0.032), ('Artificial Intelligence', 0.025), ('Algorithm', 0.015)]. Thus, the term "Neural Network" is placed at the very beginning.

[0054] S103. Based on the sorting results, select the top N keywords and assign a unique code to each keyword to generate the word dictionary, where N is a preset positive integer.

[0055] Specifically, starting from the top of the sorted list, the first N keywords are selected sequentially. The value of the parameter N can be determined according to the actual application scenario and model requirements; for example, it can be set to 50, 100, or 200 to achieve a balance between representational power and computational complexity. Then, each selected keyword is assigned a unique integer code, for example, sequentially numbered starting from 0 or 1. Finally, the mapping relationship between these keywords and their unique codes is stored in a data structure (such as a Python dictionary or hash table), forming the word dictionary. This word dictionary will serve as one of the key inputs for subsequent text processing by the model.

[0056] For example, assuming N=2, select the first two keywords "neural network" and "artificial intelligence" from the sorted list [('neural network', 0.032), ('artificial intelligence', 0.025), ('algorithm', 0.015), ...]. Assign them codes, for example, "neural network" -> 1, "artificial intelligence" -> 2. The resulting word dictionary would be: {'neural network': 1, 'artificial intelligence': 2}. This dictionary will be used in subsequent steps to convert the text into a keyword-based numerical representation.

[0057] S2. Extract the feature information of the original text data, perform normalization processing, and generate a data feature tensor.

[0058] like Figure 3 As shown, S2 includes:

[0059] S201. Statistically analyze the feature information of the original text data, wherein the feature information includes at least one of the following: publication year, original text length, and number of keywords.

[0060] Specifically, the "publication year" is extracted from the metadata of the original text, the total number of characters or words in the text itself is counted as the "original text length," and the total number of keywords obtained through the S1 process is counted as the "keyword count." Furthermore, the feature information can be extended to one or more structured or quantifiable metadata such as text type, data source, and author level, depending on the application scenario. These features together constitute a multi-dimensional original feature set describing the text attributes.

[0061] For example, suppose there is a scientific paper abstract published in 2022, whose metadata contains "year":2022. The abstract is calculated to have 850 characters. Using the Text-Rank algorithm, 15 keywords are extracted from its content. Therefore, the initial feature vector obtained for this text is [2022, 850, 15].

[0062] S202. Perform logarithmic transformation and linear normalization on each feature information to map it to the interval [0,1].

[0063] The logarithmic transformation includes taking the natural logarithm after subtracting the minimum value from each feature data.

[0064] Specifically, this step aims to address the issues of inconsistent scales and severe skewed distribution of the original features, thereby improving the stability and convergence speed of model training. The process is as follows: First, for each feature dimension (e.g., "original text length"), find the minimum value (min) of that feature across the entire training dataset. Subtract this minimum value (x' = x - min) from all data values ​​of that feature. This shifts the feature values ​​to a non-negative interval, preparing for subsequent logarithmic calculations. Next, take the natural logarithm (x" = ln(x')) of the shifted non-negative values. This operation effectively compresses the numerical range of features with long-tailed distributions (e.g., text length), making their distribution closer to a normal distribution and reducing the impact of extreme values. Finally, use a min-max normalization method to map the logarithmic feature values ​​to the [0,1] interval. Specifically, calculate the maximum (max_log) and minimum (min_log) of the feature after taking the logarithm on the training set, and then use the formula: x_final = (x" - min_log) / (max_log - min_log) for calculation. Finally, all normalized features are concatenated according to their dimensions to form a numerical feature vector; the numerical feature vectors of the batch of texts are combined to construct the data feature tensor, which serves as one of the inputs to the deep neural network model.

[0065] For example, continuing from the example in S201, let's assume that the statistics obtained on the training set are:

[0066] Feature “Original text length”: minimum value min = 10, maximum value of (ln(x-10+1)) after logarithmic transformation max_log = 8.5, minimum value min_log = 0.

[0067] For the example text with a length of 850: first calculate 850-10=840; then take the logarithm ln(840+1)≈6.73; finally normalize (6.73-0) / (8.5-0)≈0.79.

[0068] Similarly, after processing the "publication year" and "number of keywords", the initial feature vector [2022,850,15] is normalized to a numerical vector similar to [0.75,0.79,0.62]. This vector is a component of the final data feature tensor.

[0069] S3. Use a pre-trained language model as the backbone network to extract contextual semantic features from the original text data.

[0070] S4. Combine data feature tensors with contextual semantic features to construct a deep neural network model to extract features from long texts.

[0071] Example 2:

[0072] Based on Example 1, Example 2 of this application provides a more specific method for deep neural network feature extraction applicable to long texts, including:

[0073] S1. Obtain the original text data, extract keywords from the original text data, and generate a word dictionary including the keywords.

[0074] S2. Extract the feature information of the original text data, perform normalization processing, and generate a data feature tensor.

[0075] S3. Use a pre-trained language model as the backbone network to extract contextual semantic features from the original text data.

[0076] The pre-trained language model is a pre-trained Fast-former model.

[0077] like Figure 4 As shown, S3 includes:

[0078] S301. Obtain the language model and its weights pre-trained on a large corpus.

[0079] Specifically, obtain the underlying Fast-former model from publicly available model repositories (such as Hugging Face Model Hub) or through other means. Fast-former is an efficient model based on Transformer, with its core goal being to reduce the computational complexity of the attention mechanism from O(n log n). 2The computation time is reduced to O(n) while retaining global dependency modeling capabilities. It generally follows the "encoder-decoder" framework, with a key innovation in the additive attention sublayer of the encoder layer: by generating a global query vector (aggregating overall sequence information), it replaces the traditional query vector for each token interacting with all keys and values. It first calculates global weights using key-value pairs to obtain a global vector, then calculates local weights using the global vector and keys, and finally weights the output to obtain the attention output. This is combined with residual connections, layer normalization, and a feedforward network to complete feature processing. The decoder layer structure is similar to the Transformer. Fast-former is designed with linear complexity to adapt to long sequence scenarios. This model should have been fully pre-trained on a large general-purpose corpus (such as Wikipedia) through self-supervised tasks (such as masked language modeling), thus giving it powerful general-purpose language understanding capabilities. When loading the model, its pre-trained weight parameters, which encode rich linguistic knowledge, must also be loaded. This step utilizes the idea of ​​transfer learning, transferring the knowledge learned by the model on general data to specific long text feature extraction tasks, thereby overcoming the problem of large amounts of data and computational resources required for training the model from scratch.

[0080] For example, in one specific implementation of this embodiment, a pre-trained model named fastformer-base is loaded. This model has a variant structure of a 12-layer Transformer, a hidden layer dimension of 768, 12 attention heads, and has been pre-trained on more than 10GB of English text data. By calling the model loading interface of a deep learning framework (such as PyTorch or TensorFlow), the weight file of fastformer-base is loaded into memory, and a model instance with the same structure is initialized.

[0081] S302. Remove the final task output layer of the pre-trained language model and define its output as the contextual semantic features extracted from the original long text.

[0082] Specifically, the word dictionary obtained in S1 is used to transform the original text into dictionary vectors, which are then input into the pre-trained model to obtain contextual semantic features. For example, the original Fast-former model typically includes an output layer for a specific downstream task (such as text classification), for instance, a linear classifier connected to a sequence summation vector (usually corresponding to the output of the [CLS] token). To convert the model into a general feature extractor (i.e., the backbone network), the output layer for this specific task needs to be removed. After removal, the output of the backbone network is defined as the sequence of output vectors corresponding to all tokens in the last layer of the model body (i.e., the Fast-former encoder), or a summary representation of that sequence (such as the vector corresponding to the [CLS] token, or a vector obtained by average pooling all token outputs). This high-dimensional, dense vector or sequence of vectors contains deep contextual semantic information of the input long text and will serve as input to the subsequent feature fusion module.

[0083] For example, continuing from the previous example, the original fastformer-base model ends with a classification layer that maps a 768-dimensional [CLS] vector to a 10-dimensional vector (corresponding to 10 classification labels). In S302, this classification layer is removed. For an input text with 512 tokens, the output of the modified backbone network can be a matrix of shape [512, 768] (representing the context vector of each token), or the output corresponding to the [CLS] token can be selected to obtain a [1, 768]-dimensional text-level semantic feature vector. For example, if the output of the [CLS] token is selected as the context semantic feature of the text, this 768-dimensional vector will be fed into the subsequent S4 step.

[0084] S4. Using the aforementioned word dictionary, data feature tensor, and backbone network, construct a deep neural network model to extract long text features.

[0085] like Figure 5 As shown, S4 includes:

[0086] S401. The data feature tensor is fused with the context semantic features to generate fused features.

[0087] The fusion in S401 is a splicing operation.

[0088] Specifically, this step aims to effectively integrate the macroscopic structural information (data feature tensor) of the text with deep semantic information (contextual semantic features). First, ensure that the two types of features are aligned in the batch size dimension. Assume the contextual semantic features extracted by the backbone network are a tensor of dimension [batch_size, D_semantic] (e.g., D_semantic = 768), while the data feature tensor is a tensor of dimension [batch_size, D_structural] (e.g., D_structural = 3, corresponding to three features such as publication year and text length). Next, concatenation is used as the core fusion strategy, connecting these two tensors along the feature dimension (the first dimension) to generate a fused feature tensor with dimensions [batch_size, D_semantic + D_structural]. This concatenation operation preserves all the information of the original features and provides a foundation for subsequent network layers to learn the interaction relationship between the two features. In addition to concatenation, alternative implementations can also use weighted addition or a shallow neural network for fusion.

[0089] For example, assume a batch size of 32. The semantic feature shape output by the backbone network is [32, 768], and the data feature tensor generated by S2 has a shape of [32, 3]. After concatenation and fusion, the resulting fused feature shape is [32, 771]. This means that each text sample is ultimately represented by a 771-dimensional fused vector, where the first 768 dimensions encode rich semantic context, and the last 3 dimensions supplement the macroscopic attributes of the text.

[0090] S402. Input the fused features into a deep neural network model, perform nonlinear transformation and dimensionality reduction, and output the final long text feature representation.

[0091] Specifically, the fused features obtained in step S401 are input into a feature transformation module consisting of one or more stacked fully connected layers. This module mainly performs two functions:

[0092] 1. Nonlinear transformation: Each fully connected layer is usually followed by a nonlinear activation function (such as ReLU or GELU), which enables the model to learn complex nonlinear relationships between features and enhances the representation ability of features.

[0093] 2. Dimensionality Reduction and Refinement: The output dimension of the final fully connected layer is set to the target feature dimension D_final (e.g., 128, 256, etc., much smaller than the fused feature dimension 771). This design aims to project the high-dimensional fused features into a lower-dimensional, denser, and more discriminative vector space, while also preventing overfitting and accelerating subsequent computations. The output of this layer is the final long text feature representation, which can be directly used for downstream tasks such as clustering, classification, and retrieval.

[0094] For example, continuing from the previous example, we input the fused features of shape [32, 771] into a two-layer fully connected network. The first layer maps the dimension from 771 to 256, followed by the GELU activation function; the second layer (output layer) maps the dimension from 256 to 128. The final output of this network is a tensor of shape [32, 128]. This 128-dimensional vector is the long text feature representation refined by the deep neural network and suitable for downstream tasks.

[0095] S5. Use a long text dataset to perform end-to-end supervised training on the deep neural network model built from S1 to S4 to optimize the model parameters.

[0096] Specifically, this step aims to enable the model to learn the most effective feature representations for a specific downstream task through supervised learning. First, a large-scale long text dataset with task-related labels is prepared as the training set. Then, a loss function directly related to the downstream task is selected, for example:

[0097] For text classification tasks, the cross-entropy loss function is used; for text similarity matching tasks, the contrastive loss or triplet loss is used; for retrieval tasks, the interval-based loss function is used.

[0098] During training, backpropagation and a gradient descent optimizer (such as AdamW) are used to calculate the gradient of the loss function with respect to all model parameters and update the parameters. The model parameters here include: the parameters of the pre-trained Fast-former backbone network loaded in step S3 (which can be fine-tuned), and the parameters of the fully connected layers introduced in step S4. Training adopts an "end-to-end" approach, meaning that from the initial input text through steps S1 to S4, to the final output features and loss calculation, the parameters of all modules are collaboratively optimized based on the final supervision signal. To prevent overfitting, regularization techniques such as Dropout, weight decay, or early stopping can be introduced during training.

[0099] For example, in a specific text classification task, suppose the downstream task is news topic classification. We use a dataset containing 100,000 long news articles and their topic tags (such as "sports", "technology", "finance", etc.).

[0100] Model Construction: As described in S1 to S4, a complete deep neural network model is constructed, where the last fully connected layer in S402 outputs a 128-dimensional feature vector. After this layer, we add an additional task-specific head for classification, such as a linear classifier that maps the 128-dimensional features to 10 dimensions (corresponding to 10 news topics).

[0101] Training configuration: The training epochs are set to 10, the batch size to 32, the AdamW optimizer is used, the initial learning rate is 2e-5, and weight decay is applied. The loss function used is cross-entropy loss.

[0102] Training Process: For a batch of training data, the model performs forward propagation: the original text passes through S1 (keyword extraction), S2 (feature statistical normalization), S3 (Fast-former backbone network), and S4 (feature fusion and transformation), outputting 128-dimensional features. These features are then passed through a classifier to obtain the predicted probabilities of 10 topics. The cross-entropy loss between the predicted probabilities and the true labels is calculated. Then, through backpropagation, gradients are calculated and model parameters are updated, including the parameters of the Fast-former and the fully connected layers. After multiple rounds of iterative training, the model parameters are optimized so that the extracted 128-dimensional long text features can best serve the news topic classification task. After training, the temporarily added classifier can be removed, retaining the optimized S1 to S4 parts as a powerful general long text feature extractor.

[0103] Furthermore, to verify the effectiveness of the deep neural network feature extraction method for long texts proposed in this invention, the classification performance of various deep neural network models (such as TextCNN, TextRNN, etc.) under different text features (with and without keywords) was tested. The experimental dataset contained text sequence numbers, text titles, long text content, and text labels. In the experiment, the dataset was divided into a training set (80%) and a test set (20%) in an 8:2 ratio. The training set was used for model training and parameter tuning, while the test set was used to evaluate the model's classification performance. Comparison methods included linear regression and traditional machine learning methods. The performance metric was classification accuracy, which measures the accuracy of the model's classification prediction of the input text; a value closer to 100% indicates a better model fit.

[0104] The experimental results are shown in Table 1 below:

[0105] Table 1

[0106]

[0107] As shown in Table 1, the deep neural network feature extraction method for long texts proposed in this invention outperforms traditional text feature extraction methods in terms of accuracy, indicating that this method can extract more valuable content from the text more accurately. Furthermore, the effect of keyword extraction is generally better than that of not extracting keywords, demonstrating the effectiveness of the keyword extraction method of this invention.

[0108] It should be noted that the parts in this embodiment that are the same as or similar to those in Embodiment 1 can be referred to each other, and will not be repeated in this application.

[0109] Example 3:

[0110] Based on Example 1, Example 3 of this application provides a deep neural network feature extraction system suitable for long texts, including:

[0111] The acquisition module is used to acquire raw text data, extract keywords from the raw text data, and generate a word dictionary including the keywords;

[0112] The first extraction module is used to extract the feature information of the original text data, perform normalization processing, and generate a data feature tensor.

[0113] The second extraction module is used to extract the contextual semantic features of the original text data by using a pre-trained language model as the backbone network.

[0114] The module is used to construct a deep neural network model to extract features from long texts using the word dictionary, data feature tensor, and backbone network.

[0115] It should be noted that the method provided in this embodiment is the corresponding system of the method provided in embodiment 1. Therefore, the parts that are the same as or similar to those in embodiment 1 in this embodiment can be referred to each other, and will not be repeated in this application.

Claims

1. A deep neural network feature extraction method suitable for long text, characterized in that, The method comprises the following steps: S1, obtaining original text data, extracting keywords from the original text data, and generating a word dictionary comprising the keywords; S2, extracting feature information of the original text data and performing normalization processing to generate a data feature tensor; S3, using a pre-trained language model as a backbone network to extract context semantic features of the original text data; S4, combining the data feature tensor and the context semantic features to construct a deep neural network model to extract long text features.

2. The deep neural network feature extraction method suitable for long text according to claim 1, wherein, S1 comprises the following steps: S101, obtaining original text data, extracting keywords from the original text data, and calculating an importance score of each keyword; S102, sorting all the keywords according to the importance scores; S103, selecting the top N keywords based on the sorting result, assigning a unique code to each keyword, and generating the word dictionary, wherein N is a predetermined positive integer.

3. The deep neural network feature extraction method for long text according to claim 2, wherein S2 The method comprises the following steps: S201, counting feature information of the original text data, wherein the feature information comprises at least one of a publication year, an original text length, and a keyword number; S202, performing logarithmic transformation and linear normalization processing on each item of feature information to map it to the [0, 1] interval.

4. The deep neural network feature extraction method for long text according to claim 3, wherein S3 The method comprises the following steps: S301, obtaining a pre-trained language model and its weights on a large corpus; S302, removing the final task output layer of the pre-trained language model, converting the original text into a dictionary vector using the word dictionary obtained in S1, inputting the pre-trained model, and obtaining context semantic features.

5. The method for long text according to claim 4, wherein S4 The method comprises the following steps: S401, fusing the data feature tensor and the context semantic features to generate a fusion feature; S402, inputting the fusion feature into a deep neural network model to perform nonlinear transformation and dimension reduction, and outputting a final long text feature representation.

6. The method for long text according to claim 5, wherein, The fusion in S401 is a splicing operation.

7. The method for long text according to claim 6, wherein, The method further comprises the following step: S5, using a long text data set to perform end-to-end supervised training on the deep neural network model constructed by S1 to S4 to optimize model parameters.

8. A deep neural network feature extraction system suitable for long text, characterized in that, The computer storage medium stores a computer program; when the computer program runs on a computer, the computer program makes the computer execute the method of any one of claims 1 to 7. The computer storage medium stores a computer program; when the computer program runs on a computer, the computer program makes the computer execute the method of any one of claims 1 to 7. The computer storage medium stores a computer program; when the computer program runs on a computer, the computer program makes the computer execute the method of any one of claims 1 to 7. The computer storage medium stores a computer program; when the computer program runs on a computer, the computer program makes the computer execute the method of any one of claims 1 to 7. The computer storage medium stores a computer program; when the computer program runs on a computer, the computer program makes the computer execute the method of any one of claims 1 to 7.

9. A computer storage medium, characterized in that The computer storage medium stores a computer program; when the computer program runs on a computer, the computer program makes the computer execute the method of any one of claims 1 to 7.

10. An electronic device, comprising: ​ ​ ​

Images