Text big data hierarchical classification method and system based on word frequency probability fusion

Abstract

The present disclosure relates to a hierarchical classification method and system for text big data based on word frequency probability fusion, including: obtaining original text data, performing data cleaning and data enhancement operations on the original text data to obtain input data; constructing a topic model based on probability distribution according to the input data, obtaining the topic-lexical relationship of the input data through the topic model based on probability distribution, and performing feature fusion of the topic-lexical relationship with the word embedding model to obtain a new word embedding model; constructing a deep learning network model based on the new word embedding model using an improved Transformer network model and a convolutional neural network; training and parameter optimization of the deep learning network model to obtain a trained deep learning network model; inputting the data to be classified into the trained deep learning network model to obtain a hierarchical classification result. The disclosed method can achieve accurate and stable classification of data.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data grading and classification, and in particular to a text big data grading and classification method based on word frequency probability fusion and a system thereof. Background Art

[0002] At present, with the development of the big data era, the amount and complexity of data are constantly increasing, and the problem of data classification and management has always been a key issue for academia and commercial data groups. Modern big data has the characteristics of large capacity, multiple categories, multiple associations, and diversified forms. This type of big data is mostly structured data, but it also contains some extremely long and complex unstructured data information. The current common text classification methods can be divided into rule-based methods, methods based on naive Bayes classifiers, methods based on support vector machine classifiers, methods based on deep learning classifiers, etc.

[0003] Word segmentation is to segment a continuous text stream into meaningful words. In many languages, there are usually no explicit separators between words, so word segmentation is a necessary preprocessing step and is crucial for subsequent language processing tasks such as part-of-speech tagging, grammatical analysis, information extraction, etc. Common text segmentation technologies include: dictionary-based methods, rule-based methods, statistical-based methods, machine learning-based methods, and deep learning-based methods.

[0004] In big data classification, word frequency information is extremely important due to its unique nature. Existing text representation methods cannot directly use this information. At the same time, in this type of big data, the data length varies greatly, and the extremely important relative position information cannot be fully used or is easily lost during the training process. How to effectively obtain the sparse features of big data while taking into account the extraction of complex data features and accurately identify the category to which the data belongs is an important issue currently faced. Summary of the invention

[0005] In order to solve the problems of inaccurate data classification, low precision, and inability to ensure data security in traditional data classification research methods when data features are sparse, data is extremely unbalanced, and data types are complex, this paper proposes a text big data classification method based on word frequency probability fusion to solve the above problems.

[0006] According to one aspect of the present disclosure, a text big data hierarchical classification method based on word frequency probability fusion is provided, comprising:

[0007] S10, obtaining original text data, and performing data cleaning and data enhancement operations on the original text data to obtain input data;

[0008] S20, constructing a topic model based on probability distribution according to the input data, obtaining the topic-word relationship of the input data through the topic model based on probability distribution, and fusing the topic-word relationship with the word embedding model to obtain a new word embedding model;

[0009] S30. Based on the new word embedding model, a deep learning network model is constructed using the improved Transformer network model and convolutional neural network;

[0010] S40, training and optimizing parameters of the deep learning network model to obtain a trained deep learning network model;

[0011] S50, input the data to be classified into the trained deep learning network model to obtain a hierarchical classification result.

[0012] Preferably, the topic-word relationship of the input data is obtained through a topic model based on probability distribution, which is expressed as:

[0013] ,

[0014] In the formula, represents the sampling theme, Indicates words, For sampling theme Chinese words The corresponding weight.

[0015] Preferably, the topic-vocabulary relationship is feature-fused with the word embedding model to obtain a new word embedding model, which is expressed as:

[0016] ,

[0017] In the formula, For each word fused word vector, For each word The corresponding word vector, K Indicates the number of topics.

[0018] Preferably, the deep learning network model includes a learnable position encoding layer that fuses relative positions and an improved encoder layer based on an attention convolution module.

[0019] Preferably, the learnable position encoding layer integrating relative positions includes introducing relative position encoding and learnable position encoding matrix, adding the position information of input data to the data features, which is expressed as:

[0020] ,

[0021] In the formula, LRPE represents the result after weighted average, PERepresents the relative position encoding result, L is a learnable position encoding matrix.

[0022] Preferably, the improved encoder layer based on the attention convolution module includes adding the position information of the input data to the attention convolution module, and performing function activation after amplifying the features of the input data through the convolution kernel, which is expressed as:

[0023] ,

[0024] The new attention calculation formula is expressed as:

[0025] ,

[0026] In the formula, Q, K, V is the input of the attention mechanism, is the dimension of word embedding.

[0027] Preferably, the data to be classified is input into a trained deep learning network model, including: after the data to be classified is input into the trained deep learning network model, field prediction, table prediction, and library prediction are performed in sequence.

[0028] According to one aspect of the present disclosure, a text big data hierarchical classification system based on word frequency probability fusion is provided, comprising:

[0029] An input data acquisition module acquires original text data and performs data cleaning and data enhancement operations on the original text data to obtain input data;

[0030] A word embedding feature fusion module, which constructs a topic model based on probability distribution according to the input data, obtains the topic-word relationship of the input data through the topic model based on probability distribution, and performs feature fusion of the topic-word relationship with the word embedding model to obtain a new word embedding model;

[0031] Model building module, based on the new word embedding model, uses the improved Transformer network model and convolutional neural network to build a deep learning network model;

[0032] A model training module is used to train and optimize the parameters of the deep learning network model to obtain a trained deep learning network model;

[0033] The data grading and classification module inputs the data to be classified into the trained deep learning network model to obtain the grading and classification results.

[0034] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to: execute the above-mentioned text big data hierarchical classification method based on word frequency probability fusion.

[0035] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above-mentioned text big data hierarchical classification method based on word frequency probability fusion is implemented.

[0036] Compared with the prior art, the beneficial effects of the present disclosure are:

[0037] 1) This paper proposes a word embedding feature fusion method based on word frequency probability. By extracting the topic-vocabulary relationship of the document and fusing it with the word embedding feature, word frequency probability information is added to the word embedding data, thereby improving the classification accuracy in special fields.

[0038] 2) This disclosure proposes an improved learnable relative position encoding, which introduces relative position encoding and a learnable position encoding matrix to add position information to data features, thereby solving the shortcomings of traditional position encoding, namely, insufficient relative position information and insufficient generalization ability.

[0039] 3) This paper proposes an attention convolution module (AC module), which adds position information to the attention operation and activates the ReLU function after amplifying the features with a 1*1 convolution kernel, so that the position information can be better captured and used by the attention mechanism, avoiding the problem of position information loss in deep networks.

[0040] 4) A cascade model based on the Transformer module and the CNN module is proposed. By extracting features globally and locally, the data features are strengthened, and by adding improved position information, it can still maintain good and accurate results in the case of data classification with extremely unbalanced data length and extremely complex data types.

[0041] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0042] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0044] Figure 1 The flowchart of the text big data classification method based on word frequency probability fusion is shown;

[0045] Figure 2A flow chart of a method for intelligent data classification in an embodiment of the present disclosure is shown;

[0046] Figure 3 The structure diagram of the model based on the improved Transformer in the embodiment of the present disclosure is shown;

[0047] Figure 4 A schematic diagram of an improved Transformer network model based on an AC module in an embodiment of the present disclosure is shown;

[0048] Figure 5 A flowchart of hierarchical classification processing in an embodiment of the present disclosure is shown;

[0049] Figure 6 The structural block diagram of the text big data hierarchical classification system based on word frequency probability fusion is shown. DETAILED DESCRIPTION

[0050] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0051] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0052] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0053] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0054] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0055] Example 1

[0056] Based on the above ideas, the present invention proposes a text big data hierarchical classification method based on word frequency probability fusion. Figure 1 A flowchart of a text big data classification method based on word frequency probability fusion is shown. The method includes:

[0057] S10, obtaining original text data, and performing data cleaning and data enhancement operations on the original text data to obtain input data;

[0058] S20, constructing a topic model based on probability distribution according to the input data, obtaining the topic-word relationship of the input data through the topic model based on probability distribution, and fusing the topic-word relationship with the word embedding model to obtain a new word embedding model;

[0059] S30. Based on the new word embedding model, a deep learning network model is constructed using the improved Transformer network model and convolutional neural network;

[0060] S40, training and optimizing parameters of the deep learning network model to obtain a trained deep learning network model;

[0061] S50, input the data to be classified into the trained deep learning network model to obtain a hierarchical classification result.

[0062] The present disclosure provides a text big data hierarchical classification method based on word frequency probability fusion, including the following steps:

[0063] S10: Obtain original text data, and perform data cleaning and data enhancement operations on the original text data to obtain input data.

[0064] The purpose of this embodiment is to intelligently classify and organize database data through a deep learning network model. Based on an improved Transformer model and combined with a CNN network, a method for intelligent data classification is provided to achieve accurate and stable data classification. The flow chart of the intelligent data classification method is as follows: Figure 2 shown.

[0065] Collect raw text data, perform data cleaning and data enhancement operations to obtain complete input data. The actual application is deployed on the data processing system, through which the system collects and cleans the data, and performs data preprocessing after converting the data format. The data acquisition stage and the cleaning stage can be implemented in the data storage location to ensure data security to the greatest extent.

[0066] S20. Construct a topic model based on probability distribution according to the input data, obtain the topic-word relationship of the input data through the topic model based on probability distribution, and fuse the topic-word relationship with the word embedding model to obtain a new word embedding model.

[0067] In this embodiment, represents the prior parameters of the document-topic distribution, represents the prior parameters of the topic-vocabulary distribution, K Represents the number of topics, N Represents the number of words, M represents the number of documents. Then for each document d , from the document-topic distribution Random sampling was conducted in , and then according to Corresponding topic-word distribution Extract words from it, . Repeat the above steps until all the words in the document are generated.

[0068] The topic-word relationship of the input data is obtained through the topic model based on probability distribution, which can be expressed as a weight matrix, expressed as:

[0069] ,

[0070] In the formula, represents the sampling theme, Indicates words, For sampling theme Chinese words The corresponding weight.

[0071] For each word, the word embedding model word2vec will have a one-to-one corresponding word vector. The topic-vocabulary relationship is fused with the word embedding model to obtain a new word embedding model, which is expressed as:

[0072] ,

[0073] In the formula, For each word fused word vector, For each word The corresponding word vector,K Indicates the number of topics.

[0074] For an input, the input tensor shape before word embedding is:

[0075] ,

[0076] After word embedding, its dimension increases and the shape becomes:

[0077] ,

[0078] In the formula, The value of the same batch data size, padding (filling 0) operation is performed to fill the insufficient data, The maximum length of each data that can be read. is the word embedding dimension of the result obtained when each word is vectorized.

[0079] The feature fusion of topic-vocabulary relationship and word embedding model does not change the shape of the tensor. This process obtains the topic-vocabulary relationship of the input data through the topic model based on probability distribution, and maps this relationship to obtain the corresponding weight value. By incorporating topic-vocabulary information into the word embedding representation, the hidden information of the data at the word level is reflected, so that the data can be more accurately mapped to the corresponding topic after feature calculation in high-dimensional space, so that the data already contains its related topic information before feature calculation, so that the model can better understand the information contained in the data itself.

[0080] S30. Based on the new word embedding model, a deep learning network model is constructed using the improved Transformer network model and convolutional neural network.

[0081] In this embodiment, a deep learning network model is constructed using an improved Transformer network and a convolutional neural network. For the improved Transformer network model, on the one hand, the original position coding based on sine and cosine functions is improved to a learnable position coding that integrates relative position coding, and on the other hand, an attention convolution (AC) module is introduced before the original self-attention module. The deep learning network model includes a learnable position coding layer that integrates relative positions and an improved encoder layer based on an attention convolution module. Specifically:

[0082] The improved Transformer network model is defined, including a learnable position encoding layer that integrates relative positions, an improved multi-head attention layer, the first Dropout layer (hidden layer), the first residual connection and normalization layer, a feedforward fully connected layer, the second Dropout layer (hidden layer), the second residual connection and normalization layer, a convolutional layer, an activation layer, a maximum pooling layer, a linear layer, and a Softmax classification layer. The structure of the model based on the improved Transformer is shown in the figure below. Figure 3 shown.

[0083] The learnable position encoding layer that integrates relative positions includes introducing relative position encoding and learnable position encoding matrix, adding the position information of input data to the data features, expressed as:

[0084] ,

[0085] In the formula, LRPE represents the result after weighted average, PE Represents the relative position encoding result, L It is a learnable position encoding matrix, which is added to the parameters to continuously update the parameters during the training process. Through the layer normalization operation, the numerical stability is maintained and the gradient explosion is prevented, so as to achieve generalization effect and adapt to the current task.

[0086] The input of this layer is the input tensor after word embedding. The learnable position encoding layer that integrates relative position does not change its tensor shape. It aims to integrate data position information as subsequent known information, so the output of this layer is still the input tensor after word embedding. The learnable position encoding layer that integrates relative position improves the position encoding so that while integrating relative position information, relative position information is essentially a fixed trigonometric function and cannot handle complex position information. A learnable parameter matrix is ​​introduced to make it have better generalization effect and better adapt to complex data content with complex relative position information in text data classification tasks.

[0087] Furthermore, an improved encoder layer based on the attention convolution module is constructed, and the improved encoder layer is composed of an improved multi-head attention layer based on the AC module, a first Dropout layer (hidden layer), a first residual connection and normalization layer, a feedforward fully connected layer, a second Dropout layer (hidden layer), a second residual connection and normalization layer.

[0088] The improved multi-head attention layer is based on the original self-attention mechanism. On the basis of the original Q (query) and K (key) feature similarity calculation, an AC module is introduced to process V (value).

[0089] The basic idea of ​​the attention convolution module is to introduce position information into V (value) so that it can be integrated into each attention calculation to avoid the problem of position information loss caused by encoder stacking. Due to the complexity of government data, the information contained in its position information will be richer in some cases and have stronger guidance. By improving the AC module, this problem of the original model is overcome.

[0090] Among them, the improved encoder layer based on the attention convolution module includes adding the position information of the input data to the attention convolution module, amplifying the features of the input data through the convolution kernel and then activating the function, performing weighted summation of the original Value tensor and the LRPE tensor, passing through a convolution layer with a convolution kernel of 1*1, and then activating it through the Relu function as a new Value input, which is expressed as:

[0091] ,

[0092] The new attention calculation formula is expressed as:

[0093] ,

[0094] In the formula, Q, K, V is the input of the attention mechanism, is the dimension of word embedding.

[0095] The improved encoder layer based on the attention convolution module avoids the loss of position information caused by the stacking of encoder layers by introducing the AC module, and introduces the position information to participate in the calculation of each attention calculation, which increases the proportion of position information and makes it more accurate for data classification with complex position information such as government data.

[0096] The first Dropout layer discards some parameters by using the Dropout function and reduces the correlation between neurons. This layer mainly plays a role in preventing overfitting and improving generalization ability.

[0097] The schematic diagram of the improved Transformer network model based on the AC module is as follows Figure 4As shown. The model includes a first residual connection level normalization layer, a feedforward fully connected layer, a second residual connection and normalization layer, an improved multi-head attention mechanism and an AC module, wherein the AC module includes an activation function, a 1*1 convolution layer and position information fusion. The first residual connection and normalization layer performs a weighted summation of the input before attention calculation and the input after calculation through a residual connection, and maintains its value in a normal range through a normalization layer, so that the deep learning network model avoids the phenomenon of gradient explosion or gradient disappearance, so that the model can be better trained, and the residual connection operation retains the original information, improving the robustness of the deep learning network model.

[0098] The feed-forward fully connected layer is a fully connected network consisting of two linear layers, which is intended to increase the fitting degree of the deep learning network model in complex situations.

[0099] The second Dropout layer (hidden layer) functions the same as the first Dropout layer (hidden layer).

[0100] The second residual connection and normalization layer have the same function as the first residual connection and normalization layer.

[0101] At this point, the construction of the deep learning network model is completed. The complete model is composed of 6 stacked encoders, with the input and output connected in series, and the encoder does not change the shape of the input and output tensors, so the shape of the output tensor is the same as the input tensor after word embedding.

[0102] The convolutional neural network includes convolutional layers, activation layers, maximum pooling layers and linear layers. The global information of the data is extracted through the improved encoder layer based on the AC module, and then the local features are partially extracted through the convolutional neural network to enhance the data feature extraction effect and improve the accuracy.

[0103] By cascading the improved encoder layer based on the attention convolution module and the convolutional neural network, after obtaining the position information and the global features of the data through the improved encoder layer, the local features are further extracted through the convolutional neural network to obtain better classification results.

[0104] S40: training and optimizing parameters of the deep learning network model to obtain a trained deep learning network model.

[0105] The deep learning network model constructed by the method described in this embodiment is used to train the mapped data, and the model is continuously updated iteratively through continuous iterative calculations until the training effect is optimal. The training results are estimated, and the optimal result is determined by the evaluation function to achieve the screening of the deep learning network model.

[0106] In the deep learning network model training, model verification, and deployment application phase, the trained deep learning network model is obtained through training on existing equipment and deployed in the production environment. Traditional deep learning network models are extremely unstable for such database classification tasks, because although CNN has an advantage in obtaining local information, it cannot handle the positional relationship contained in the data, resulting in instability in its results. After the introduction of Transformer and the addition of improvements, the stability was greatly improved while increasing its accuracy for such tasks, achieving good results.

[0107] S50, input the data to be classified into the trained deep learning network model to obtain a hierarchical classification result.

[0108] In this embodiment, the data to be classified is input into the trained deep learning network model, including: after the data to be classified is input into the trained deep learning network model, field prediction, table prediction, and library prediction are performed in sequence. Figure 5 It includes a single data classification model training module, a table structure model training module and a library structure model training module, wherein the single data classification model training module is used to train text information, the table structure model training module is used to train table structure information, and the library structure model training module is used to train library structure information.

[0109] After completing the classification of the input big data, the existing deep learning model is reused, and the input data of the table structure prediction model is constructed in combination with the field name and classification results to predict and classify the table structure information. For the existing standard library structure, the library structure prediction model input data is constructed in combination with the table information to predict and classify the library structure information. The specific process is as follows: preprocess the single table prediction data, perform data prediction through the probability fusion model to obtain the classification prediction results of the entire table, further perform data processing on the classification prediction results of the entire table, obtain the table prediction results through table prediction, further perform data processing on the table prediction results, and obtain the library results through library prediction. Based on the above steps, the hierarchical classification prediction of big data is completed, and the intelligent governance of big data is realized.

[0110] Example 2

[0111] As another aspect of the embodiment of the present disclosure, a text big data hierarchical classification system 100 based on word frequency probability fusion is also provided. Figure 6 As shown, including:

[0112] An input data acquisition module 1 is used to acquire original text data and to perform data cleaning and data enhancement operations on the original text data to obtain input data;

[0113] A word embedding feature fusion module 2 constructs a topic model based on probability distribution according to the input data, obtains the topic-word relationship of the input data through the topic model based on probability distribution, and performs feature fusion of the topic-word relationship with the word embedding model to obtain a new word embedding model;

[0114] Model building module 3, based on the new word embedding model, uses the improved Transformer network model and convolutional neural network to build a deep learning network model;

[0115] Model training module 4, training and parameter optimization of the deep learning network model to obtain a trained deep learning network model;

[0116] The data grading and classification module 5 inputs the data to be classified into the trained deep learning network model to obtain the grading and classification results.

[0117] In the absence of any contradiction, the above modules in the system of the embodiment of the present disclosure can implement any implementation of the above method.

[0118] Based on the description of the above embodiments, it can be seen that the embodiments of the present disclosure can achieve the following technical effects:

[0119] 1) This paper proposes a word embedding feature fusion method based on word frequency probability. By extracting the topic-vocabulary relationship of the document and fusing it with the word embedding feature, word frequency probability information is added to the word embedding data, thereby improving the classification accuracy in special fields.

[0120] 2) This disclosure proposes an improved learnable relative position encoding, which introduces relative position encoding and a learnable position encoding matrix to add position information to data features, thereby solving the shortcomings of traditional position encoding, namely, insufficient relative position information and insufficient generalization ability.

[0121] 3) This paper proposes an attention convolution module (AC module), which adds position information to the attention operation and activates the ReLU function after amplifying the features with a 1*1 convolution kernel, so that the position information can be better captured and used by the attention mechanism, avoiding the problem of position information loss in deep networks.

[0122] 4) A cascade model based on the Transformer module and the CNN module is proposed. By extracting features globally and locally, the data features are strengthened, and by adding improved position information, it can still maintain good and accurate results in the case of data classification with extremely unbalanced data length and extremely complex data types.

[0123] The embodiment of the present disclosure also proposes an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above-mentioned text big data hierarchical classification method based on word frequency probability fusion. The electronic device can be provided as a terminal, a server or other forms of equipment.

[0124] The present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the text big data hierarchical classification method based on word frequency probability fusion is implemented. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0125] Those skilled in the art will understand that in the above-mentioned text big data hierarchical classification method and system based on word frequency probability fusion in the specific implementation mode, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0126] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0127] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A hierarchical classification method for text big data based on word frequency probability fusion, characterized in that: The steps include: S10, obtaining original text data, and performing data cleaning and data enhancement operations on the original text data to obtain input data; S20, construct a topic model based on probability distribution according to the input data, obtain the topic-word relationship of the input data through the topic model based on probability distribution, fuse the topic-word relationship with the word embedding model to obtain a new word embedding model; obtain the topic-word relationship of the input data through the topic model based on probability distribution, expressed as: ， In the formula, represents the sampling theme, Indicates words, For sampling theme Chinese words The corresponding weighted new word embedding model is expressed as: ， In the formula, For each word fused word vector, For each word The corresponding word vector, K Indicates the number of topics; S30. According to the new word embedding model, a deep learning network model is constructed using an improved Transformer network model and a convolutional neural network; the deep learning network model includes a learnable position encoding layer that integrates relative positions and an improved encoder layer based on an attention convolution module; The improved Transformer network model includes a learnable position encoding layer integrating relative positions, an improved multi-head attention layer, a first Dropout layer, a first residual connection and normalization layer, a feedforward fully connected layer, a second Dropout layer, a second residual connection and normalization layer, a convolution layer, an activation layer, a maximum pooling layer, a linear layer and a Softmax classification layer, wherein the learnable position encoding layer integrating relative positions includes introducing relative position encoding and a learnable position encoding matrix, and adding the position information of the input data to the data features, which is expressed as: ， In the formula, LRPE represents the result after weighted average, PE Represents the relative position encoding result, L is a learnable position encoding matrix; The improved encoder layer based on the attention convolution module includes adding the position information of the input data to the attention convolution module, amplifying the features of the input data through the convolution kernel, and performing function activation, which is expressed as: ， The new attention calculation formula is expressed as: ， In the formula, Q, K, V is the input of the attention mechanism, is the dimension of word embedding; S40, training and optimizing parameters of the deep learning network model to obtain a trained deep learning network model; S50, input the data to be classified into the trained deep learning network model to obtain a hierarchical classification result.

2. The method according to claim 1, characterized in that The data to be classified is input into the trained deep learning network model, including: after the data to be classified is input into the trained deep learning network model, field prediction, table prediction, and library prediction are performed in sequence.

3. A text big data hierarchical classification system based on word frequency probability fusion, characterized by: include: An input data acquisition module acquires original text data and performs data cleaning and data enhancement operations on the original text data to obtain input data; The word embedding feature fusion module constructs a topic model based on probability distribution according to the input data, obtains the topic-word relationship of the input data through the topic model based on probability distribution, and fuses the topic-word relationship with the word embedding model to obtain a new word embedding model; the topic-word relationship of the input data is obtained through the topic model based on probability distribution, which is expressed as: ， In the formula, represents the sampling theme, Indicates words, For sampling theme Chinese words The corresponding weighted new word embedding model is expressed as: ， In the formula, For each word fused word vector, For each word The corresponding word vector, K Indicates the number of topics; A model building module, which builds a deep learning network model based on the new word embedding model using an improved Transformer network model and a convolutional neural network; the deep learning network model includes a learnable position encoding layer that integrates relative positions and an improved encoder layer based on an attention convolution module; The improved Transformer network model includes a learnable position encoding layer integrating relative positions, an improved multi-head attention layer, a first Dropout layer, a first residual connection and normalization layer, a feedforward fully connected layer, a second Dropout layer, a second residual connection and normalization layer, a convolution layer, an activation layer, a maximum pooling layer, a linear layer and a Softmax classification layer, wherein the learnable position encoding layer integrating relative positions includes introducing relative position encoding and a learnable position encoding matrix, and adding the position information of the input data to the data features, which is expressed as: ， In the formula, LRPE represents the result after weighted average, PE Represents the relative position encoding result, L is a learnable position encoding matrix; The improved encoder layer based on the attention convolution module includes adding the position information of the input data to the attention convolution module, amplifying the features of the input data through the convolution kernel, and performing function activation, which is expressed as: ， The new attention calculation formula is expressed as: ， In the formula, Q, K, V is the input of the attention mechanism, is the dimension of word embedding; A model training module is used to train and optimize the parameters of the deep learning network model to obtain a trained deep learning network model; The data grading and classification module inputs the data to be classified into the trained deep learning network model to obtain the grading and classification results.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the text big data hierarchical classification method based on word frequency probability fusion as described in any one of claims 1 to 2 is implemented.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the text big data hierarchical classification method based on word frequency probability fusion as described in any one of claims 1 to 2 is implemented.

Images