An intelligent classification method, system and storage medium for complaint texts with multi-dimensional feature extraction based on cross-layer attention mechanism

Abstract

The present invention discloses an intelligent complaint text classification system for multi-dimensional feature extraction based on a cross-layer attention mechanism, which includes a text data acquisition layer, a text data storage layer, a text data preprocessing layer, a word embedding layer, a text data feature extraction layer, and a text classification information application layer. By adopting the above intelligent complaint text classification system for multi-dimensional feature extraction based on a cross-layer attention mechanism, the present invention uses a more meticulous text preprocessing method to improve the credibility of text information, thereby enhancing the final classification effect of the text, comprehensively extracting various aspects of text features, making the system have strong robustness, and can be used for enterprises to timely supervise customer complaints, reduce the manpower input in complaint handling, and improve the efficiency of complaint handling. It can also be used in fields such as smart home and smart healthcare involving text classification. The present invention also proposes an intelligent complaint text classification method and storage medium for multi-dimensional feature extraction based on a cross-layer attention mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and in particular to an intelligent classification method, system and storage medium for complaint texts with multi-dimensional feature extraction based on a cross-layer attention mechanism. Background Art

[0002] With the continuous development of network technology, there is a vast amount of information data in the Internet, especially text data, which has the richest data volume. These data have high practical value, and they can be utilized to dig out the hidden information therein, helping decision-makers reduce the workload, lower the labor cost, and observe the development trend of things, which is beneficial to the development of subsequent work. How to quickly and effectively obtain the required text information is an important branch studied in the field of artificial intelligence, and the classification task based on text content is gradually integrated into everyone's daily life. Various intelligent devices based on Natural Language Processing (NLP) technology have been widely applied in various fields such as smart home, smart healthcare, and even production and life.

[0003] As a basic task in natural language processing, text classification plays a crucial role in many tasks such as sentiment analysis and question classification. As is well known, different NLP tasks require different language features. Tasks such as text classification require more semantic features, while tasks such as dependency parsing require more syntactic features. Most of the existing methods mainly improve performance by mixing and calibrating features without distinguishing the feature types and corresponding effects, resulting in a discount in the accuracy of classification recognition. Before feature extraction, data preprocessing, which has the same influence on the classification result, is also very important. If the original data is directly modeled, this approach is very irresponsible. On the one hand, the quality of the data is uneven, filled with various noises, and there may even be errors. We must carefully examine the distribution of the original data, grasp the overall characteristics of the data, and eliminate these incorrect data. On the other hand, preprocessing before modeling can reduce the burden on the model and improve the recognition accuracy.

[0004] Text-based classification and recognition tasks mainly adopt machine learning and deep learning algorithms. From the 1960s to around 2010, shallow learning text classification models based on machine learning dominated. Shallow learning refers to statistical-based models, such as Naive Bayes (NB), K-Nearest Neighbor (KNN), and Support Vector Machine (SVM). Compared with early rule-based methods, this method has obvious advantages in terms of accuracy and stability. However, these methods still require functional design, which is both time-consuming and expensive. In addition, they usually ignore the natural order structure or context information in text data, making it difficult to learn the semantic information of words. Since 2010, the mainstream models used in the text classification process have gradually changed from shallow learning models to deep learning models based on artificial neural networks. Compared with shallow learning-based methods, deep learning methods avoid manual rule and function design and automatically provide semantically meaningful representations for text mining. Therefore, most text classification research works are based on Deep Neural Networks (DNN), which is a data-driven method with high computational complexity. Few studies focus on using shallow learning models to address computational and data limitations. Summary of the Invention

[0005] The purpose of the present invention is to provide an intelligent classification method, system, and storage medium for multi-dimensional feature extraction of complaint texts based on a cross-layer attention mechanism, which uses a more meticulous text preprocessing method to improve the credibility of text information, thereby enhancing the final classification effect of the text, being able to extract various aspects of features of the text more comprehensively, making the system have strong robustness, and can be used for enterprises to timely supervise customer complaints, reduce the manpower input in complaint handling, and improve the efficiency of complaint handling. It can also be used in fields such as smart home and smart healthcare involving text classification.

[0006] To achieve the above purpose, the present invention provides an intelligent classification system for multi-dimensional feature extraction of complaint texts based on a cross-layer attention mechanism, including a text data acquisition layer, a text data storage layer, a text data preprocessing layer, a word embedding layer, a text data feature extraction layer, and a text classification information application layer connected in sequence;

[0007] The text data acquisition layer uses a crawler program to acquire the required commodity complaint text data;

[0008] The text data storage layer stores the original commodity complaint text data collected in a local server or a cloud server;

[0009] The word embedding layer embeds the preprocessed text information into word vectors using the GloVe technique, enabling each word or character to be represented in a low-dimensional vector space, which facilitates subsequent feature extraction by the classifier.

[0010] Preferably, the text data preprocessing layer includes preprocessing the crawled original text data to obtain preprocessed complaint text information.

[0011] Preferably, the text data feature extraction layer includes an MDFECA network module, and the MDFECA network module includes a stacked Transformer structure global feature extraction unit, a max pooling unit, a cross-layer attention unit, a convolutional block attention module spatio-temporal feature extraction unit, a feature fusion unit, a multi-layer perceptron unit, and an output unit.

[0012] Preferably, the text classification information application layer is used to: classify and store the text data and the recognition results in the database server simultaneously, and distribute the complaint information of the same type to the corresponding departments for processing.

[0013] The above-mentioned intelligent classification method for complaint text with multi-dimensional feature extraction based on cross-layer attention mechanism includes the following steps:

[0014] Step S1: Obtain complaint text data. Select users on the consumer service platform to crawl text data of the complaint content.

[0015] Step S2: Store text data. Store the obtained original text data in the local server or the cloud server.

[0016] Step S3: Preprocess the original text data. Preprocess the obtained original text data to obtain preprocessed complaint text data, which is convenient for subsequent word embedding operations.

[0017] Step S4: Construct a text pre-training model. Embed the preprocessed text information into word vectors using the GloVe technique, enabling each word or character to be represented in a low-dimensional vector space.

[0018] Step S5: Construct and train a text classification model. Input the data processed by the complaint text pre-training module into the text classifier in the trained complaint text classification module for classification and recognition, and output the classification recognition result corresponding to the text content.

[0019] Step S6: Store and apply text classification information. Through the text classification information storage and application module, store the results of complaint text classification, and the information can be classified and delivered to the departments for processing different types of complaint information.

[0020] Preferably, in step S1, for data crawling, the PyCharm editor is used, and the urllib module in Python is utilized. By defining a crawler class, a crawler program is written to convert various unstructured information on the web page into semi-structured tables and store them in a local server or a cloud server for subsequent analysis and processing.

[0021] Preferably, in step S3, the preprocessing of the original text data includes the following steps:

[0022] Step S31: Redundant information processing. The obtained original text data contains information in many fields, and a part of it is fields that are useless for text classification in this task. After redundant information processing, this part of the content is removed.

[0023] Step S32: Text cleaning. The text cleaning stage involves a stop word library specifically formulated for the content of complaint texts.

[0024] Step S33: Standardization. For the traditional Chinese characters that often appear in the complaint content, they need to be standardized into simplified Chinese characters, and the standardization is completed using the Opencc toolkit.

[0025] Step S34: Sentence splitting. The re module is introduced for sentence splitting processing, and re.split(·) is used to perform the sentence splitting operation, and the string to be split is set.

[0026] Step S35: Word segmentation. The Chinese word segmentation library Jieba is used to complete the Chinese word segmentation work, and after word segmentation, the fully preprocessed text data is obtained.

[0027] Preferably, in step S4, the text pre-training model is:

[0028] S = W( e )V

[0029] In the formula, V represents the form of the sentence converted into a vector, W (e) represents the word embedding weight matrix, and S represents the low-dimensional dense vector representation of the sentence.

[0030] Preferably, in step S5, the construction and training of the text classification model is to obtain word vectors through the application of the GloVe technology, and input them into the text classifier of the constructed MDFECA network module for text classification output, including the following steps:

[0031] Step S51: Capture global context information;

[0032] Step S52: Cross-layer attention mechanism;

[0033] Step S53: Capture important local features;

[0034] Step S54: Feature splicing;

[0035] Step S55: Fully connected layer.

[0036] A computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of the above-mentioned intelligent classification method for complaint texts with multi-dimensional feature extraction based on cross-layer attention mechanism are implemented.

[0037] The advantages and positive effects of the intelligent classification method, system and storage medium for complaint texts with multi-dimensional feature extraction based on cross-layer attention mechanism of the present invention are as follows:

[0038] 1. Practicality: The classification of text information has high requirements for the quality of the original data and the accuracy rate. The present invention can accurately obtain text data for specific task scenarios by using a suitable data acquisition method; at the same time, it also has certain advantages in the accuracy rate of text classification.

[0039] 2. Adaptability: For different application scenarios, by modifying the number of stacked Transformer layers, the network unit structure of the convolutional block attention module, etc., the universality of the input data is improved.

[0040] 3. High reliability: Compared with the mainstream model algorithms, the sufficient data preprocessing process and the classification neural network that can extract richer features make the present system have strong robustness, and at the same time, the accuracy rate is further improved.

[0041] Next, through the drawings and embodiments, the technical solutions of the present invention will be further described in detail. Description of the Drawings

[0042] Figure 1 It is a schematic diagram of the main module structure framework and connection relationship of an intelligent classification system for complaint texts with multi-dimensional feature extraction based on cross-layer attention mechanism of the present invention;

[0043] Figure 2 It is a schematic diagram of the module composition and connection relationship implemented by an intelligent classification system for complaint texts with multi-dimensional feature extraction based on cross-layer attention mechanism of the present invention;

[0044] Figure 3 It is a schematic diagram of the flow of an intelligent classification method for complaint texts with multi-dimensional feature extraction based on cross-layer attention mechanism of the present invention;

[0045] Figure 4 It is a schematic diagram of the working principle of the MDFECA network module in an intelligent classification system for complaint texts with multi-dimensional feature extraction based on cross-layer attention mechanism of the present invention. Detailed Embodiments

[0046] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0047] Unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the ordinary meanings understood by those with ordinary skills in the field to which the present invention pertains.

[0048] Embodiment 1

[0049] An intelligent classification and recognition system for multi-dimensional feature extraction of complaint texts based on a cross-layer attention mechanism, which is applied to the intelligent classification of commodity complaint texts, such as Figure 1 shown, including a text data acquisition layer, a text data storage layer, a text data preprocessing layer, a word embedding layer, a text data feature extraction layer, and a text classification information application layer connected in sequence.

[0050] The text data acquisition layer is used to: acquire the required commodity complaint text data using a crawler program.

[0051] The text data storage layer is used to: store the original commodity complaint text data collected in a local server or a cloud server for subsequent analysis and processing.

[0052] The text data preprocessing layer is used to: preprocess the crawled original text data, specifically referring to: performing operations such as redundant information processing, text cleaning, standardization, sentence splitting, and word segmentation in sequence to obtain the preprocessed complaint text information, which is convenient for subsequent word embedding operations.

[0053] Redundant information processing means that: when classifying commodity complaint texts, what we care about is the content of the complaint itself, rather than who made the complaint and when it was made, etc. Therefore, information such as the customer's ID number, nickname, and complaint date belongs to redundant information in this task and should be removed during data preprocessing. Doing so can reduce the dimension of the original data, reduce the difficulty of analysis, and is conducive to grasping the essence of things and analyzing the main contradictions.

[0054] Text cleaning means that: the text after removing redundant information will still contain some stop words, tag symbols, expressions, and emoticons that are useless for text classification. It is necessary to observe the crawled commodity complaint text data to formulate a special stop word library to filter out useless characters, words, and symbols, etc., to obtain the denoised text, which will be more conducive to improving the accuracy of text classification.

[0055] Standardization means that: due to user habits in complaint text information, the text often contains traditional Chinese characters, which usually has a certain negative impact on the classification accuracy. Therefore, it is crucial to convert the traditional Chinese characters in the text into simplified Chinese characters and standardize them.

[0056] The word embedding layer is used to: apply the GloVe technology to embed the preprocessed text information into word vectors, so that each word and character can be expressed in a low-dimensional vector space, facilitating the subsequent classifier to extract features.

[0057] The text data feature extraction layer includes the MDFECA network module, and the MDFECA network module includes a stacked Transformer structure global feature extraction unit, a max pooling unit, a cross-layer attention unit, a convolutional block attention module spatio-temporal feature extraction unit, a feature fusion unit, a multi-layer perceptron unit, and an output unit.

[0058] The text data feature extraction layer is used to: input the text vector into the model, and respectively pass through the stacked Transformer structure global feature extraction unit, the max pooling unit, the cross-layer attention unit, and the convolutional block attention module spatio-temporal feature extraction unit to obtain the important features of the text. The output results of each unit are input into the feature fusion unit for splicing, and then the final classification result is output by the multi-layer perceptron unit and the output unit.

[0059] The text classification information application layer is used to: classify and store the text data and the recognition results in the database server at the same time, and distribute the complaint information of the same type to the corresponding departments that handle this type of complaint for processing.

[0060] A relatively optimized system is proposed in four aspects of specific text data acquisition, text information preprocessing, text classification recognition, and application. It not only provides a solution to the problem of the lack of text classification data sets for a specific task in the current research field, but also makes up for the problem of insufficient comprehensive text feature extraction in the current text classification field. Moreover, there is also a certain improvement in the accuracy of text classification recognition, making the system more stable.

[0061] As Figure 2 shown, the text data acquisition layer includes a complaint text data acquisition module. Using the PyCharm editor and the third-party library of Python, it is realized to crawl the text data of the commodity complaints of the users on the JD.com platform from the Black Cat Complaint, a consumer service platform under Sina.

[0062] The text data storage layer includes a database server storage unit, which can use a local server or a cloud server to realize the real-time storage of the obtained complaint text data.

[0063] The text data preprocessing layer is used to preprocess the commodity complaint text data, thereby improving the credibility of the text information. Compared with some systems that directly perform text classification recognition on the original text information, after the present invention provides a more detailed text preprocessing method, the system stability and recognition accuracy and other aspects have been greatly improved.

[0064] The text data preprocessing layer includes a redundant information processing module, a text cleaning module, a normalization module, a sentence splitting and word segmentation module. The redundant information processing module removes data fields irrelevant to this task in the original text. The text cleaning module can be regarded as a denoising unit to denoise the text. Normalization refers to the correction of the text, that is, the correction of traditional Chinese characters and typos. Sentence splitting and word segmentation are to split the text at the sentence level and word level.

[0065] The text data feature extraction layer includes a stacked Transformer structure global feature extraction unit, a max pooling unit, a cross-layer attention unit, and a convolutional block attention module spatio-temporal feature extraction unit to obtain important features of the text. The output results of each unit are input into the feature fusion unit for splicing, and then the final classification result is output by the multi-layer perceptron unit and the output unit; the feature extraction module can learn the context information of the sentence. This module adopts a multi-layer Transformer structure. Generally speaking, when the number of layers is small, the model can learn syntactic and lexical information, etc., while when the number of layers is deep, it can learn the semantic features of the sentence. The word vectors obtained in this way combine various features of the text and context information, and the representation is more accurate. The algorithm combines a cross-layer attention mechanism and adopts a multi-layer Transformer structure. The max pooling operation is performed on the output of the Kth layer and the Attention operation is performed with the output of the Lth layer respectively. In addition, the application of a convolutional neural network is added to extract important words and combined information in the sentence, capture important local features, and this model also combines channel attention and spatial attention to improve the representation ability of the CNN. After the above steps are completed, feature splicing is performed, and the output results of multiple modules are spliced to obtain a more abundant representation of various global and local features. Then, it passes through the fully connected layer and the Softmax classifier in sequence, performs operations on all feature data, and identifies the current text classification.

[0066] The server module includes a database server unit for storing the original commodity complaint text data and the complaint text classification results.

[0067] The text classification information application layer classifies and stores the text data and the recognition results in the database server at the same time, and involves the distribution of information. The complaint information of the same type is distributed to the corresponding department handling this type of complaint for processing. In terms of application, an urgent complaint content expedited processing module can be added to set a priority dispatching processing mechanism for complaint content with a higher sensitivity level.

[0068] Embodiment 2

[0069] Taking the classification and recognition of commodity complaint texts as an example, the text content involved in different application scenarios is different, and the redundant information and noise therein are also different. In order to more accurately and reliably identify the categories in the text, it is necessary to perform special preprocessing operations on the original dataset to improve the credibility of the information. Finally, classification and recognition are carried out through the proposed text classification model.

[0070] A method for an intelligent classification system of complaint texts with multi-dimensional feature extraction based on a cross-layer attention mechanism, as Figure 4 shown, includes the following steps:

[0071] Step S1: Acquisition of complaint text data

[0072] From the consumer service platform under Sina - Black Cat Complaint, select the commodity complaint content of users on the JD platform for text data crawling; different platforms can be selected for data crawling according to practical scenarios and task requirements.

[0073] Step S2: Storage of text data

[0074] Store the obtained original text data in the local server or cloud server.

[0075] Step S3: Preprocessing of original text data

[0076] Preprocess the obtained original text data, specifically referring to: successively performing operations such as redundant information processing, text cleaning, standardization, sentence splitting, and word segmentation to obtain the preprocessed complaint text data for subsequent word embedding operations.

[0077] Step S4: Construction of a text pre-training model

[0078] Apply the GloVe technology to embed the preprocessed text information into word vectors so that each word and character can be expressed in the low - dimensional vector space.

[0079] Step S5: Construction and training of a text classification model

[0080] The text classification model, that is, the MDFECA network module, includes a stacked Transformer structure global feature extraction unit, a max - pooling unit, a cross - layer attention unit, a convolutional block attention module spatio - temporal feature extraction unit, a feature fusion unit, a multi - layer perceptron unit, and an output unit; input the data processed by the complaint text pre - training module into the text classifier in the trained complaint text classification module for classification and recognition, and output the classification and recognition results corresponding to the text content.

[0081] Step S6: Storage and application of text classification information

[0082] Through the text classification information storage application module, the results of complaint text classification are stored, and this information can be classified and delivered to departments for processing different types of complaint information.

[0083] In step S1, for data crawling, the PyCharm editor is used, and the urllib module in Python is utilized. By defining a crawler class, a crawler program is written to respectively implement functions such as automatic acquisition of comment URLs, data parsing, data formatting, and file saving. Eventually, various unstructured information on the web page is converted into semi-structured tables and stored in a local server or a cloud server for subsequent analysis and processing.

[0084] Step S3, the preprocessing of text data includes the following steps:

[0085] Step S31: Redundant information processing

[0086] The obtained original text data contains information of many fields, and a part of it is fields that are useless for text classification in this task, such as buyer ID, nickname, complaint initiation time, etc. After redundant information processing, this part of the content is removed.

[0087] Step S32: Text cleaning

[0088] The text cleaning stage involves a stop word library formulated specifically for complaint text content, and the library includes common stop words, emojis, pictures, special symbols, and Japanese emoticons, etc.

[0089] Step S33: Standardization

[0090] For the frequently appearing traditional Chinese characters in the complaint content, they need to be standardized to simplified Chinese characters, and the standardization is completed using the Opencc toolkit.

[0091] Step S34: Sentence splitting

[0092] The re module is introduced for sentence splitting processing, and re.split(·) is used to perform the sentence splitting operation, and the string to be split is set.

[0093] Step S35: Word segmentation

[0094] The Chinese word segmentation library Jieba is used to complete the Chinese word segmentation work, and after word segmentation, the completely preprocessed text data is obtained.

[0095] Step S4, in the NLP task, a sentence is usually regarded as a discrete token sequence, that is, words or characters. Sentence X can be regarded as consisting of m words, denoted as X = [x1, x2,..., x m , sentence X can be represented as V = [v1, v2,..., v n , where v iIt can be a one-hot vector, and the dimension length is equal to the number of different tokens N. Apply a pre-trained token embedding - GloVe to V, and all discrete tokens are converted into a low-dimensional dense vector representation sequence S = [s1, s2,..., s n , s i ∈R d . Here, s i represents the i-th token of the sentence S.

[0096] This preprocessing process can be written as an equation:

[0097] S = W (e) V

[0098] where V represents the sentence in vector form, W (e) represents the word embedding weight matrix, and S represents the low-dimensional dense vector representation of the sentence.

[0099] Step S5, constructing and training the text classification model is to use the word vectors obtained by applying the GloVe technology, input them into the text classifier of the constructed MDFECA network module for text classification output, including the following steps:

[0100] Step S51: Capturing global context information

[0101] To be able to learn the context information of the sentence, a multi-layer Transformer structure is adopted. The Transformer encoding unit consists of two parts: the self-attention mechanism and the feed-forward neural network. The input part of the self-attention mechanism is composed of three different vectors from the same word, namely Query, that is, the query vector Q; Key, that is, the key vector K; Value, that is, the value vector V. The similarity between the input part word vectors is represented by multiplying the Query vector and the Key vector, denoted as QK t , and is scaled by d k to ensure that the resulting result is of appropriate size. Finally, after the Softmax(·) activation function for normalization operation, a probability distribution is obtained, and then the weighted sum representation of all word vectors in the sentence is obtained. The word vectors obtained in this way combine the context information and are more accurate in representation. The calculation method is as follows:

[0102]

[0103] where d k is the dimension key_size of Q and K.

[0104] Step S52: Cross-layer attention mechanism

[0105] Adopt a multi-layer Transformer structure. K and L represent the number of layers respectively. The output result of the L-th layer is LT The information X1 is obtained through the max-pooling layer and then combined with the output result H of the K-th layer T to perform the Attention operation to obtain the corresponding weight values, and then continue to perform feature scaling on H T to obtain the information X2. The calculation formula is as follows:

[0106] X1 = MaxPooling(L T )

[0107] X2 = Softmax(tanh(W (1) X1 + W (2) H T )) * H T

[0108] where H T is the output of the K-th layer, W (1) is the conversion weight of X1, and W (2) is the weight of the features of H T .

[0109] Step S53: Capture important local features

[0110] The application of convolutional neural networks in images is very common and can effectively capture important regions in pictures. Similarly, applying convolutional neural networks in text can extract important words and combined information in sentences. Channel attention and spatial attention are combined to improve the representation ability of CNNs

[0111] The text features output by the convolutional layer are F. Using the CBAM module, the channel attention M c (F) and the spatial attention M s (F) are derived in sequence; the overall attention process can be summarized as: M c (F) performs the Attention operation in the C dimension, and then multiplies the feature weights of Attention and F in matrix form to obtain F 1 , and then performs the Attention operation with the spatial attention in the H and W dimensions. Similarly, the F 2 feature information is obtained, where F 2 is X3 in the figure:

[0112] F ∈ R C*H*W , M c ∈ R C*1*1 , M s ∈ R 1*H*W

[0113]

[0114] Channel attention is used to generate a channel attention map by leveraging the channel relationship between features. Since each channel of the feature map is considered a feature detector, the attention of the channel focuses on "which" words in the given input text are meaningful;

[0115] Spatial attention is used to generate a spatial attention map by leveraging the spatial relationship between features. Different from the channel attention module, the spatial attention module focuses on "where" the important information is, as a supplement to the channel attention module.

[0116] Step S54: Feature concatenation

[0117] According to steps S52 and S53, the feature outputs X1, X2, and X3 are obtained respectively. In previous experience, in the Transformer structure for NLP tasks, the model can often learn syntactic and lexical information when the number of layers is small, and can learn the semantic features of sentences when the number of layers is deep. X1 obtains semantic features in this model. X2 is obtained by combining syntactic and lexical features, and X3 obtains the features of the local lexical importance of the sentence through a convolutional neural network. This module concatenates the three types of features, and the formula is as follows:

[0118]

[0119] Step S55: Fully connected layer

[0120] The feature fusion unit is used to merge the obtained important local features and global features to obtain the complaint text information features, that is, to concatenate the output features of the max pooling unit, the cross-layer attention unit, and the spatio-temporal feature extraction unit of the convolutional block attention module. Here, a two-layer multi-layer perceptron (MLP) structure is adopted to convert the information X into the final label category probability. First, through two fully connected layers, using the ReLU(·) function as the activation function, h2 is obtained. Finally, through the classification layer and the Softmax(·) activation function, the probability of each label is obtained. Select the subscript with the highest probability, which is the classification result of the sentence, and the formula is as follows:

[0121] h1 = ReLU(W hidden1 X + b hidden1 )

[0122] h2 = ReLU(W hidden2 h1 + b hidden2 )

[0123] logit = ReLU(W output h2 + b output )

[0124] label_id = Argmax(Softmax(logit)).

[0125] There are three layers of linear transformation involved here, where W is the weight of the hidden layer and b is the offset of the hidden layer. The final prediction result logit is obtained. The features are normalized through the Softmax(·) function, and then the subscript of the label with the highest probability is queried through the Argmax(·) function to obtain label_id, and the final classification result is queried according to the mapping file.

[0126] Embodiment 3

[0127] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the method for an intelligent classification system of multi-dimensional feature extraction complaint text based on a cross-layer attention mechanism in Embodiment 2.

[0128] Therefore, the present invention adopts the above-mentioned method, system and storage medium for intelligent classification of multi-dimensional feature extraction complaint text based on a cross-layer attention mechanism, and uses a more detailed text preprocessing method to improve the credibility of text information, thereby improving the final classification effect of the text, being able to extract various aspects of features of the text more comprehensively, making the system have strong robustness, and can be used for enterprises to timely supervise customer complaints, reduce the labor input in complaint handling and improve the efficiency of complaint handling. It can also be used in fields such as smart home and smart healthcare involving text classification.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that they can still modify or equivalently replace the technical solutions of the present invention, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. An intelligent classification system for complaint texts with multi-dimensional feature extraction based on a cross-layer attention mechanism, characterized in that: It includes a text data acquisition layer, a text data storage layer, a text data preprocessing layer, a word embedding layer, a text data feature extraction layer, and a text classification information application layer that are connected in sequence; The text data acquisition layer uses a crawler program to acquire the required complaint text data; The text data storage layer stores the original commodity complaint text data collected in a local server or a cloud server; The word embedding layer applies the GloVe technology to embed the preprocessed text information into word vectors, enabling each word and character to be expressed in a low-dimensional vector space, which is convenient for subsequent classifiers to extract features; The text data feature extraction layer includes an MDFECA network module, and the MDFECA network module includes a stacked Transformer structure global feature extraction unit, a max pooling unit, a cross-layer attention unit, a convolutional block attention module spatio-temporal feature extraction unit, a feature fusion unit, a multi-layer perceptron unit, and an output unit.

2. The intelligent classification system for complaint texts with multi-dimensional feature extraction based on a cross-layer attention mechanism according to claim 1, characterized in that, The text data preprocessing layer includes preprocessing the crawled original text data to obtain preprocessed complaint text information.

3. The intelligent classification system for complaint texts with multi-dimensional feature extraction based on a cross-layer attention mechanism according to claim 1, characterized in that: The text classification information application layer classifies and stores the text data and the recognition results in a database server at the same time, and distributes the complaint information of the same type to the corresponding departments for processing.

4. A method for the intelligent classification system for complaint texts with multi-dimensional feature extraction based on a cross-layer attention mechanism according to any one of claims 1-3, characterized in that, It includes the following steps: Step S1: Complaint text data acquisition. Select a user on the consumer service platform to crawl text data of the complaint content; Step S2: Text data storage. Store the obtained original text data in a local server or a cloud server; Step S3: Original text data preprocessing. Preprocess the obtained original text data to obtain preprocessed complaint text data, which is convenient for subsequent word embedding operations; Step S4: Construct a text pre-training model. Apply the GloVe technology to embed the preprocessed text information into word vectors, enabling each word and character to be expressed in a low-dimensional vector space; Step S5: Construct and train a text classification model. Input the data processed by the complaint text pre-training module into the text classifier in the trained complaint text classification module for classification and recognition, and output the classification recognition result corresponding to the text content; Step S6: Text classification information storage and application. Through the text classification information storage and application module, store the results of complaint text classification, and the information can be classified and delivered to the departments for processing different types of complaint information.

5. The method for the intelligent classification system for complaint texts with multi-dimensional feature extraction based on a cross-layer attention mechanism according to claim 4, characterized in that: In step S1, the data crawling uses the PyCharm editor, utilizes the urllib module in Python, and writes a crawler program by defining a crawler class to convert various unstructured information on the web page into semi-structured tables and store them in a local server or a cloud server for subsequent analysis and processing.

6. The method for the intelligent classification system for complaint texts with multi-dimensional feature extraction based on a cross-layer attention mechanism according to claim 4, characterized in that: In step S3, the text data preprocessing includes the following steps: Step S31: Redundant information processing. The obtained original text data contains information in many fields, and a part of it is fields that are useless for text classification in this task. After redundant information processing, this part of the content is removed; Step S32: Text cleaning. The text cleaning stage involves a stop word library specifically formulated for complaint text content; Step S33: Standardization. For the traditional Chinese characters that often appear in the complaint content, they need to be standardized into simplified Chinese characters, and the standardization is completed using the Opencc toolkit; Step S34: Sentence splitting. Introduce the re module for sentence splitting processing, use re.split(·) to perform the sentence splitting operation, and set the string to be split; Step S35: Word segmentation. Use the Jieba Chinese word segmentation library to complete the Chinese word segmentation work. After the word segmentation is completed, the fully preprocessed text data is obtained.

7. The method for the intelligent classification system for complaint texts with multi-dimensional feature extraction based on a cross-layer attention mechanism according to claim 4, characterized in that: Step S4, the text pre-training model is: S = W (e) V Wherein, V represents the form of converting a sentence into a vector, and W (e) represents the word embedding weight matrix, and S represents the low-dimensional dense vector representation of the sentence.

8. The method for the intelligent classification system for complaint texts with multi-dimensional feature extraction based on a cross-layer attention mechanism according to claim 4, characterized in that: Step S5. The construction and training of the text classification model is to obtain word vectors by applying the GloVe technology, and input them into the text classifier of the constructed MDFECA network module for text classification output, including the following steps: Step S51: Capture global context information; Step S52: Cross-layer attention mechanism; Step S53: Capture important local features; Step S54: Feature concatenation; Step S55: Fully connected layer.

9. A computer device, comprising a memory and a processor, characterized in that: The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the method for an intelligent classification system of complaint texts with multi-dimensional feature extraction based on a cross-layer attention mechanism according to any one of claims 4-8.

Images