Method and system for constructing text classification corpus

By combining deep learning and big data algorithms, multi-scale correlation features are constructed using web page content and topic data, and text classification is automatically performed, which solves the problems of high labor costs and subjective factors in text classification by traditional search engines, and improves corpus construction accuracy and efficiency.

CN117349399BActive Publication Date: 2024-11-01SHENZHEN LEHAOYANG TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311349292.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-18
Publication Date
2024-11-01
Estimated Expiration
2043-10-18

AI Technical Summary

Technical Problem

Traditional search engines require a lot of labor costs and time when classifying texts, and are easily affected by human subjective factors, making it difficult to adapt to changes in classification methods.

Method used

Using artificial intelligence technology based on deep learning and big data algorithms, we can obtain label data of web page content data and web page theme data, extract multi-scale correlation features, build a corpus, and use the global correlation of user-provided text data in high-dimensional space. Query the features and automatically classify the topic.

Benefits of technology

It reduces the influence of human factors, improves the accuracy and efficiency of corpus construction, and reduces the difficulty of adapting to changes in classification methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117349399B_ABST
    Figure CN117349399B_ABST
Patent Text Reader

Abstract

The present application relates to the field of intelligent text classification, and specifically discloses a method and system for constructing a text classification corpus, which uses web page content data and label data of web page topic data as reference data for corpus construction, and extracts the globally dynamically changing multi-scale correlation features of each corpus construction reference data, and then uses this to construct a data feature library between the multi-scale web page data correlation features in the corpus construction reference data and the labels of the web page topics. The global correlation features of the text data provided by the user in the high-dimensional space are then used as query features to query the label data features of the text topic from the data feature library, and then output the corpus according to the topic. In this way, the artificial intelligence technology based on deep learning is combined with the big data algorithm, and the text to be classified is automatically classified by topic to intelligently construct a text corpus, which reduces the influence of human factors and improves the accuracy of corpus construction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent text classification, and more specifically to a method and system for constructing a text classification corpus. Background Art

[0002] The amount of information on the Internet is huge and growing. Search engines make it easier for users to find what they need by indexing and organizing this information. Search engines provide a simple and easy-to-use interface where users can search for relevant information by entering keywords or phrases. Search engines return relevant results based on the user's query to help users quickly find what they need. However, because the data to index web pages is too large, traditional search engines often search based on user-specified categories. For traditional search engines, they often use manual methods to annotate training corpus based on the associated words entered by users to train classifiers based on the classification methods defined in advance by experts. Traditional annotation methods require a lot of labor and time costs, are easily affected by human subjective factors, and need to be reclassified according to changes in classification methods.

[0003] Therefore, an optimized method for constructing a text classification corpus is expected. Summary of the invention

[0004] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a method and system for constructing a text classification corpus, which uses web page content data and label data of web page topic data as corpus construction reference data, and extracts the globally dynamically changing multi-scale correlation features of each corpus construction reference data, and then uses this to construct a data feature library between the multi-scale web page data correlation features in the corpus construction reference data and the labels of the web page topics. Then, the global correlation features of the text data provided by the user in the high-dimensional space are used as query features to query the label data features of the topic of the text from the data feature library, and then output the corpus according to the topic. In this way, the artificial intelligence technology based on deep learning is combined with the big data algorithm, and the classified text is automatically classified by topic to intelligently construct the text corpus, which reduces the influence of human factors and improves the accuracy of corpus construction.

[0005] According to one aspect of the present application, a method for constructing a text classification corpus is provided, which comprises:

[0006] Get text data provided by the user;

[0007] Obtaining web page topic tag data and web page content data;

[0008] Passing the user-provided text data through a transformer-based context encoder including an embedding layer to obtain a query feature vector;

[0009] Passing the tag data of the webpage topic and the webpage content data through a converter-based context encoder including an embedding layer to obtain a plurality of webpage feature vectors;

[0010] Arranging the plurality of web page feature vectors in two dimensions into a web page feature matrix and then passing the matrix through a convolutional neural network model including a plurality of hybrid convolutional layers to obtain a multi-scale web page feature matrix;

[0011] Multiplying the query feature vector by the multi-scale web page feature matrix to obtain a classification feature vector;

[0012] Performing order priori on the classification feature vector to obtain an optimized classification feature matrix;

[0013] Passing the optimized classification feature matrix through a classifier to obtain a classification result, wherein the classification result is used to represent a label of the subject of the text;

[0014] A corpus is output based on the classification result.

[0015] In the above-mentioned method for constructing a text classification corpus, the text data provided by the user is passed through a transformer-based context encoder including an embedding layer to obtain a query feature vector, including:

[0016] Performing word segmentation processing on the text data provided by the user to obtain a plurality of user words;

[0017] Passing the multiple user words through an embedding layer to convert each user word in the multiple user words into a user word embedding vector to obtain a sequence of user word embedding vectors, wherein the embedding layer uses a learnable embedding matrix to embed the each user word;

[0018] Inputting the sequence of user word embedding vectors into the transformer-based context encoder to obtain the plurality of user feature vectors;

[0019] The multiple user feature vectors are cascaded to obtain the query feature vector.

[0020] 3. The method for constructing a text classification corpus according to claim 2, characterized in that the sequence of user word embedding vectors is input into the transformer-based context encoder to obtain the plurality of user feature vectors, comprising:

[0021] Arranging the sequence of user word embedding vectors as an input vector;

[0022] Convert the input vector into a query vector and a key vector respectively through a learnable embedding matrix;

[0023] Calculating the product between the query vector and the transposed vector of the key vector to obtain a self-attention association matrix;

[0024] Normalizing the self-attention association matrix to obtain a standardized self-attention association matrix;

[0025] Input the standardized self-attention association matrix into a Softmax activation function for activation to obtain a self-attention feature matrix;

[0026] The self-attention feature matrix is ​​multiplied by each user word embedding vector in the sequence of user word embedding vectors as a value vector to obtain the multiple user feature vectors.

[0027] In the above-mentioned method for constructing a text classification corpus, the label data of the web page topic and the web page content data are passed through a transformer-based context encoder including an embedding layer to obtain a plurality of web page feature vectors, including:

[0028] One-hot encoding the label data of the webpage topic and the webpage content data through the embedding layer of the converter-based context encoder respectively, so as to convert the label data of the webpage topic and the webpage content data into input vectors respectively to obtain a sequence of input vectors;

[0029] The sequence of input vectors is encoded based on global context semantics using a converter of the converter-based context encoder to obtain the plurality of web page feature vectors.

[0030] In the above-mentioned method for constructing a text classification corpus, using the converter of the converter-based context encoder to perform global context semantic encoding on the sequence of the input vectors to obtain the multiple web page feature vectors includes:

[0031] Calculating the product of the sequence of the input vectors and the transposed vector of each input vector in the sequence of the input vectors to obtain a plurality of self-attention association matrices;

[0032] Normalizing each of the plurality of self-attention association matrices to obtain a plurality of standardized self-attention association matrices;

[0033] Passing each of the plurality of standardized self-attention association matrices through a Softmax classification function to obtain a plurality of probability values;

[0034] Each input vector in the sequence of input vectors is weighted using each probability value in the multiple probability values ​​as a weight to obtain the multiple web page feature vectors.

[0035] In the above-mentioned method for constructing a text classification corpus, the plurality of web page feature vectors are arranged two-dimensionally into a web page feature matrix and then passed through a convolutional neural network model including a plurality of hybrid convolutional layers to obtain a multi-scale web page feature matrix, including:

[0036] Each mixed convolutional layer of the convolutional neural network model is used to perform the following operations on the input data in the forward pass of the layer:

[0037] Performing convolution processing on the input data based on a first convolution kernel to obtain a first scale feature map;

[0038] Performing convolution processing on the input data based on a second convolution kernel to obtain a second scale feature map, wherein the second convolution kernel is a dilated convolution kernel with a first dilation rate;

[0039] Performing convolution processing on the input data based on a third convolution kernel to obtain a third scale feature map, wherein the third convolution kernel is a dilated convolution kernel with a second dilation rate;

[0040] Performing convolution processing on the input data based on a fourth convolution kernel to obtain a fourth scale feature map, wherein the fourth convolution kernel is a dilated convolution kernel with a third dilation rate;

[0041] Cascading the first scale feature map, the second scale feature map, the third scale feature map, and the fourth scale feature map to obtain a multi-scale feature map;

[0042] Performing pooling processing on the multi-scale feature map to obtain a pooled feature map;

[0043] Performing activation processing on the pooled feature map to obtain an activated feature map;

[0044] Among them, the output of the last hybrid convolutional layer of the convolutional neural network model is the multi-scale web page feature matrix.

[0045] In the above-mentioned method for constructing a text classification corpus, the first scale feature map, the second scale feature map, the third scale feature map and the fourth scale feature map are cascaded to obtain a multi-scale feature map, including:

[0046] Cascading the first scale feature map, the second scale feature map, the third scale feature map, and the fourth scale feature map using the following cascade formula to obtain a multi-scale feature map;

[0047] Wherein, the cascade formula is:

[0048] X=Concat[F1,F2,F3,F4]

[0049] Among them, F1, F2, F3 and F4 respectively represent the first scale feature map, the second scale feature map, the third scale feature map and the fourth scale feature map, and Concat[·,·] represents a cascade function.

[0050] In the above-mentioned method for constructing a text classification corpus, the classification feature vector is ordered a priori to obtain an optimized classification feature matrix, including:

[0051] Taking Gaussian distribution as a prior distribution, enhancing the feature expression of the classification feature vector based on the prior distribution to obtain a Gaussian enhanced classification feature matrix;

[0052] Calculating the mutual information between each row vector in the Gaussian enhanced classification feature matrix and the classification feature vector to obtain multiple mutual information;

[0053] Obtaining an order priori mask matrix based on a comparison between the plurality of mutual information and a predetermined threshold;

[0054] The point-wise multiplication between the order priori mask matrix and the Gaussian enhanced classification feature matrix is ​​calculated to obtain an optimized classification feature matrix.

[0055] According to another aspect of the present application, a system for constructing a text classification corpus is also provided, comprising:

[0056] A text data acquisition module, used to acquire text data provided by a user;

[0057] A web page data acquisition module is used to acquire the tag data of the web page theme and the web page content data;

[0058] A text encoding module, configured to pass the user-provided text data through a transformer-based context encoder including an embedding layer to obtain a query feature vector;

[0059] A web page encoding module, used for passing the tag data of the web page theme and the web page content data through a converter-based context encoder including an embedding layer to obtain a plurality of web page feature vectors;

[0060] A hybrid convolution module, used for arranging the plurality of web page feature vectors in two dimensions into a web page feature matrix and then passing the matrix through a convolutional neural network model including a plurality of hybrid convolution layers to obtain a multi-scale web page feature matrix;

[0061] An associative coding module, used for multiplying the query feature vector with the multi-scale web page feature matrix to obtain a classification feature vector;

[0062] An optimization module, used for performing order priori on the classification feature vector to obtain an optimized classification feature matrix;

[0063] A topic generation module, used for passing the optimized classification feature matrix through a classifier to obtain a classification result, wherein the classification result is used to represent a label of the topic of the text;

[0064] The corpus generation module is used to output a corpus based on the classification result.

[0065] Compared with the prior art, the method and system for constructing a text classification corpus provided by the present application use web page content data and label data of web page topic data as reference data for corpus construction, and extract the globally dynamically changing multi-scale correlation features of each corpus construction reference data, and then use this to construct a data feature library between the multi-scale web page data correlation features in the corpus construction reference data and the labels of the web page topics. Then, the global correlation features of the text data provided by the user in the high-dimensional space are used as query features to query the label data features of the text topic from the data feature library, and then output the corpus according to the topic. In this way, the artificial intelligence technology based on deep learning is combined with the big data algorithm to automatically classify the classified text into topics to intelligently construct the text corpus, reduce the influence of human factors, and improve the accuracy of corpus construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0067] Figure 1 The figure illustrates a flow chart of a method for constructing a text classification corpus according to an embodiment of the present application.

[0068] Figure 2 The diagram illustrates an architecture diagram of a method for constructing a text classification corpus according to an embodiment of the present application.

[0069] Figure 3 The diagram illustrates a flow chart of a method for constructing a text classification corpus according to an embodiment of the present application, in which the multiple web page feature vectors are two-dimensionally arranged into a web page feature matrix and then passed through a convolutional neural network model including multiple mixed convolutional layers to obtain a multi-scale web page feature matrix.

[0070] Figure 4 The figure illustrates a block diagram of a system for constructing a text classification corpus according to an embodiment of the present application.

[0071] Figure 5 A block diagram of an electronic device according to an embodiment of the present application is illustrated. DETAILED DESCRIPTION

[0072] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0073] Exemplary Methods

[0074] Figure 1 Flow chart of a method for constructing a text classification corpus according to an embodiment of the present application. Figure 1 As shown, according to the method for constructing a text classification corpus of an embodiment of the present application, the method includes: S110, obtaining text data provided by a user; S120, obtaining label data of a web page topic and web page content data; S130, passing the text data provided by the user through a converter-based context encoder including an embedding layer to obtain a query feature vector; S140, passing the label data of the web page topic and web page content data through a converter-based context encoder including an embedding layer to obtain multiple web page feature vectors; S150, arranging the multiple web page feature vectors in two dimensions into a web page feature matrix and then passing them through a convolutional neural network model including multiple hybrid convolutional layers to obtain a multi-scale web page feature matrix; S160, multiplying the query feature vector with the multi-scale web page feature matrix to obtain a classification feature vector; S170, ordering the classification feature vector a priori to obtain an optimized classification feature matrix; S180, passing the optimized classification feature matrix through a classifier to obtain a classification result, and the classification result is used to represent the label of the text topic; and S190, outputting a corpus based on the classification result.

[0075] Figure 2 FIG. 1 is an architecture diagram of a method for constructing a text classification corpus according to an embodiment of the present application. Figure 2As shown, in this architecture, first, the text data provided by the user is obtained. Then, the label data of the web page topic and the web page content data are obtained. Then, the text data provided by the user is passed through a converter-based context encoder including an embedding layer to obtain a query feature vector. Then, the label data of the web page topic and the web page content data are passed through a converter-based context encoder including an embedding layer to obtain multiple web page feature vectors. Then, after the multiple web page feature vectors are arranged in two dimensions into a web page feature matrix, they are passed through a convolutional neural network model including multiple mixed convolution layers to obtain a multi-scale web page feature matrix. Then, the query feature vector is multiplied by the multi-scale web page feature matrix to obtain a classification feature vector. Then, the classification feature vector is order-prioritized to obtain an optimized classification feature matrix. Then, the optimized classification feature matrix is ​​passed through a classifier to obtain a classification result, and the classification result is used to represent the label of the text topic. Then, a corpus is output based on the classification result.

[0076] In step S110, the text data provided by the user is obtained. As mentioned in the above background technology, the search engine will return relevant results based on the user's query to help users quickly find the content they need. However, because the data of the web pages to be indexed is too large, traditional search engines often search according to the classification specified by the user. For traditional search engines, it is often based on the classification method defined in advance by experts, and the training corpus is manually annotated according to the associated words input by the user to train the classifier. The traditional annotation method requires a lot of labor cost and time cost, is easily affected by human subjective factors, and needs to be reclassified according to the change of the classification method. Therefore, a method for constructing an optimized text classification corpus is expected.

[0077] In response to the above technical problems, a text classification corpus construction scheme is proposed, which uses web page content data and label data of web page topic data as corpus construction reference data, and extracts the globally dynamic multi-scale correlation features of each corpus construction reference data, and then uses this to construct a data feature library between the multi-scale web page data correlation features in the corpus construction reference data and the labels of the web page topics. The global correlation features of the text data provided by the user in the high-dimensional space are then used as query features to query the label data features of the text topic from the data feature library, and then output the corpus according to the topic. In this way, the artificial intelligence technology based on deep learning is combined with the big data algorithm to automatically perform topic classification on the classified text to intelligently construct the text corpus, reduce the influence of human factors, and improve the accuracy of corpus construction.

[0078] At present, deep learning and neural networks have been widely used in computer vision, natural language processing, speech signal processing and other fields. In addition, deep learning and neural networks have also shown a level close to or even beyond that of humans in image classification, object detection, semantic segmentation, text translation and other fields.

[0079] In recent years, the development of deep learning and neural networks has provided new solutions and solutions for the construction of text classification corpora.

[0080] Specifically, first, the text data provided by the user is obtained. The text data provided by the user may contain content of a specific field, a specific topic or personal interest.

[0081] In step S120, the tag data of the web page topic and the web page content data are obtained. Obtaining the tag data of the web page topic and the web page content data can increase the data diversity of the corpus. Web pages cover content of various topics and fields, which often contain rich contextual information, including titles, abstracts, text, etc., which can provide more comprehensive and accurate features. The tag data of the web page topic can provide prior knowledge about the web page content topic. By obtaining the tag data of the web page topic, the web page can be associated with a specific topic, thereby better performing text classification.

[0082] In step S130, the text data provided by the user is passed through a converter-based context encoder including an embedding layer to obtain a query feature vector. In order to convert the text data into a vector form with semantic representation, so as to perform subsequent semantic matching and text classification tasks, the text data is converted into a query feature vector to capture the semantic information of the text data. The converter model (such as BERT, GPT, etc.) has a strong semantic understanding ability, which can map the text data into a high-dimensional space and capture the semantic association between words. The query feature vector obtained can better represent the semantic meaning of the text, which is helpful for subsequent text classification tasks. The embedding layer can convert the text data into a continuous vector representation. The embedding layer is a learned mapping function that can convert discrete words or characters into a continuous vector representation. Such a continuous representation is more suitable for calculation and comparison, and can better capture the semantic similarity between words. The converter-based context encoder can perform context modeling on the text data. It will consider the context in which the words appear, so as to better understand the meaning of the words. Through the context encoder, the model can use the local and global context information of the text data to obtain a more accurate and rich query feature vector.

[0083] Specifically, in the above-mentioned method for constructing a text classification corpus, the text data provided by the user is passed through a converter-based context encoder including an embedding layer to obtain a query feature vector, including: performing word segmentation processing on the text data provided by the user to obtain a plurality of user words; passing the plurality of user words through an embedding layer to convert each of the plurality of user words into a user word embedding vector to obtain a sequence of user word embedding vectors, wherein the embedding layer uses a learnable embedding matrix to perform embedding encoding on each user word; inputting the sequence of user word embedding vectors into the converter-based context encoder to obtain the plurality of user feature vectors; and, cascading the plurality of user feature vectors to obtain the query feature vector.

[0084] Specifically, in the above-mentioned method for constructing a text classification corpus, the sequence of user word embedding vectors is input into the converter-based context encoder to obtain the multiple user feature vectors, including: arranging the sequence of user word embedding vectors as input vectors; converting the input vectors into query vectors and key vectors respectively through a learnable embedding matrix; calculating the product between the query vector and the transposed vector of the key vector to obtain a self-attention association matrix; normalizing the self-attention association matrix to obtain a standardized self-attention association matrix; inputting the standardized self-attention association matrix into a Softmax activation function for activation to obtain a self-attention feature matrix; and, multiplying the self-attention feature matrix with each user word embedding vector in the sequence of user word embedding vectors as a value vector to obtain the multiple user feature vectors.

[0085] In step S140, the tag data of the web page topic and the web page content data are passed through a converter-based context encoder including an embedding layer to obtain multiple web page feature vectors. In order to convert the web page data into a vector form with semantic representation and to capture the semantic similarities and differences between different web pages, the web page data is converted into a feature vector to capture the semantic information of the web page. The converter-based context encoder can perform context modeling on the web page data. It considers the context in which the web page tags and content appear, so as to better understand the meaning of the web page. Through the context encoder, the model can use the local and global context information of the web page data to obtain more accurate and rich web page feature vectors.

[0086] Specifically, in the above-mentioned method for constructing a text classification corpus, the label data of the web page topic and the web page content data are passed through a converter-based context encoder including an embedding layer to obtain multiple web page feature vectors, including: respectively passing the label data of the web page topic and the web page content data through the embedding layer of the converter-based context encoder to perform one-hot encoding to convert the label data of the web page topic and the web page content data into input vectors respectively to obtain a sequence of input vectors; using the converter of the converter-based context encoder to perform global context semantic encoding on the sequence of input vectors to obtain the multiple web page feature vectors.

[0087] Specifically, in the above-mentioned method for constructing a text classification corpus, the converter of the converter-based context encoder is used to perform global context semantic encoding on the sequence of input vectors to obtain the multiple web page feature vectors, including: calculating the product between the sequence of input vectors and the transposed vectors of each input vector in the sequence of input vectors to obtain multiple self-attention association matrices; standardizing each of the multiple self-attention association matrices to obtain multiple standardized self-attention association matrices; passing each of the multiple standardized self-attention association matrices through a Softmax classification function to obtain multiple probability values; and weighting each input vector in the sequence of input vectors using each of the multiple probability values ​​as a weight to obtain the multiple web page feature vectors.

[0088] In step S150, the multiple web page feature vectors are arranged two-dimensionally into a web page feature matrix and then passed through a convolutional neural network model including multiple hybrid convolutional layers to obtain a multi-scale web page feature matrix. In order to extract web page features from different scales, capture local and global information of web pages, and enrich the representation ability of web page features, the multiple web page feature vectors are arranged two-dimensionally into a web page feature matrix and then passed through a convolutional neural network model including multiple hybrid convolutional layers to obtain a multi-scale web page feature matrix. By using multiple hybrid convolutional layers, web page features can be extracted at different scales. The hybrid convolutional layer has convolution kernels of different sizes and can capture local features of different sizes. Through multi-scale convolution operations, the details and overall information of the web page can be considered at the same time, thereby more comprehensively describing the features of the web page. By stacking multiple hybrid convolutional layers, a multi-layer convolutional neural network model can be formed. The convolution operation of each layer can be regarded as an abstraction and combination of different levels of features. The lower-level convolutional layer can extract low-level local features, while the higher-level convolutional layer can extract more abstract global features. This hierarchical feature representation can better capture the semantic information and structural features of the web page. Through the convolutional neural network model, the ability to represent features can be improved by learning weights and parameters. The weights in the convolutional layer can automatically learn the feature representation method suitable for the task, thereby improving the discriminative and expressive ability of web page features. The multi-scale web page feature matrix can better represent the diversity and complexity of web pages.

[0089] Figure 3 The figure shows a flowchart of the method for constructing a text classification corpus according to an embodiment of the present application, in which the plurality of web page feature vectors are arranged two-dimensionally into a web page feature matrix and then passed through a convolutional neural network model including a plurality of hybrid convolutional layers to obtain a multi-scale web page feature matrix. Figure 3As shown, after the multiple web page feature vectors are two-dimensionally arranged into a web page feature matrix, a convolutional neural network model including multiple mixed convolutional layers is used to obtain a multi-scale web page feature matrix, including: using each mixed convolutional layer of the convolutional neural network model to perform the following on the input data in the forward pass of the layer: S151, convolution processing is performed on the input data based on a first convolution kernel to obtain a first scale feature map; S152, convolution processing is performed on the input data based on a second convolution kernel to obtain a second scale feature map, wherein the second convolution kernel is a hollow convolution kernel with a first hollow rate; S153, convolution processing is performed on the input data based on a third convolution kernel to obtain a third scale feature map, wherein the The third convolution kernel is a dilated convolution kernel with a second dilation rate; S154, performing convolution processing on the input data based on the fourth convolution kernel to obtain a fourth-scale feature map, wherein the fourth convolution kernel is a dilated convolution kernel with a third dilation rate; S155, cascading the first-scale feature map, the second-scale feature map, the third-scale feature map and the fourth-scale feature map to obtain a multi-scale feature map; S156, performing pooling processing on the multi-scale feature map to obtain a pooled feature map; S157, performing activation processing on the pooled feature map to obtain an activated feature map; wherein, the output of the last mixed convolution layer of the convolutional neural network model is the multi-scale web page feature matrix.

[0090] Specifically, in the above-mentioned method for constructing a text classification corpus, the first scale feature map, the second scale feature map, the third scale feature map and the fourth scale feature map are cascaded to obtain a multi-scale feature map, including: the first scale feature map, the second scale feature map, the third scale feature map and the fourth scale feature map are cascaded according to the following cascade formula to obtain a multi-scale feature map; wherein the cascade formula is:

[0091] X=Concat[F1,F2,F3,F4]

[0092] Among them, F1, F2, F3 and F4 respectively represent the first scale feature map, the second scale feature map, the third scale feature map and the fourth scale feature map, and Concat[·,·] represents a cascade function.

[0093] In step S160, the query feature vector is multiplied by the multiscale web page feature matrix to obtain a classification feature vector. In order to fuse the query feature with the web page feature, the query feature vector is multiplied by the multiscale web page feature matrix. By multiplying the query feature vector with the multiscale web page feature matrix, the query feature can be multiplied element by element with each web page feature, thereby fusing the query feature with the web page feature. This fusion can combine the important information of the query feature with the semantic information of the web page feature to obtain a more representative classification feature vector. By multiplying the query feature vector with the web page feature matrix, the correlation between the query feature and each web page feature can be modeled. The multiplication operation can associate each dimension of the query feature with the corresponding dimension of the web page feature, thereby capturing the correlation between the query feature and the web page feature. This helps to better understand the semantic relationship between the query and the web page.

[0094] In step S170, the classification feature vector is ordered prior to obtain an optimized classification feature matrix. In the technical solution of the present application, considering that the classification feature vector as a feature set itself satisfies a predetermined prior distribution, the feature expression can be enhanced based on the prior information of the feature distribution, that is, with Gaussian distribution as the prior distribution, the classification feature vector is enhanced based on the prior distribution to obtain a Gaussian enhanced classification feature matrix. However, the classification feature vector is obtained from the original data, and the original data will inevitably introduce noise during data collection. In the process of feature learning using a deep neural network model, noise information will inevitably be learned. Therefore, when the classification feature vector is enhanced by using the prior distribution, the noise information will also be amplified synchronously, affecting the accuracy of the classification result of the classification feature vector through the classifier.

[0095] In view of the above technical problems, in the technical solution of the present application, considering that if two feature vectors have a predetermined similarity in the original data space, then after feature enhancement or feature transformation, the similarity between the two is still retained to a certain extent, based on this, the mutual information between each row vector in the Gaussian enhanced classification feature matrix and the classification feature vector is first calculated to obtain multiple mutual information, and the mutual information is used to represent the similarity between the enhanced feature expression and the original feature expression. Furthermore, based on the comparison between the multiple mutual information and the predetermined threshold, the order priori mask matrix is ​​obtained. Specifically, if the i-th mutual information is greater than or equal to the predetermined threshold, the vector of the i-th row of the order priori mask matrix is ​​[1,1,.....1]; if the i-th mutual information is less than the predetermined threshold, the vector of the i-th row of the order priori mask matrix is ​​[0,0,.....0]. Next, the point-by-point multiplication between the order prior mask matrix and the Gaussian enhanced classification feature matrix is ​​calculated to obtain the optimized classification feature matrix. This is essentially based on the similarity between the enhanced feature expression and the original feature representation to perform feature masking screening to eliminate noise information, thereby improving the signal-to-noise ratio of the information expression of the feature distribution, so as to improve the accuracy of the classification judgment of the optimized classification feature matrix through the classifier.

[0096] Specifically, in the above-mentioned method for constructing a text classification corpus, the classification feature vector is ordered prior to obtain an optimized classification feature matrix, including: taking Gaussian distribution as the prior distribution, enhancing the feature expression of the classification feature vector based on the prior distribution to obtain a Gaussian enhanced classification feature matrix; calculating the mutual information between each row vector in the Gaussian enhanced classification feature matrix and the classification feature vector to obtain multiple mutual information; obtaining an order prior mask matrix based on the comparison between the multiple mutual information and a predetermined threshold; calculating the position point multiplication between the order prior mask matrix and the Gaussian enhanced classification feature matrix to obtain an optimized classification feature matrix.

[0097] Specifically, in the above-mentioned method for constructing a text classification corpus, an order priori mask matrix is ​​obtained based on the comparison between the multiple mutual information and a predetermined threshold. If the i-th mutual information is greater than or equal to the predetermined threshold, the vector of the i-th row of the order prior mask matrix is ​​[1,1,.....1]; if the i-th mutual information is less than the predetermined threshold, the vector of the i-th row of the order prior mask matrix is ​​[0,0,.....0].

[0098] In step S180, the optimized classification feature matrix is ​​passed through a classifier to obtain a classification result, and the classification result is used to represent the label of the subject of the text. By using a classifier, a classification model can be constructed based on the features and label information in the training data. The model can map the input classification feature vector to different categories, thereby realizing automatic classification of the text.

[0099] Specifically, in the above-mentioned method for constructing a text classification corpus, the optimized classification feature matrix is ​​passed through a classifier to obtain a classification result, and the classification result is used to represent the label of the subject of the text, including: using the classifier to process the optimized classification feature matrix with the following classification formula to obtain the classification result; wherein the classification formula is:

[0100] O=softmax{(W c ,B c )|Project(F)}

[0101] Among them, Project(F) represents projecting the optimized classification feature matrix into a vector, W c is the weight matrix, B c represents a bias vector, softmax represents a normalized exponential function, and O represents the classification result.

[0102] In step S190, a corpus is output based on the classification result. In this way, the artificial intelligence technology based on deep learning is combined with the big data algorithm to automatically classify the text to be classified into topics, so as to intelligently construct a text corpus, thereby improving the accuracy of corpus construction.

[0103] In summary, the method for constructing a text classification corpus according to the embodiment of the present application has been explained, which uses web page content data and label data of web page topic data as corpus construction reference data, and extracts the globally dynamically changing multi-scale correlation features of each corpus construction reference data, and then uses this to construct a data feature library between the multi-scale web page data correlation features in the corpus construction reference data and the labels of the web page topics. Then, the global correlation features of the text data provided by the user in the high-dimensional space are used as query features to query the label data features of the topic of the text from the data feature library, and then output the corpus according to the topic. In this way, the artificial intelligence technology based on deep learning is combined with the big data algorithm, and the classified text is automatically classified by topic to intelligently construct the text corpus, which reduces the influence of human factors and improves the accuracy of corpus construction.

[0104] Exemplary Systems

[0105] Figure 4FIG. 1 is a block diagram of a system for constructing a text classification corpus according to an embodiment of the present application. Figure 4 As shown, according to the text classification corpus construction system 100 of the embodiment of the present application, it includes: a text data acquisition module 110, which is used to acquire text data provided by a user; a web page data acquisition module 120, which is used to acquire label data of web page topics and web page content data; a text encoding module 130, which is used to pass the text data provided by the user through a converter-based context encoder including an embedding layer to obtain a query feature vector; a web page encoding module 140, which is used to pass the label data of the web page topics and the web page content data through a converter-based context encoder including an embedding layer to obtain multiple web page feature vectors; a hybrid convolution module 150, which is used to convert the The multiple web page feature vectors are arranged in two dimensions into a web page feature matrix and then passed through a convolutional neural network model including multiple mixed convolutional layers to obtain a multi-scale web page feature matrix; an association coding module 160 is used to multiply the query feature vector with the multi-scale web page feature matrix to obtain a classification feature vector; an optimization module 170 is used to perform order priorization on the classification feature vector to obtain an optimized classification feature matrix; a topic generation module 180 is used to pass the optimized classification feature matrix through a classifier to obtain a classification result, and the classification result is used to represent a label of the topic of the text; and a corpus generation module 190 is used to output a corpus based on the classification result.

[0106] Here, those skilled in the art will appreciate that the specific functions and operations of the various units and modules in the text classification corpus construction system 100 have been described in detail above. Figures 1 to 3 The description of the construction method of the text classification corpus has been described in detail, and therefore, its repeated description will be omitted.

[0107] In summary, the system for constructing a text classification corpus according to the embodiment of the present application has been explained, which uses web page content data and label data of web page topic data as corpus construction reference data, and extracts the globally dynamically changing multi-scale correlation features of each corpus construction reference data, and then uses this to construct a data feature library between the multi-scale web page data correlation features in the corpus construction reference data and the labels of the web page topics. The global correlation features of the text data provided by the user in the high-dimensional space are then used as query features to query the label data features of the topic of the text from the data feature library, and then output the corpus according to the topic. In this way, the artificial intelligence technology based on deep learning is combined with the big data algorithm, and the classified text is automatically classified by topic to intelligently construct the text corpus, which reduces the influence of human factors and improves the accuracy of corpus construction.

[0108] As described above, the system 100 for constructing a text classification corpus according to an embodiment of the present application can be implemented in various terminal devices, such as a construction server for a text classification corpus, etc. In one example, the system 100 for constructing a text classification corpus according to an embodiment of the present application can be integrated into a terminal device as a software module and / or a hardware module. For example, the system 100 for constructing a text classification corpus can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the system 100 for constructing a text classification corpus can also be one of the many hardware modules of the terminal device.

[0109] Alternatively, in another example, the text classification corpus construction system 100 and the terminal device may also be separate devices, and the text classification corpus construction system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0110] Exemplary Electronic Devices

[0111] Below, reference Figure 5 To describe an electronic device according to an embodiment of the present application. Figure 5 FIG. 1 is a block diagram of an electronic device according to an embodiment of the present application. Figure 5 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0112] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0113] The memory 12 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the functions of the method for constructing the text classification corpus of each embodiment of the present application described above and / or other desired functions. Various contents such as text data provided by a user may also be stored in the computer-readable storage medium.

[0114] In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0115] The input device 13 may include, for example, a keyboard, a mouse, etc.

[0116] The output device 14 can output various information to the outside, including outputting a corpus based on the classification result, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0117] Of course, to simplify, Figure 5 Only some of the components related to the present application in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0118] Exemplary computer program products and computer-readable storage media

[0119] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to perform the steps in the functions of the method for constructing a text classification corpus according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0120] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0121] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to execute the steps in the functions of the method for constructing a text classification corpus according to various embodiments of the present application described in the above “Exemplary Method” section of this specification.

[0122] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0123] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0124] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0125] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0126] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0127] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for constructing a text classification corpus, characterized in that: include: Get text data provided by the user; Obtaining web page topic tag data and web page content data; Passing the user-provided text data through a transformer-based context encoder including an embedding layer to obtain a query feature vector; Passing the tag data of the webpage topic and the webpage content data through a converter-based context encoder including an embedding layer to obtain a plurality of webpage feature vectors; Arranging the plurality of web page feature vectors in two dimensions into a web page feature matrix and then passing the matrix through a convolutional neural network model including a plurality of hybrid convolutional layers to obtain a multi-scale web page feature matrix; Multiplying the query feature vector by the multi-scale web page feature matrix to obtain a classification feature vector; Performing order priori on the classification feature vector to obtain an optimized classification feature matrix; Passing the optimized classification feature matrix through a classifier to obtain a classification result, wherein the classification result is used to represent a label of the subject of the text; outputting a corpus based on the classification result; The classification feature vector is subjected to order priori to obtain an optimized classification feature matrix, including: Taking Gaussian distribution as a prior distribution, enhancing the feature expression of the classification feature vector based on the prior distribution to obtain a Gaussian enhanced classification feature matrix; Calculating the mutual information between each row vector in the Gaussian enhanced classification feature matrix and the classification feature vector to obtain multiple mutual information; Obtaining an order priori mask matrix based on a comparison between the plurality of mutual information and a predetermined threshold; Calculate the position point multiplication between the order priori mask matrix and the Gaussian enhanced classification feature matrix to obtain an optimized classification feature matrix; Among them, first, the mutual information between each row vector in the Gaussian enhanced classification feature matrix and the classification feature vector is calculated to obtain multiple mutual information, and the mutual information is used to represent the similarity between the enhanced feature expression and the original feature representation. Then, based on the comparison between the multiple mutual information and the predetermined threshold, the order prior mask matrix is ​​obtained. Specifically, if the i-th mutual information is greater than or equal to the predetermined threshold, the vector of the i-th row of the order prior mask matrix is ​​[1,1,.....1]; if the i-th mutual information is less than the predetermined threshold, the vector of the i-th row of the order prior mask matrix is ​​[0,0,.....0]. Then, the position point multiplication between the order prior mask matrix and the Gaussian enhanced classification feature matrix is ​​calculated to obtain the optimized classification feature matrix.

2. The method for constructing a text classification corpus according to claim 1, characterized in that: Passing the user-provided text data through a transformer-based context encoder including an embedding layer to obtain a query feature vector, comprising: Performing word segmentation processing on the text data provided by the user to obtain a plurality of user words; Passing the multiple user words through an embedding layer to convert each user word in the multiple user words into a user word embedding vector to obtain a sequence of user word embedding vectors, wherein the embedding layer uses a learnable embedding matrix to embed the each user word; Inputting the sequence of user word embedding vectors into the transformer-based context encoder to obtain a plurality of user feature vectors; The multiple user feature vectors are cascaded to obtain the query feature vector.

3. The method for constructing a text classification corpus according to claim 2, characterized in that: Inputting the sequence of user word embedding vectors into the converter-based context encoder to obtain the plurality of user feature vectors, comprising: Arranging the sequence of user word embedding vectors as an input vector; Convert the input vector into a query vector and a key vector respectively through a learnable embedding matrix; Calculating the product between the query vector and the transposed vector of the key vector to obtain a self-attention association matrix; Normalizing the self-attention association matrix to obtain a standardized self-attention association matrix; Input the standardized self-attention association matrix into a Softmax activation function for activation to obtain a self-attention feature matrix; The self-attention feature matrix is ​​multiplied by each user word embedding vector in the sequence of user word embedding vectors as a value vector to obtain the multiple user feature vectors.

4. The method for constructing a text classification corpus according to claim 3, characterized in that: The tag data of the webpage topic and the webpage content data are passed through a converter-based context encoder including an embedding layer to obtain a plurality of webpage feature vectors, including: One-hot encoding the label data of the webpage topic and the webpage content data through the embedding layer of the converter-based context encoder respectively, so as to convert the label data of the webpage topic and the webpage content data into input vectors respectively to obtain a sequence of input vectors; The sequence of input vectors is encoded based on global context semantics using a converter of the converter-based context encoder to obtain the plurality of web page feature vectors.

5. The method for constructing a text classification corpus according to claim 4, characterized in that: Using the converter of the converter-based context encoder to perform global context semantic encoding on the sequence of the input vectors to obtain the plurality of web page feature vectors, comprising: Calculating the product of the sequence of the input vectors and the transposed vector of each input vector in the sequence of the input vectors to obtain a plurality of self-attention association matrices; Normalizing each of the plurality of self-attention association matrices to obtain a plurality of standardized self-attention association matrices; Passing each of the plurality of standardized self-attention association matrices through a Softmax classification function to obtain a plurality of probability values; Each input vector in the sequence of input vectors is weighted using each probability value in the multiple probability values ​​as a weight to obtain the multiple web page feature vectors.

6. The method for constructing a text classification corpus according to claim 5, characterized in that: Arranging the plurality of web page feature vectors in two dimensions into a web page feature matrix and then passing the matrix through a convolutional neural network model including a plurality of hybrid convolutional layers to obtain a multi-scale web page feature matrix, including: Each mixed convolutional layer of the convolutional neural network model is used to perform the following operations on the input data in the forward pass of the layer: Performing convolution processing on the input data based on a first convolution kernel to obtain a first scale feature map; Performing convolution processing on the input data based on a second convolution kernel to obtain a second scale feature map, wherein the second convolution kernel is a dilated convolution kernel with a first dilation rate; Performing convolution processing on the input data based on a third convolution kernel to obtain a third scale feature map, wherein the third convolution kernel is a dilated convolution kernel with a second dilation rate; Performing convolution processing on the input data based on a fourth convolution kernel to obtain a fourth scale feature map, wherein the fourth convolution kernel is a dilated convolution kernel with a third dilation rate; Cascading the first scale feature map, the second scale feature map, the third scale feature map, and the fourth scale feature map to obtain a multi-scale feature map; Performing pooling processing on the multi-scale feature map to obtain a pooled feature map; Performing activation processing on the pooled feature map to obtain an activated feature map; Among them, the output of the last hybrid convolutional layer of the convolutional neural network model is the multi-scale web page feature matrix.

7. The method for constructing a text classification corpus according to claim 6, characterized in that: Cascading the first scale feature map, the second scale feature map, the third scale feature map, and the fourth scale feature map to obtain a multi-scale feature map, including: Cascading the first scale feature map, the second scale feature map, the third scale feature map, and the fourth scale feature map using the following cascade formula to obtain a multi-scale feature map; Wherein, the cascade formula is: in, , , and represent the first scale feature map, the second scale feature map, the third scale feature map and the fourth scale feature map respectively, Represents a cascade function.

8. The method for constructing a text classification corpus according to claim 7, characterized in that: The optimized classification feature matrix is ​​passed through a classifier to obtain a classification result, and the classification result is used to represent a label of the subject of the text, including: Using the classifier to process the optimized classification feature matrix with the following classification formula to obtain the classification result; Among them, the classification formula is: ,in represents the projection of the optimized classification feature matrix into a vector, is the weight matrix, represents the bias vector, represents the normalized exponential function, Indicates the classification result.

9. A system for constructing a text classification corpus, characterized in that: include: A text data acquisition module, used to acquire text data provided by a user; A web page data acquisition module is used to acquire the tag data of the web page theme and the web page content data; A text encoding module, configured to pass the user-provided text data through a transformer-based context encoder including an embedding layer to obtain a query feature vector; A web page encoding module, used for passing the tag data of the web page theme and the web page content data through a converter-based context encoder including an embedding layer to obtain a plurality of web page feature vectors; A hybrid convolution module, used for arranging the plurality of web page feature vectors in two dimensions into a web page feature matrix and then passing the matrix through a convolutional neural network model including a plurality of hybrid convolution layers to obtain a multi-scale web page feature matrix; An associative coding module, used for multiplying the query feature vector with the multi-scale web page feature matrix to obtain a classification feature vector; An optimization module, used for performing order priori on the classification feature vector to obtain an optimized classification feature matrix; A topic generation module, used for passing the optimized classification feature matrix through a classifier to obtain a classification result, wherein the classification result is used to represent a label of the topic of the text; A corpus generation module, used for outputting a corpus based on the classification result; Wherein, the optimization module includes: Taking Gaussian distribution as a prior distribution, enhancing the feature expression of the classification feature vector based on the prior distribution to obtain a Gaussian enhanced classification feature matrix; Calculating the mutual information between each row vector in the Gaussian enhanced classification feature matrix and the classification feature vector to obtain multiple mutual information; Obtaining an order priori mask matrix based on a comparison between the plurality of mutual information and a predetermined threshold; Calculate the position point multiplication between the order priori mask matrix and the Gaussian enhanced classification feature matrix to obtain an optimized classification feature matrix; Among them, first, the mutual information between each row vector in the Gaussian enhanced classification feature matrix and the classification feature vector is calculated to obtain multiple mutual information, and the mutual information is used to represent the similarity between the enhanced feature expression and the original feature representation. Then, based on the comparison between the multiple mutual information and the predetermined threshold, the order prior mask matrix is ​​obtained. Specifically, if the i-th mutual information is greater than or equal to the predetermined threshold, the vector of the i-th row of the order prior mask matrix is ​​[1,1,.....1]; if the i-th mutual information is less than the predetermined threshold, the vector of the i-th row of the order prior mask matrix is ​​[0,0,.....0]. Then, the position point multiplication between the order prior mask matrix and the Gaussian enhanced classification feature matrix is ​​calculated to obtain the optimized classification feature matrix.