Text data classification method and device, storage medium and electronic device

Abstract

The invention discloses a text data classification method and device, a storage medium and an electronic device, and relates to the technical field of natural language process.The method comprises the steps that semantic embedding vectors corresponding to original texts are grouped to obtain multiple sets of vector data, and the semantic embedding vectors are coding results of the original texts; determining a subject text of each group of vector data, and determining segmented embedded vectors according to the original text and the subject text; a context semantic embedding vector obtained by encoding a to-be-detected text is obtained, and the to-be-detected text comprises one of an original text and a subject text; generating a fusion embedding vector according to the context semantic embedding vector and the segmentation embedding vector; the fusion embedded vector is input into a classification model, a classification result output by the classification model is obtained, and the classification model is trained to carry out emotion tendency classification on the text data. By adopting the technical scheme, the technical problem of unstable classification performance of the text classification model is solved.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and more specifically, to a text data classification method, device, storage medium, and electronic device. Background Art

[0002] Existing text classification models include neural network models and generative large language models. Traditional neural network models are suitable for processing structured and standardized text, but they perform poorly with non-standard text, such as social media posts, due to their limited language comprehension capabilities. While generative large language models possess powerful language comprehension and generation capabilities, most current generative large language models typically use a generative text-to-text model for sentiment analysis, converting text sentiment classification into a question-and-answer task. Therefore, generative large language models rely on the design of cue word templates and text labels for sentiment analysis. Non-standard text containing a large amount of noise can disrupt the cue word templates, leading to uncontrollable output content and, in turn, unstable classification performance.

[0003] Regarding the technical problem of unstable classification performance of text classification models in related technologies, no effective solution has been proposed yet. Summary of the Invention

[0004] The embodiments of the present application provide a text data classification method, device, storage medium, and electronic device to at least solve the technical problem of unstable classification performance of text classification models in related technologies.

[0005] According to one embodiment of the embodiments of the present application, a method for classifying text data is provided, including: grouping semantic embedding vectors corresponding to original texts to obtain multiple groups of vector data, wherein the semantic embedding vectors are encoding results of the original texts; determining the subject text of each group of vector data, and determining a segment embedding vector based on the original text and the subject text; obtaining a contextual semantic embedding vector obtained by encoding the text to be detected, wherein the text to be detected includes one of the following: the original text, the subject text; generating a fused embedding vector based on the contextual semantic embedding vector and the segment embedding vector; inputting the fused embedding vector into a classification model to obtain a classification result output by the classification model, wherein the classification model is trained to classify text data for sentiment tendency.

[0006] In an exemplary embodiment, the semantic embedding vectors corresponding to the original text are grouped to obtain multiple groups of vector data, including: classifying the semantic embedding vectors through a clustering algorithm to obtain multiple semantic embedding vector clusters; determining the semantic embedding vector corresponding to each semantic embedding vector cluster as a group of semantic embedding vector data to obtain the multiple groups of vector data.

[0007] In an exemplary embodiment, determining the topic text of each group of semantic embedding vector data includes: calculating the cosine similarity between each semantic embedding vector in each group of semantic embedding vector data and a central vector, wherein the central vector is the central vector of the vector cluster corresponding to each group of semantic embedding vector data; sorting each group of semantic embedding vector data in descending order of the cosine similarity; determining a preset number of target semantic embedding vectors from the sorted semantic embedding vectors; inputting the original text corresponding to the target semantic embedding vector into a first large language model, and determining the summary text output by the first large language model as the topic text, wherein the first large language model includes at least an encoder and a decoder, the encoder is used to extract semantic information and context information of the text, and the decoder is used to generate summary text based on the semantic information and context information of the text.

[0008] In an exemplary embodiment, before grouping the semantic embedding vectors corresponding to the original text to obtain multiple groups of vector data, the method further includes: performing word segmentation on the original text and determining a first word segmentation sequence based on multiple words obtained by word segmentation; mapping the first word segmentation sequence to an initial vector of fixed dimension to obtain a first initial embedding vector; inputting the first initial embedding vector into a second large language model to obtain the semantic embedding vector output by the second large language model after performing semantic enhancement processing on the first initial embedding vector, wherein the semantic enhancement processing includes performing global context information extraction, residual connection and layer normalization on the first initial embedding vector.

[0009] In an exemplary embodiment, determining a segment embedding vector based on the original text and the topic text includes: determining the original text segment embedding vector corresponding to all original texts, and determining the topic text segment embedding vector corresponding to all topic texts, wherein the vector dimension of the topic text segment embedding vector and the vector dimension of the original text segment embedding vector are the same as the vector dimension of the semantic embedding vector; and concatenating the topic text segment embedding vector and the original text segment embedding vector to obtain the segment embedding vector.

[0010] In an exemplary embodiment, obtaining a contextual semantic embedding vector obtained by encoding a text to be detected includes: performing word segmentation on the text to be detected, and determining a second word segmentation sequence based on multiple words obtained by word segmentation; mapping the second word segmentation sequence to an initial vector of fixed dimension to obtain a second initial embedding vector; inputting the second initial embedding vector into a third largest language model to obtain the contextual semantic embedding vector output by the third largest language model after encoding and pooling the second initial embedding vector, wherein the encoding process is implemented by a multi-layer conversion network of the third largest language model, each layer of the multi-layer conversion network includes a self-attention module, a multi-head attention module, and a feedforward neural network, the self-attention module is used to extract contextual information of the second initial embedding vector, the multi-head attention module is used to learn weights of different types of contextual information, and the feedforward neural network is used to extract semantic information of the second initial embedding vector.

[0011] In an exemplary embodiment, a fused embedding vector is generated based on the contextual semantic embedding vector and the segmentation embedding vector, including: when it is determined that the text to be detected is the subject text, the contextual semantic embedding vector and the subject text segmentation embedding vector in the segmentation embedding vector are superimposed dimension by dimension to obtain the fused embedding vector; when it is determined that the text to be detected is the original text, the contextual semantic embedding vector and the original text segmentation embedding vector in the segmentation embedding vector are superimposed dimension by dimension to obtain the fused embedding vector.

[0012] According to another aspect of an embodiment of the present application, a text data classification device is also provided, including: a grouping module, used to group the semantic embedding vectors corresponding to the original text to obtain multiple groups of vector data, wherein the semantic embedding vector is the encoding result of the original text; a determination module, used to determine the subject text of each group of vector data, and determine the segment embedding vector based on the original text and the subject text; an acquisition module, used to obtain the context semantic embedding vector obtained by encoding the text to be detected, wherein the text to be detected includes one of the following: the original text, the subject text; a generation module, used to generate a fused embedding vector based on the context semantic embedding vector and the segment embedding vector; a classification module, used to input the fused embedding vector into a classification model to obtain a classification result output by the classification model, wherein the classification model is trained to classify text data for sentiment tendency.

[0013] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to execute the above-mentioned text data classification method when running.

[0014] According to another aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the text data classification method through the computer program.

[0015] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, which implements the above-mentioned text data classification method when executed by a processor.

[0016] In an embodiment of the present application, the semantic embedding vectors corresponding to the original text are grouped to obtain multiple groups of vector data, wherein the semantic embedding vectors are the encoding results of the original text; the subject text of each group of vector data is determined, and the segment embedding vector is determined according to the original text and the subject text; the contextual semantic embedding vector obtained by encoding the text to be detected is obtained, wherein the text to be detected includes one of the following: the original text, the subject text; a fusion embedding vector is generated according to the contextual semantic embedding vector and the segment embedding vector; the fusion embedding vector is input into a classification model to obtain a classification result output by the classification model, wherein the classification model is trained to classify the text data for sentiment tendency. Using the above technical solution, the subject text is obtained by clustering the semantic embedding vectors corresponding to the original text, and then the original text and the subject text are segmented and encoded to obtain the segment embedding vector. The contextual semantic embedding vector of the text to be detected is combined with the segment embedding vector to generate a fusion embedding vector, and then the fusion embedding vector is input into the classification model to obtain the classification result. The effect of text noise on the text classification task is reduced, the technical problem of unstable classification performance of the text classification model is solved, and the text classification efficiency is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 This is a hardware structure block diagram of a text data classification method according to an embodiment of the present application;

[0020] Figure 2 is a flowchart of a method for classifying text data according to an embodiment of the present application;

[0021] Figure 3 1 is a schematic diagram of a text data classification method according to an embodiment of the present application (I);

[0022] Figure 4 Schematic diagram (2) of a text data classification method according to an embodiment of the present application;

[0023] Figure 5 4 is a structural block diagram of a text data classification device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] The method embodiments provided in the embodiments of the present application can be executed in a computer device or a similar computing device. Taking running on a computer device as an example, Figure 1 FIG is a hardware structure block diagram of a computer device for the task allocation method of an embodiment of the present application. Figure 1 As shown, the computer device may include one or more ( Figure 1Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a microprocessor (Microprocessor Unit, referred to as MPU) or a programmable logic device (Programmable logic device, referred to as PLD)) and a memory 104 for storing data. In an exemplary embodiment, the above-mentioned computer device may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer device. For example, the computer device may also include Figure 1 More or fewer components than shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 Shown are different configurations with more functionality.

[0027] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the task allocation method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, which is equivalent to implementing the above method. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0028] The transmission device 106 is used to receive or send data via a network. A specific example of the aforementioned network may include a wireless network provided by a communications provider of the computer device. In one embodiment, the transmission device 106 may include a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0029] The aforementioned network may include, but is not limited to, at least one of the following: a wired network and a wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: a wide area network, a metropolitan area network, and a local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity) and Bluetooth.

[0030] In this embodiment, a text data classification method is provided, which is applied to the above terminal device. Figure 2 4 is a flowchart of a method for classifying text data according to an embodiment of the present application, the process comprising the following steps:

[0031] Step S202: grouping the semantic embedding vectors corresponding to the original text to obtain multiple groups of vector data, wherein the semantic embedding vectors are encoding results of the original text;

[0032] Optionally, in step S202, the original text represents unprocessed text data, such as user comment text extracted from a social networking platform. The semantic embedding vector corresponding to the original text is a highly robust semantic vector obtained by encoding the original text.

[0033] Step S204, determining the subject text of each set of vector data, and determining a segment embedding vector based on the original text and the subject text;

[0034] Step S206, obtaining a contextual semantic embedding vector obtained by encoding the text to be detected, wherein the text to be detected includes one of the following: the original text, the subject text;

[0035] Optionally, in the above step S206, the contextual semantic embedding vector is obtained by extracting the contextual semantics of the text to be detected and then encoding it. The encoding process can be implemented by a GLM (General Language Model) model. For example, the above encoding process can be implemented by the ChatGLM-6B model, an open source language model based on the GLM architecture.

[0036] Step S208, generating a fused embedding vector according to the contextual semantic embedding vector and the segment embedding vector;

[0037] Step S210: input the fused embedding vector into a classification model to obtain a classification result output by the classification model, wherein the classification model is trained to perform sentiment tendency classification on text data.

[0038] Through the above steps, the semantic embedding vectors corresponding to the original text can be grouped to obtain multiple groups of vector data, wherein the semantic embedding vectors are the encoding results of the original text; the theme text of each group of vector data is determined, and the segment embedding vector is determined based on the original text and the theme text; the contextual semantic embedding vector obtained by encoding the text to be detected is obtained, wherein the text to be detected includes one of the following: the original text, the theme text; a fused embedding vector is generated based on the contextual semantic embedding vector and the segment embedding vector; the fused embedding vector is input into a classification model to obtain a classification result output by the classification model, wherein the classification model is trained to classify the text data for sentiment tendency. Using the above technical solution, the theme text is obtained by clustering the semantic embedding vectors corresponding to the original text, then the original text and the theme text are segmented and encoded to obtain segment embedding vectors, and the contextual semantic embedding vector of the text to be detected is combined with the segment embedding vector to generate a fused embedding vector, and then the fused embedding vector is input into the classification model to obtain a classification result. This reduces the impact of text noise on the text classification task, solves the technical problem of unstable classification performance of the text classification model, and significantly improves text classification efficiency.

[0039] In an exemplary embodiment, the specific implementation process of the above-mentioned step S202 grouping the semantic embedding vectors corresponding to the original text to obtain multiple groups of vector data includes: classifying the semantic embedding vectors through a clustering algorithm to obtain multiple semantic embedding vector clusters; determining the semantic embedding vector corresponding to each semantic embedding vector cluster as a group of semantic embedding vector data to obtain the multiple groups of vector data.

[0040] In an exemplary embodiment, the process of determining the topic text of each group of semantic embedding vector data in the above step S04 can be achieved in the following manner: calculating the cosine similarity between each semantic embedding vector in each group of semantic embedding vector data and the central vector, wherein the central vector is the central vector of the vector cluster corresponding to each group of semantic embedding vector data; sorting each group of semantic embedding vector data in order of the cosine similarity from high to low; determining a preset number of target semantic embedding vectors from the sorted semantic embedding vectors; inputting the original text corresponding to the target semantic embedding vector into the first large language model, and determining the summary text output by the first large language model as the topic text, wherein the first large language model includes at least an encoder and a decoder, the encoder is used to extract semantic information and context information of the text, and the decoder is used to generate summary text based on the semantic information and context information of the text.

[0041] Optionally, in the above embodiment, the process of extracting the topic text of the original text is as follows: first, a clustering algorithm is used to classify the semantic embedding vectors corresponding to the original text to obtain multiple vector clusters, the cluster center of each vector cluster is determined to obtain the center vector, and then the cosine similarity between other vectors in the vector cluster and the center vector is calculated, and the text corresponding to the top N vectors with the highest cosine similarity (equivalent to the above target semantic vector) is selected as the representative text, and this part of the representative text is spliced ​​into a merged input and input as a prompt text into the ChatGLM-6B model. After receiving the representative text, ChatGLM-6B uses the encoding structure to perform semantic extraction and context modeling, and then generates the corresponding summary text as the topic text of the vector cluster through the decoding structure.

[0042] In an exemplary embodiment, before executing the above step S202 to group the semantic embedding vectors corresponding to the original text to obtain multiple groups of vector data, the method also includes: performing word segmentation on the original text, and determining a first word segmentation sequence based on multiple words obtained by word segmentation; mapping the first word segmentation sequence to an initial vector of a fixed dimension to obtain a first initial embedding vector; inputting the first initial embedding vector into a second large language model to obtain the semantic embedding vector output by the second large language model after performing semantic enhancement processing on the first initial embedding vector, wherein the semantic enhancement processing includes performing global context information extraction, residual connection and layer normalization on the first initial embedding vector.

[0043] Optionally, in the above embodiment, the semantic embedding vector corresponding to the original text is obtained in the following manner: first, the original text is segmented to obtain a segmentation result, and the segmentation result is converted into a Token sequence (equivalent to the above segmentation sequence). In natural language processing, Token is the basic unit of text, and Token can be a word or a character. Each Token is then mapped to an initial vector of a fixed dimension to obtain a first embedding vector (EmbeddingVector), wherein the initial vector of a fixed dimension is used to capture the semantics and contextual information of the Token. The above first embedding vector is input into the LLM Embedder model (large language embedder model), and the LLM Embedder obtains global context information through a multi-layer Transformer (a neural network structure based on a self-attention mechanism). The multi-layer Transformer structure means that the input data will pass through multiple self-attention layers, and each layer will further refine and enrich the contextual information, thereby enabling the model to capture more complex semantic relationships. Model stability is then enhanced through residual connections and layer normalization. Residual connections allow inputs to be passed directly to subsequent layers, alleviating the vanishing gradient problem in deep network training. Specifically, if the input of a layer is x and the output is F(x), then the residual connection will use x + F(x) as the input of the next layer. Layer normalization normalizes each feature of each input sample so that the mean of each feature is 0 and the variance is 1, thereby reducing the model's internal covariate shift and improving the model's convergence speed and stability.

[0044] Optionally, in the above embodiment, the embedding expression capability of the LLM Embedder model for specific sentiment analysis tasks can be enhanced through task-oriented fine-tuning and knowledge distillation mechanisms, outputting a semantic embedding vector with a fixed dimension. Fine-tuning refers to further training for a specific task based on the pre-trained model, so that the model can better adapt to the needs of the specific task. In this process, the model weights are adjusted according to the loss function of the specific task, thereby improving the model's performance on the task. Knowledge distillation is a model compression technique used to transfer the knowledge of a large and complex model (teacher model) to a small model (student model). Specifically, the output of the teacher model is used as a training target for the student model, allowing the student model to learn the complex behavior of the teacher model. In this process, the student model not only learns the label information of the data, but also learns the intermediate representation and decision-making process of the teacher model. After the above steps, the LLM Embedder model ultimately outputs a semantic embedding vector with a fixed dimension. This vector can effectively represent the semantic information of the input text and has good robustness and adaptability, making it suitable for downstream tasks such as sentiment analysis.

[0045] In an exemplary embodiment, the specific implementation process of determining the segment embedding vector based on the original text and the topic text in the above step S204 includes: determining the original text segment embedding vector corresponding to all original texts, and determining the topic text segment embedding vector corresponding to all topic texts, wherein the vector dimension of the topic text segment embedding vector and the vector dimension of the original text segment embedding vector are the same as the vector dimension of the semantic embedding vector; splicing the topic text segment embedding vector and the original text segment embedding vector to obtain the segment embedding vector.

[0046] In an optional embodiment, if Figure 3 As shown in the figure, sentence embedding is used for segment encoding to distinguish the topic text from the original text. For the topic text part, the segment encoding is 0; for the original text part, the segment encoding is 1. Sentence Embedding accepts two inputs, 0 and 1, and maps the segment information (0 / 1 encoding) of the topic text part and the original text part into an embedding vector with the same dimension as the semantic embedding vector. The two encodings are input into the trainable embedding representation layer to obtain the original text segment embedding vector and the topic text segment embedding vector.

[0047] In an exemplary embodiment, the implementation process of the above-mentioned step S206 for obtaining the contextual semantic embedding vector obtained by encoding the text to be detected includes: performing word segmentation on the text to be detected, and determining a second word segmentation sequence based on multiple words obtained by word segmentation; mapping the second word segmentation sequence to an initial vector of fixed dimension to obtain a second initial embedding vector; inputting the second initial embedding vector into a third language model to obtain the contextual semantic embedding vector output by the third language model after encoding and pooling the second initial embedding vector, wherein the encoding process is implemented by a multi-layer conversion network of the third language model, each layer of the multi-layer conversion network includes a self-attention module, a multi-head attention module, and a feedforward neural network, the self-attention module is used to extract contextual information of the second initial embedding vector, the multi-head attention module is used to learn weights of different types of contextual information, and the feedforward neural network is used to extract semantic information of the second initial embedding vector.

[0048] Optionally, in the above embodiment, the text to be detected is first segmented and serialized to obtain a segmentation sequence, the segmentation sequence is embedded into the initial vector to obtain a second initial embedding vector, and then the second initial embedding vector is input into the multi-layer Transformer structure of the ChatGLM-6B model (equivalent to the above-mentioned multi-layer conversion network). Each layer of the Transformer structure includes a self-attention mechanism, a multi-head attention module and a feedforward neural network. The contextual semantics of the second initial embedding vector is obtained in combination with position encoding, and then the vector corresponding to each word in the segmentation sequence is obtained, and the obtained vector is average pooled to obtain the contextual semantic embedding vector of the entire text.

[0049] In an exemplary embodiment, the above-mentioned step S208 can generate a fused embedding vector based on the contextual semantic embedding vector and the segmentation embedding vector by the following manner, specifically including: when it is determined that the text to be detected is the subject text, the contextual semantic embedding vector and the subject text segmentation embedding vector in the segmentation embedding vector are superimposed dimension by dimension to obtain the fused embedding vector; when it is determined that the text to be detected is the original text, the contextual semantic embedding vector and the original text segmentation embedding vector in the segmentation embedding vector are superimposed dimension by dimension to obtain the fused embedding vector.

[0050] In an optional embodiment, if Figure 4 As shown in the figure, the topic text and the original text are segmented and encoded to obtain segmented embedding vectors. Then, the topic text and the original text are encoded into semantic embedding vectors and input into the ChatGLM-6B model. The contextual semantic embedding vector is extracted by ChatGLM-6B, where the contextual semantic embedding vector has the same dimension as the segmented embedding vector. The fused embedding vector that integrates structural information and semantic information is obtained by superimposing the contextual semantic embedding vector and the segmented embedding vector. The fused embedding vector is then input into the classification model for classification to obtain the classification result.

[0051] Optionally, in the above embodiment, for example, the above classification model is a binary classification prediction model for sentiment tendency through a fully connected neural network. The TextCNN (Text Convolutional Neural Network) layer of the fusion embedding vector input model extracts local features in the text through a convolution operation, and then reduces the dimension and enhances the robustness through a pooling layer, and then outputs the probability value of each sentiment category through the activation function of one or more fully connected layers to obtain the classification result. The cross entropy loss function is used in the classification model training process, and the optimizer is used to update the parameters, thereby achieving accurate prediction of sentiment tendency.

[0052] In the above embodiments, the use of aggregated topic extraction reduces text noise interference, semantic embedding can effectively avoid the problem of prompt word template destruction, and the fusion of segmented coding and semantic embedding can combine the advantages of large models and the excellent characteristics of traditional algorithms, providing a more flexible and efficient solution for sentiment classification tasks.

[0053] In an optional embodiment, for example, for a binary text sentiment analysis task, the sample labels are positive and negative. First, the LLM Embedder model is used to obtain the semantic embedding vectors of all text data in the text dataset to be analyzed, and then the obtained semantic embedding vectors are clustered by a clustering algorithm to obtain vector clusters. Several texts that are most relevant to the cluster center vector are extracted from each vector cluster and input into the ChatGLM-6B model to generate the topic summary of the vector cluster, which serves as the aggregate topic of all texts in the vector cluster. For example: {Topic: Protests broke out in many places due to Brexit, text content: [text 1, text 2, ..., text n]}. Segment coding is introduced to distinguish the topic summary from the original text. The topic summary is encoded as 0 and the original text is encoded as 1. The segment coding is input into the trainable embedding representation layer to obtain the segment embedding table vector of the topic summary and the segment embedding vector of the original text. The topic summary and the original text are respectively input into the ChatGLM-6B model to extract the contextual semantic embedding vector. The segment embedding table vector is superimposed with the contextual semantic embedding vector to obtain a fused embedding vector that combines the topic information and the original text semantics. Finally, the fused embedding vector is input into a classification model based on a fully connected neural network to perform a binary classification prediction of sentiment tendency.

[0054] Through the above embodiments, the impact of text noise on the classification task is reduced, and at the same time, the large language model is compatible with the traditional text classification algorithm, thereby improving the performance and stability of the classification model.

[0055] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0056] Figure 5 1 is a structural block diagram of a text data classification device according to an embodiment of the present application, the device comprising:

[0057] A grouping module 50 is used to group the semantic embedding vectors corresponding to the original text to obtain multiple groups of vector data, wherein the semantic embedding vectors are encoding results of the original text;

[0058] A determination module 52 is configured to determine a subject text of each set of vector data, and determine a segment embedding vector based on the original text and the subject text;

[0059] An acquisition module 54 is configured to acquire a contextual semantic embedding vector obtained by encoding the text to be detected, wherein the text to be detected includes one of the following: the original text, the subject text;

[0060] A generating module 56, configured to generate a fused embedding vector based on the contextual semantic embedding vector and the segment embedding vector;

[0061] The classification module 58 is used to input the fused embedding vector into a classification model to obtain a classification result output by the classification model, wherein the classification model is trained to classify the sentiment tendency of text data.

[0062] The above-mentioned device can group the semantic embedding vectors corresponding to the original text to obtain multiple groups of vector data, wherein the semantic embedding vectors are the encoding results of the original text; determine the theme text of each group of vector data, and determine the segment embedding vector based on the original text and the theme text; obtain the contextual semantic embedding vector obtained by encoding the text to be detected, wherein the text to be detected includes one of the following: the original text, the theme text; generate a fused embedding vector based on the contextual semantic embedding vector and the segment embedding vector; input the fused embedding vector into a classification model to obtain a classification result output by the classification model, wherein the classification model is trained to classify text data for sentiment tendency. Using the above-mentioned technical solution, the theme text is obtained by clustering the semantic embedding vectors corresponding to the original text, then the original text and the theme text are segmented and encoded to obtain segment embedding vectors, and the contextual semantic embedding vector of the text to be detected is combined with the segment embedding vector to generate a fused embedding vector, and then the fused embedding vector is input into the classification model to obtain a classification result. This reduces the impact of text noise on text classification tasks, solves the technical problem of unstable classification performance of text classification models, and significantly improves text classification efficiency.

[0063] In an exemplary embodiment, the grouping module 50 is further used to classify the semantic embedding vectors through a clustering algorithm to obtain multiple semantic embedding vector clusters; and determine the semantic embedding vector corresponding to each semantic embedding vector cluster as a group of semantic embedding vector data to obtain the multiple groups of vector data.

[0064] In an exemplary embodiment, the determination module 52 is further used to calculate the cosine similarity between each semantic embedding vector in each group of semantic embedding vector data and the central vector, wherein the central vector is the central vector of the vector cluster corresponding to each group of semantic embedding vector data; sort each group of semantic embedding vector data in order of the cosine similarity from high to low; determine a preset number of target semantic embedding vectors from the sorted semantic embedding vectors; input the original text corresponding to the target semantic embedding vector into the first large language model, and determine the summary text output by the first large language model as the topic text, wherein the first large language model includes at least an encoder and a decoder, the encoder is used to extract semantic information and context information of the text, and the decoder is used to generate summary text based on the semantic information and context information of the text.

[0065] In an exemplary embodiment, the grouping module 50 is further used to perform word segmentation on the original text, determine a first word segmentation sequence based on multiple words obtained from the word segmentation; map the first word segmentation sequence to an initial vector of fixed dimension to obtain a first initial embedding vector; input the first initial embedding vector into a second large language model to obtain the semantic embedding vector output by the second large language model after performing semantic enhancement processing on the first initial embedding vector, wherein the semantic enhancement processing includes performing global context information extraction, residual connection and layer normalization on the first initial embedding vector.

[0066] In an exemplary embodiment, the determination module 52 is also used to determine the original text segment embedding vectors corresponding to all original texts, and to determine the topic text segment embedding vectors corresponding to all topic texts, wherein the vector dimension of the topic text segment embedding vector and the vector dimension of the original text segment embedding vector are the same as the vector dimension of the semantic embedding vector; the topic text segment embedding vector and the original text segment embedding vector are concatenated to obtain the segment embedding vector.

[0067] In an exemplary embodiment, the acquisition module 54 is also used to segment the text to be detected, determine a second segmentation sequence based on multiple words obtained by the segmentation; map the second segmentation sequence to an initial vector of fixed dimension to obtain a second initial embedding vector; input the second initial embedding vector into a third language model to obtain the contextual semantic embedding vector output by the third language model after encoding and pooling the second initial embedding vector, wherein the encoding process is implemented by a multi-layer conversion network of the third language model, each layer of the multi-layer conversion network includes a self-attention module, a multi-head attention module, and a feedforward neural network, the self-attention module is used to extract contextual information of the second initial embedding vector, the multi-head attention module is used to learn the weights of different types of contextual information, and the feedforward neural network is used to extract semantic information of the second initial embedding vector.

[0068] In an exemplary embodiment, the generation module 56 is further used to, when it is determined that the text to be detected is the subject text, superimpose the contextual semantic embedding vector and the subject text segment embedding vector in the segment embedding vector dimension by dimension to obtain the fused embedding vector; when it is determined that the text to be detected is the original text, superimpose the contextual semantic embedding vector and the original text segment embedding vector in the segment embedding vector dimension by dimension to obtain the fused embedding vector.

[0069] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0070] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0071] S1, grouping the semantic embedding vectors corresponding to the original text to obtain multiple groups of vector data, wherein the semantic embedding vectors are encoding results of the original text;

[0072] S2, determining the subject text of each set of vector data, and determining a segment embedding vector based on the original text and the subject text;

[0073] S3, obtaining a contextual semantic embedding vector obtained by encoding the text to be detected, wherein the text to be detected includes one of the following: the original text, the subject text;

[0074] S4, generating a fused embedding vector according to the contextual semantic embedding vector and the segment embedding vector;

[0075] S5, inputting the fused embedding vector into a classification model to obtain a classification result output by the classification model, wherein the classification model is trained to perform sentiment tendency classification on text data.

[0076] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0077] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0078] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0079] S1, grouping the semantic embedding vectors corresponding to the original text to obtain multiple groups of vector data, wherein the semantic embedding vectors are encoding results of the original text;

[0080] S2, determining the subject text of each set of vector data, and determining a segment embedding vector based on the original text and the subject text;

[0081] S3, obtaining a contextual semantic embedding vector obtained by encoding the text to be detected, wherein the text to be detected includes one of the following: the original text, the subject text;

[0082] S4, generating a fused embedding vector according to the contextual semantic embedding vector and the segment embedding vector;

[0083] S5, inputting the fused embedding vector into a classification model to obtain a classification result output by the classification model, wherein the classification model is trained to perform sentiment tendency classification on text data.

[0084] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0085] Optionally, an embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0086] Optionally, an embodiment of the present application further provides another computer program product, comprising a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above method embodiments are implemented.

[0087] Optionally, an embodiment of the present application further provides a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any of the above method embodiments.

[0088] Optionally, specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0089] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices. Alternatively, they can be implemented using program code executable by the computing device, so that they can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described can be performed in a different order than herein, or they can be made into separate integrated circuit modules, or multiple modules or steps can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0090] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A text data classification method, characterized in that: include: Grouping semantic embedding vectors corresponding to the original text to obtain multiple groups of vector data, wherein the semantic embedding vectors are encoding results of the original text; Determine the subject text of each set of vector data, and determine a segment embedding vector based on the original text and the subject text; Obtaining a contextual semantic embedding vector obtained by encoding the text to be detected, wherein the text to be detected includes one of the following: the original text, the subject text; Generating a fused embedding vector according to the contextual semantic embedding vector and the segment embedding vector; The fused embedding vector is input into a classification model to obtain a classification result output by the classification model, wherein the classification model is trained to classify the sentiment tendency of text data.

2. The text data classification method according to claim 1, characterized in that: The semantic embedding vectors corresponding to the original text are grouped to obtain multiple groups of vector data, including: The semantic embedding vectors are classified by a clustering algorithm to obtain a plurality of semantic embedding vector clusters; the semantic embedding vector corresponding to each semantic embedding vector cluster is determined as a group of semantic embedding vector data to obtain the plurality of groups of vector data.

3. The text data classification method according to claim 1, characterized in that: Determine the topic text of each set of semantic embedding vector data, including: Calculating the cosine similarity between each semantic embedding vector in each set of semantic embedding vector data and a central vector, wherein the central vector is the central vector of the vector cluster corresponding to each set of semantic embedding vector data; Sorting each set of semantic embedding vector data in descending order of the cosine similarity; Determining a preset number of target semantic embedding vectors from the sorted semantic embedding vectors; The original text corresponding to the target semantic embedding vector is input into a first large language model, and the summary text output by the first large language model is determined as the topic text, wherein the first large language model includes at least an encoder and a decoder, the encoder is used to extract semantic information and context information of the text, and the decoder is used to generate a summary text based on the semantic information and context information of the text.

4. The text data classification method according to claim 1, characterized in that: Before grouping the semantic embedding vectors corresponding to the original text to obtain multiple groups of vector data, the method further includes: Segmenting the original text, and determining a first segmentation sequence based on a plurality of words obtained by the segmentation; mapping the first segmentation sequence to an initial vector of a fixed dimension to obtain a first initial embedding vector; The first initial embedding vector is input into a second large language model to obtain the semantic embedding vector output by the second large language model after performing semantic enhancement processing on the first initial embedding vector, wherein the semantic enhancement processing includes performing global context information extraction, residual connection, and layer normalization on the first initial embedding vector.

5. The text data classification method according to claim 1, characterized in that: Determining a segment embedding vector according to the original text and the subject text includes: Determine the original text segment embedding vectors corresponding to all original texts, and determine the topic text segment embedding vectors corresponding to all topic texts, wherein the vector dimension of the topic text segment embedding vector and the vector dimension of the original text segment embedding vector are the same as the vector dimension of the semantic embedding vector; splice the topic text segment embedding vector and the original text segment embedding vector to obtain the segment embedding vector.

6. The text data classification method according to claim 1, characterized in that: Obtain the contextual semantic embedding vector obtained by encoding the text to be detected, including: Performing word segmentation on the text to be detected, and determining a second word segmentation sequence based on a plurality of words obtained by the word segmentation; Mapping the second word segmentation sequence to an initial vector of fixed dimension to obtain a second initial embedding vector; The second initial embedding vector is input into a third language model to obtain the contextual semantic embedding vector output by the third language model after encoding and pooling the second initial embedding vector, wherein the encoding process is implemented by a multi-layer conversion network of the third language model, each layer of the multi-layer conversion network includes a self-attention module, a multi-head attention module, and a feedforward neural network, the self-attention module is used to extract contextual information of the second initial embedding vector, the multi-head attention module is used to learn weights of different types of contextual information, and the feedforward neural network is used to extract semantic information of the second initial embedding vector.

7. The text data classification method according to claim 5, characterized in that: Generating a fused embedding vector according to the contextual semantic embedding vector and the segment embedding vector, comprising: When it is determined that the text to be detected is the subject text, the context semantic embedding vector is superimposed dimension by dimension with the subject text segment embedding vector in the segment embedding vector to obtain the fused embedding vector; When it is determined that the text to be detected is the original text, the contextual semantic embedding vector and the original text segment embedding vector in the segment embedding vector are superimposed dimension by dimension to obtain the fused embedding vector.

8. A text data classification device, characterized in that: include: A grouping module, configured to group the semantic embedding vectors corresponding to the original text to obtain multiple groups of vector data, wherein the semantic embedding vectors are encoding results of the original text; A determination module, configured to determine a subject text for each set of vector data, and determine a segment embedding vector based on the original text and the subject text; An acquisition module is used to acquire a contextual semantic embedding vector obtained by encoding a text to be detected, wherein the text to be detected includes one of the following: the original text, the subject text; A generating module, configured to generate a fused embedding vector based on the contextual semantic embedding vector and the segment embedding vector; A classification module is used to input the fused embedding vector into a classification model to obtain a classification result output by the classification model, wherein the classification model is trained to classify the sentiment tendency of text data.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images