Text classification method and device, text classification model training method and device and electronic equipment

Abstract

The invention provides a text classification method, a text classification model training method and device and electronic equipment. The method comprises the steps of obtaining a to-be-classified text; extracting text elements of various description types related to the text classification task from the text; adding description type marks of description types corresponding to the text elements in positions corresponding to the text elements in the text to obtain a text sequence; obtaining an embedded vector sequence of the text sequence; encoding the embedded vector sequence to obtain a text vector representing the text and a plurality of text element vectors representing the text elements of the plurality of description types respectively; performing fusion processing on the text vector and the plurality of text element vectors to obtain a text fusion vector; and mapping the text fusion vector into a classification result of a text type related to the text classification task. Through the method and the device, the text classification accuracy can be improved.

Description

Technical Field

[0001] The present application relates to artificial intelligence technology, and in particular to a text classification method, a text classification model training method, an apparatus, and an electronic device. Background Art

[0002] Artificial Intelligence (AI) is a comprehensive technology in computer science. By studying the design principles and implementation methods of various intelligent machines, machines are enabled to have the functions of perception, reasoning, and decision-making. Natural language processing technology is an important direction in artificial intelligence technology and is applied in more and more fields and plays an increasingly important role.

[0003] Among them, text classification is an important field of application of natural language processing technology. In related technologies, text classification is usually performed by keyword matching or using a classification model. However, the keyword matching scheme cannot accurately hit a specific category and has a high misjudgment rate; the results of ordinary classification models are not refined enough, resulting in poor text classification effects. Summary of the Invention

[0004] Embodiments of the present application provide a text classification method, a text classification model training method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the accuracy of text classification.

[0005] The technical solution of the embodiments of the present application is implemented as follows:

[0006] Embodiments of the present application provide a text classification method, the method comprising:

[0007] Obtain the text to be classified;

[0008] Extract text elements of multiple description types related to the text classification task from the text;

[0009] Add a description type tag corresponding to the description type of the text element at the position corresponding to the text element in the text to obtain a text sequence;

[0010] Obtain an embedding vector sequence of the text sequence;

[0011] Encode the embedding vector sequence to obtain a text vector representing the text and multiple text element vectors respectively representing the text elements of the multiple description types;

[0012] Perform a fusion process on the text vector and the multiple text element vectors to obtain a text fusion vector;

[0013] Map the text fusion vector to a classification result of a text type related to the text classification task.

[0014] An embodiment of the present application provides a method for training a text classification model, where the text classification model includes a semantic understanding model and a classifier;

[0015] The method includes:

[0016] Obtain a text sample and an actual classification result of the text sample;

[0017] Extract text elements of multiple description types related to the text classification task from the text sample;

[0018] Add a description type marker of the description type corresponding to the text element at the position corresponding to the text element in the text sample to obtain a text sample sequence;

[0019] Obtain an embedding vector sequence of the text sample sequence;

[0020] Encode the embedding vector sequence through the semantic understanding model to obtain a text vector representing the text sample and multiple text element vectors respectively representing the text elements of the multiple description types;

[0021] Perform a fusion process on the text vector and the multiple text element vectors to obtain a text fusion vector;

[0022] Map the text fusion vector to a predicted classification result of a text type related to the text classification task through the classifier;

[0023] According to the difference between the predicted classification result and the actual classification result, reversely update the parameters of the semantic understanding model and the classifier.

[0024] In the above solution, the extracting text elements of multiple description types related to the text classification task from the text sample includes:

[0025] Obtain multiple keyword libraries respectively associated with the multiple description types;

[0026] Compare the keywords in the keyword library with the text sample to obtain a comparison result;

[0027] In response to the comparison result indicating that the text sample contains the keyword, take the part of the text sample containing the keyword as the text element of the description type associated with the keyword library.

[0028] In the above solution, the extracting text elements of multiple description types related to the text classification task from the text sample includes:

[0029] Obtain a prompt, where the prompt includes the text classification task and multiple description types to which the text elements related to the text classification task belong;

[0030] Call the semantic understanding model according to the prompt, and extract text elements of the multiple description types from the text sample.

[0031] In the above solution, the fusion processing of the text vector and the multiple text element vectors to obtain a text fusion vector includes:

[0032] Perform average pooling processing on the multiple text element vectors to obtain an average pooling vector;

[0033] Perform weighted processing on the average pooling vector to obtain a weighted vector corresponding to the multiple text element vectors;

[0034] Concatenate the text vector and the weighted vector corresponding to the multiple text element vectors to obtain a text fusion vector.

[0035] In the above solution, the obtaining of the embedding vector sequence of the text sample sequence includes:

[0036] Add a start marker at the start position of the text sample sequence and an end marker at the end position of the text sample sequence to form a marked sequence;

[0037] Perform embedding processing on the marked sequence to obtain the embedding vector sequence of the text sample sequence.

[0038] In the above solution, the performing of embedding processing on the marked sequence to obtain the embedding vector sequence of the text sample sequence includes:

[0039] Perform semantic embedding processing on the elements in the marked sequence to obtain a symbol embedding vector corresponding to each element in the marked sequence, where the elements include the text elements, the start marker, and the end marker;

[0040] Perform role embedding processing on the elements in the marked sequence to obtain a role embedding vector corresponding to each element in the marked sequence, where the role embedding vectors of the text elements with different description type markers are different;

[0041] Generate the embedding vector sequence based on the symbol embedding vector and the role embedding vector of each element.

[0042] In the above solution, the generating of the embedding vector sequence based on the symbol embedding vector and the role embedding vector of each element includes:

[0043] Perform segment embedding processing on the elements in the token sequence to obtain segment embedding vectors corresponding to the token sequence, where the segment embedding vectors represent the embedding vectors of the segments to which the elements belong in the text, and the segment embedding vectors corresponding to different segments in the text are different;

[0044] Perform positional embedding processing on the elements in the token sequence to obtain positional embedding vectors corresponding to the token sequence, where the segment embedding vectors corresponding to different texts are different;

[0045] Determine the sum of the symbol embedding vector, the role embedding vector, the segment embedding vector, and the positional embedding vector of each element, and form an embedding vector sequence with the sums corresponding to each element.

[0046] In the above solution, encoding the embedding vector sequence by the semantic understanding model to obtain a text vector representing the text sample and multiple text element vectors respectively representing text elements of multiple description types includes:

[0047] Call the semantic understanding model to encode the embedding vector sequence to obtain an encoded sequence, where the encoded sequence includes the embedding vectors corresponding to the text elements in the text sample sequence and the embedding vector corresponding to the start token;

[0048] Use the embedding vector corresponding to the start token as the text vector;

[0049] Use the embedding vectors corresponding to the text elements with the description type tags as the text element vectors representing the text elements of the description types corresponding to the description type tags.

[0050] In the above solution, mapping the text fusion vector to a predicted classification result of a text type related to the text classification task by the classifier includes:

[0051] Map the text fusion vector to a probability distribution of text types related to the text classification task by the classifier;

[0052] Use the text type corresponding to the maximum probability in the probability distribution as the predicted classification result of the text type related to the text classification task.

[0053] An embodiment of the present application provides a text classification device, and the device includes:

[0054] A first acquisition module, configured to acquire the text to be classified; acquire the embedding vector sequence of the text sequence;

[0055] A first processing module, configured to extract text elements of multiple description types related to a text classification task from the text; add description type tags of the corresponding description types of the text elements at positions corresponding to the text elements in the text to obtain a text sequence;

[0056] A first encoding module, configured to encode the embedding vector sequence to obtain a text vector representing the text and multiple text element vectors respectively representing the text elements of the multiple description types;

[0057] A first fusion module, configured to perform a fusion process on the text vector and the multiple text element vectors to obtain a text fusion vector;

[0058] A first mapping module, which maps the text fusion vector to a classification result of a text type related to the text classification task.

[0059] An embodiment of the present application provides a text classification model training device, where the text classification model includes a semantic understanding model and a classifier;

[0060] The device includes:

[0061] A second acquisition module, configured to acquire a text sample and the actual classification result of the text sample; acquire an embedding vector sequence of the text sample sequence;

[0062] A second processing module, configured to extract text elements of multiple description types related to a text classification task from the text sample; add description type tags of the corresponding description types of the text elements at positions corresponding to the text elements in the text sample to obtain a text sample sequence;

[0063] A second encoding module, configured to encode the embedding vector sequence through the semantic understanding model to obtain a text vector representing the text sample and multiple text element vectors respectively representing the text elements of the multiple description types;

[0064] A second fusion module, configured to perform a fusion process on the text vector and the multiple text element vectors to obtain a text fusion vector;

[0065] A second mapping module, configured to map the text fusion vector to a predicted classification result of a text type related to the text classification task through the classifier;

[0066] An update module, configured to reversely update the parameters of the semantic understanding model and the classifier according to the difference between the predicted classification result and the actual classification result.

[0067] An embodiment of the present application provides an electronic device, which includes:

[0068] A memory for storing computer-executable instructions;

[0069] A processor, when executing the computer-executable instructions stored in the memory, implements the text classification method or the text classification model training method provided by the embodiment of the present application.

[0070] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which when executed by a processor implement the text classification method or the text classification model training method provided by the embodiment of the present application.

[0071] An embodiment of the present application provides a computer program product including a computer program or computer-executable instructions, which when executed by a processor implement the text classification method or the text classification model training method provided by the embodiment of the present application.

[0072] The embodiment of the present application has the following beneficial effects:

[0073] By extracting text elements of multiple description types related to the text classification task from the text to be classified; and adding description type tags corresponding to the description types of the text elements at the positions of the text elements in the text to obtain a text sequence; obtaining an embedding vector sequence of the text sequence; encoding the embedding vector sequence to obtain a text vector representing the text and multiple text element vectors respectively representing the text elements of multiple description types; performing a fusion process on the text vector and the multiple text element vectors to obtain a text fusion vector; mapping the text fusion vector to a classification result of a text type related to the text classification task. By combining text elements of different description types to classify the text, the accuracy of text classification can be improved. Description of the Drawings

[0074] Figure 1 is a schematic structural diagram of a text classification system 100 provided by an embodiment of the present application;

[0075] Figure 2A is a schematic structural diagram of a terminal 400 provided by an embodiment of the present application;

[0076] Figure 2B is a schematic structural diagram of a server 200 provided by an embodiment of the present application;

[0077] Figure 3A is a schematic flowchart of the text classification method provided by an embodiment of the present application;

[0078] Figure 3BIt is the first process schematic diagram of the method for extracting text elements provided by the embodiments of the present application;

[0079] Figure 3C It is the second process schematic diagram of the method for extracting text elements provided by the embodiments of the present application;

[0080] Figure 3D It is the process schematic diagram of the method for obtaining the embedding vector sequence of the text sequence provided by the embodiments of the present application;

[0081] Figure 3E It is the process schematic diagram of the method for generating the embedding vector sequence of the text sequence provided by the embodiments of the present application;

[0082] Figure 3F It is the process schematic diagram of the method for obtaining the text vector and text element vector representing the text provided by the embodiments of the present application;

[0083] Figure 3G It is the first process schematic diagram of the method for obtaining the text fusion vector provided by the embodiments of the present application;

[0084] Figure 3H It is the process schematic diagram of the method for obtaining the text classification result provided by the embodiments of the present application;

[0085] Figure 3I It is the process schematic diagram of the method for training the text classification model provided by the embodiments of the present application;

[0086] Figure 3J It is the third process schematic diagram of the method for extracting text elements provided by the embodiments of the present application;

[0087] Figure 3K It is the fourth process schematic diagram of the method for extracting text elements provided by the embodiments of the present application;

[0088] Figure 3L It is the process schematic diagram of the method for obtaining the embedding vector sequence of the text sample sequence provided by the embodiments of the present application;

[0089] Figure 3M It is the second process schematic diagram of the method for generating the embedding vector sequence of the text sample sequence provided by the embodiments of the present application;

[0090] Figure 3N It is the process schematic diagram of the method for obtaining the text vector and text element vector representing the text sample provided by the embodiments of the present application;

[0091] Figure 3O It is the second process schematic diagram of the method for obtaining the text fusion vector provided by the embodiments of the present application;

[0092] Figure 3PIt is a schematic flowchart for obtaining the predicted classification result of a text sample provided by an embodiment of the present application;

[0093] Figure 4A It is a network structure diagram provided by an embodiment of the present application;

[0094] Figure 4B It is a schematic diagram of a role embedding vector provided by an embodiment of the present application;

[0095] Figure 4C It is a schematic diagram for generating an embedding vector sequence provided by an embodiment of the present application;

[0096] Figure 5 It is a model framework diagram of a business content scenario provided by an embodiment of the present application;

[0097] Figure 6 It is a Transformer model framework diagram provided by an embodiment of the present application. Detailed implementation manners

[0098] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be construed as limitations on the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.

[0099] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0100] In the following description, the terms "first / second / third" are merely used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0101] In the embodiments of the present application, the collection and processing of relevant data should strictly comply with the requirements of relevant national laws and regulations during actual application, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing behaviors within the scope authorized by laws and regulations and the personal information subject.

[0102] In the embodiments of the present application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other related parts to achieve a predetermined goal, and can be fully or partially implemented by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of the overall module or unit that includes the function of the module or unit.

[0103] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meaning as commonly understood by those skilled in the technical field to which the present application belongs. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0104] Before further elaborating on the embodiments of the present application, the nouns and terms involved in the embodiments of the present application are described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0105] 1) Semantic understanding model, which represents the computer's ability to understand natural language. It can enable the computer to more accurately understand human language, thereby helping people better interact with the computer. The core of the semantic understanding model is natural language processing technology, which can convert human language into a form that the computer can understand and is applied to multiple fields such as natural language question answering, machine translation, intelligent customer service, and intelligent search.

[0106] 2) Classifier. Classification is a very important method in data mining. The concept of classification is to learn a classification function or construct a classification model (i.e., the classifier we usually refer to) based on existing data. This function or model can map the data records in the database to a certain one of the given categories, so that it can be applied to data prediction. In short, the classifier is a general term for the methods of classifying samples in data mining and includes algorithms such as decision trees, logistic regression, naive Bayes, and neural networks.

[0107] 3) Risk control: That is, risk management and control, which means that risk managers take various measures and methods

[0108] 4) Token: A string of characters generated by the server as an authentication token for the client to make requests. After the first login, the server generates a Token and returns this Token to the client. After that, the client only needs to bring this Token to request data and does not need to bring the username and password again.

[0109] In the related art, the following two methods are usually adopted for text classification.

[0110] The first is the keyword matching scheme, that is, keyword matching is performed on the reviewed text, and the text that hits the keyword dictionary belongs to the corresponding category. However, on the one hand, this scheme requires continuous maintenance of the keyword dictionary, on the other hand, the misjudgment rate is relatively high, and there will be cases where multiple categories are hit, so it cannot be applied to downstream tasks.

[0111] The second method is to use a classification model for text classification, that is, classify and identify the entire reviewed text without distinguishing the importance gap of the content text. However, classifying the entire text, the result is not refined enough. For example, the descriptions of "suspected" and "confirmed" cannot be explicitly perceived, resulting in poor downstream strategy effects.

[0112] Based on the above analysis, the applicant finds that the text classification methods of related technologies cannot classify text efficiently and accurately. To solve the above problems, the embodiments of the present application provide a text classification method, which can improve the accuracy of text classification.

[0113] The embodiments of the present application provide a text classification method, a text classification model training method, a device, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the accuracy of text classification. The following describes an exemplary application of the electronic device provided by the embodiments of the present application. The electronic device provided by the embodiments of the present application can be implemented as various types of user terminals such as a laptop computer, a tablet computer, a desktop computer, a set-top box, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable game device), a smart phone, a smart speaker, a smart watch, a smart TV, a vehicle terminal, etc., or can be implemented as a server.

[0114] See Figure 1 , Figure 1 is a schematic diagram of the architecture of the text classification system 100 provided by the embodiments of the present application. To support a text classification application, the terminal 400 (graphical interface 410-1 is exemplarily shown) is connected to the server 200 through the network 300. The network 300 can be a wide area network, a local area network, or a combination of the two.

[0115] In some embodiments, the terminal 400 is used to send the text to be classified to the server 200 via the network 300; the server 200 is used to train a text classification model, classify the text to be classified submitted by the terminal 400 through the text classification model, and then send the classification result to the terminal 400 via the network 300.

[0116] In some embodiments, the terminal 400 is used to obtain the local text to be classified, call the pre-trained text classification model sent by the server 200 to classify the text to be classified, and display the classification result in the graphical display interface 410-1 of the terminal 400.

[0117] In some embodiments, the server 200 is configured to obtain an embedding vector sequence of a text sequence, encode the embedding vector sequence to obtain a text vector representing the text and a plurality of text element vectors respectively representing text elements of multiple description types; perform a fusion process on the text vector and the plurality of text element vectors to obtain a text fusion vector; and map the text fusion vector to a classification result of a text type related to the text classification task.

[0118] In some embodiments, the server 200 may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or may also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. The terminal 400 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected through wired or wireless communication means, which is not limited in the embodiments of the present application.

[0119] Exemplarily, the text classification method may be used in different application scenarios. For example: Search engine optimization: By analyzing the user's search keywords, the search engine can classify the search results into different categories, improve the relevance of the search results, and provide accurate search results for the user; Spam filtering: By classifying the email text, the system can automatically filter out spam emails to ensure that the emails received by the user are clearer and more orderly; Sentiment analysis: By analyzing the sentiment tendency in the text, it is determined whether the text is positive, negative or neutral to better understand the user feedback; News classification: By implementing the text classification method, the news can be automatically classified according to the theme, which helps media organizations and users find the news they are interested in more easily and improves the efficiency of information retrieval; Legal document classification: Text classification can be used to classify and organize legal documents to improve the efficiency of legal research and document management; Medical text classification: Text classification can be used to classify medical literature, etc., to help doctors better understand and utilize medical information.

[0120] Exemplarily, the text classification method provided in the embodiments of the present application can be applied in a vehicle-mounted scenario. By classifying the text data collected by the vehicle-mounted client, various operation instructions issued by the user through the vehicle-mounted client are identified, such as playing music, navigation, answering a call, etc.

[0121] The solution of the embodiment of the present application can be implemented through artificial intelligence. Artificial Intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines to enable machines to have the functions of perception, reasoning, and decision-making.

[0122] Artificial intelligence technology is an interdisciplinary subject that involves a wide range of fields, including both hardware-level and software-level technologies. Artificial intelligence basic technologies generally include, for example, sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models, also known as large models or foundation models, can be widely applied to downstream tasks in various directions of artificial intelligence after fine-tuning. Artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0123] For example, the solution of the embodiment of the present application can be implemented through natural language processing technology in artificial intelligence. Natural Language Processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between humans and computers in natural language. Natural language processing involves natural language, that is, the language people use in daily life, and is closely related to linguistics research; at the same time, it involves computer science and mathematics. The pre-trained model, an important technology for model training in the field of artificial intelligence, is developed from the large language model in the field of NLP. After fine-tuning, the large language model can be widely applied to downstream tasks. Natural language processing technology usually includes technologies such as text processing, semantic understanding, machine translation, robot question answering, and knowledge graphs.

[0124] As mentioned above, the electronic device for implementing the text classification method of the embodiment of the present application can be a terminal or a server. Below, an exemplary application will be described when the electronic device is implemented as the terminal 400. Refer to Figure 2A , Figure 2A is a schematic structural diagram of the terminal 400 provided by the embodiment of the present application, Figure 2AThe terminal 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. Each component in the terminal 400 is coupled together through a bus system 440. It can be understood that the bus system 440 is used to realize the connection and communication between these components. In addition to including a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 2A all kinds of buses are labeled as the bus system 440.

[0125] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0126] The user interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that assist the user in input, such as a keyboard, a mouse, a microphone, a touch screen display, a camera, other input buttons, and controls.

[0127] The memory 450 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disc drives, etc. The memory 450 optionally includes one or more storage devices that are physically located away from the processor 410.

[0128] The memory 450 includes volatile memory or non-volatile memory, and can also include both volatile and non-volatile memory. The non-volatile memory can be a read-only memory (ROM, Read Only Memory), and the volatile memory can be a random access memory (Random Access Memory, RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0129] In some embodiments, the memory 450 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, which are illustrated below.

[0130] An operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0131] A network communication module 452 for reaching other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include: Bluetooth, Wireless Fidelity (WiFi), and Universal Serial Bus (USB), etc.;

[0132] A presentation module 453 for enabling the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with the user interface 430 (such as a display screen, a speaker, etc.);

[0133] An input processing module 454 for detecting and translating one or more user inputs or interactions from one of one or more input devices 432.

[0134] In some embodiments, the device provided by the embodiments of the present application can be implemented in software. Figure 2A A text classification device 455 stored in the memory 450 is shown, which can be software in the form of a program and a plugin, etc., including the following software modules: a first acquisition module 4551, a first processing module 4552, a first encoding module 4553, a first fusion module 4554, and a first mapping module 4555. These modules are logical, and thus can be combined arbitrarily or further split according to the functions implemented. The functions of each module will be described below.

[0135] Exemplarily, the electronic device implementing the text classification model training method of the embodiments of the present application can be Figure 1 the server 200 in. Refer to Figure 2B , Figure 2B is a schematic structural diagram of the server 200 provided by the embodiments of the present application. Figure 2B The shown server 200 includes: at least one processor 210, a memory 250, and at least one network interface 220. Each component in the server 200 is coupled together through a bus system 240. It can be understood that the bus system 240 is used to realize the connection and communication between these components. The bus system 240 includes, in addition to a data bus, a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 2B all kinds of buses are labeled as the bus system 240.

[0136] The processor 210 may be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or any conventional processor, etc.

[0137] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disc drives, etc. The memory 250 optionally includes one or more storage devices that are physically located away from the processor 210.

[0138] The memory 250 includes volatile memory or non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 250 described in the embodiments of the present application is intended to include any suitable type of memory.

[0139] In some embodiments, the memory 250 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, which are illustrated below.

[0140] The operating system 251 includes system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, the core library layer, the driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0141] The network communication module 252 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 520. Exemplary network interfaces 520 include: Bluetooth, wireless compatibility certification (WiFi), and universal serial bus (USB), etc.;

[0142] In some embodiments, the device provided in the embodiments of the present application may be implemented in software. Figure 2B Shown is a text classification device 253 stored in the memory 250, which may be software in the form of programs and plugins, etc., and includes the following software modules: a second acquisition module 2531, a second processing module 2532, a second encoding module 2533, a second fusion module 2534, a second mapping module 2535, and an update module 2536. These modules are logical, and thus can be combined arbitrarily or further split according to the functions implemented. The functions of each module will be described below.

[0143] In some embodiments, the terminal or the server may implement the text classification method provided in the embodiments of the present application by running various computer-executable instructions or computer programs. For example, the computer-executable instructions may be commands at the microprogram level, machine instructions, or software instructions. The computer program may be a native program or a software module in the operating system; it may be a native application (APPlication, APP), that is, a program that needs to be installed in the operating system to run; it may also be a small program that can be embedded in any APP, that is, a program that only needs to be downloaded to the browser environment to run. In short, the above computer-executable instructions may be instructions in any form, and the above computer programs may be application programs, modules, or plugins in any form.

[0144] The exemplary applications and implementations of the electronic device provided in the embodiments of the present application will be combined to illustrate the text classification method provided in the embodiments of the present application.

[0145] Next, the text classification method provided in the embodiments of the present application will be described. As mentioned above, the electronic device implementing the text classification method in the embodiments of the present application may be a terminal or a server, or a combination of both. Therefore, the execution subject of each step will not be repeated hereinafter.

[0146] See Figure 3A , Figure 3A which is a schematic flowchart of the text classification method provided in the embodiments of the present application, and will be described in combination with the steps shown in Figure 3A it.

[0147] In step 101, the text to be classified is obtained.

[0148] In some embodiments, the text to be classified may be text obtained from any scenario. For example, text related to business operations, text related to sentiment classification, text related to intent recognition, text related to news content, etc.

[0149] In step 102, text elements of multiple description types related to the text classification task are extracted from the text.

[0150] In some embodiments, if the text to be classified is text related to business operations, the text classification task is to classify the business operations of the merchant; if the text to be classified is text related to sentiment classification, the text classification task is to classify the sentiment tendency, for example, like or dislike.

[0151] Exemplarily, if the text to be classified is text related to business operations, the text classification task is to classify the business operations of the merchant, and the text elements of multiple description types may be "business content description", "risk description", "confirmation description", etc.

[0152] In some embodiments, referring to Figure 3B , Figure 3B is the first process schematic diagram of the method for extracting text elements provided by the embodiments of the present application. Figure 3A Step 102 of Figure 3B can be implemented by steps 1021A to 1023A of

[0153] In step 1021A, multiple keyword libraries respectively associated with multiple description types are obtained.

[0154] Exemplarily, for the text elements "business content description", "risk description", and "confirmation description" in the above example, the keyword library 1 associated with the description type "business content description", the keyword library 2 associated with the description type "risk description", and the keyword library 3 associated with the description type "confirmation description" are respectively obtained.

[0155] In step 1022A, the keywords in the keyword library are compared with the text to obtain a comparison result.

[0156] In some embodiments, the keywords in the keyword library can be compared with the text through regular expressions to obtain a comparison result.

[0157] Exemplarily, the keywords in the keyword library 1 associated with the description type "business content description" in the above example are compared with the text to obtain a comparison result.

[0158] In step 1023A, in response to the comparison result indicating that the text contains the keyword, the part of the text containing the keyword is used as the text element of the description type associated with the keyword library.

[0159] Exemplarily, if the comparison result in the above example indicates that the text contains the keyword in the keyword library 1, the part of the text containing the keyword is used as the text element of the description type "business content description" associated with the keyword library 1.

[0160] In some embodiments, referring to Figure 3C , [[ID=3,6]] Figure 3C is the second process schematic diagram of the method for extracting text elements provided by the embodiments of the present application. Figure 3A Step 102 of Figure 3C can also be implemented by steps 1021B to 1022B of

[0161] In step 1021B, a prompt word is obtained, where the prompt word includes a text classification task and multiple description types to which the text elements related to the text classification task belong.

[0162] In some embodiments, the prompt words include the text classification task corresponding to the text to be classified, and the description types to which the text elements related to the text classification task belong.

[0163] Exemplarily, if the text to be classified obtained is text related to business operations, the prompt words include business operation classification and the description types to which the text elements related to the text classification task belong. For example, "description of business content", "risk description", "description of whether it is confirmed", etc.

[0164] In step 1022B, according to the prompt words, a pre-trained semantic understanding model is called to extract text elements of multiple description types from the text.

[0165] By extracting text elements of multiple description types related to the text classification task from the text, the present application embodiment fully explores the reference value of text elements of multiple description types related to the text classification task in text classification, and can improve the accuracy of text classification.

[0166] Continue to refer to Figure 3A and continue to describe based on step 102 above.

[0167] In step 103, at the positions of the text elements in the text, description type markers corresponding to the description types of the text elements are added to obtain a text sequence.

[0168] In some embodiments, at the start position and the end position of the text elements in the text respectively, start markers and end markers of the description types corresponding to the description types of the text elements are added to obtain a text sequence.

[0169] Exemplarily, an original text is: TTTTAAATTBBTTTTTCCC, where T is a text element unrelated to the description type, and the text elements of multiple description types related to the text classification task extracted from the text are A, B, and C respectively. Then, add the marker # before and after the A-type text element; add the marker & before and after the B-type text element; add the marker $ before and after the C-type text element; the obtained text sequence is: TTTT#AAA#TT&BB&TTTTT$CCC$.

[0170] In step 104, an embedding vector sequence of the text sequence is obtained.

[0171] In some embodiments, refer to Figure 3D , Figure 3D is a schematic flowchart of the method for obtaining the embedding vector sequence of the text sequence provided by the embodiment of the present application. Figure 3A Step 104 of Figure 3D can be implemented by steps 1041 to 1042 of

[0172] In step 1041 , a start tag is added at the start position of the text sequence, and an end tag is added at the end position of the text sequence to form a tag sequence.

[0173] For example, see Figure 4A , Figure 4A This is a network structure diagram provided by an embodiment of the present application. The tag sequence corresponding to the text sequence in the above example is "[CLS]T1 T2 T3…#Ta…Tb#…&Ti…Tj&…$Tm…Tn$…[SEP]"; where Ta…Tb refers to Class A text elements, Ti…Tj refers to Class B text elements, and Tm…Tn refers to Class C text elements.

[0174] In step 1042, embedding processing is performed on the tag sequence to obtain an embedding vector sequence of the text sequence.

[0175] In some embodiments, see Figure 3E , Figure 3E 1 is a flow chart of a method for generating an embedding vector sequence of a text sequence provided in an embodiment of the present application. Figure 3D Step 1042 can be achieved by Figure 3E Steps 10421 to 10423 are implemented as described below.

[0176] In step 10421, semantic embedding processing is performed on the elements in the tag sequence to obtain a symbol embedding vector corresponding to each element in the tag sequence, where the elements include text elements, start tags, and end tags.

[0177] In some embodiments, the obtained symbol embedding vector corresponding to each element in the tag sequence is a vector representation of the symbol corresponding to each element in the tag sequence.

[0178] In step 10422, role embedding processing is performed on the elements in the tag sequence to obtain a role embedding vector corresponding to each element in the tag sequence, wherein the role embedding vectors of text elements with different description type tags are different.

[0179] In some embodiments, the obtained role embedding vector corresponding to each element in the tag sequence is a vector representation of a different type corresponding to each element in the tag sequence.

[0180] In some embodiments, the role embedding vectors of text elements without a description type tag are different from the role embedding vectors of text elements with a description type tag.

[0181] For example, see Figure 4B , Figure 4B Schematic diagram of the role embedding vector provided in an embodiment of the present application. Figure 4B E shown inO Characterize the role of ordinary tokens, E I Characterize the role of type A text elements, E R Characterize the role of type B text elements, E C Characterize the role of type C text elements.

[0182] In step 10423, an embedding vector sequence is generated based on the symbol embedding vector and the role embedding vector of each element.

[0183] In some embodiments, the sum of the symbol embedding and the role embedding of each element is determined, and the sums of each element are combined into an embedding vector sequence.

[0184] In some embodiments, an embedding vector sequence is generated based on the symbol embedding vector and the role embedding vector of each element, which can be implemented in the following manner:

[0185] First, perform segment embedding processing on the elements in the token sequence to obtain the segment embedding vectors corresponding to the token sequence. Among them, the segment embedding vectors characterize the embedding vectors of the segments to which the elements belong in the text, and the segment embedding vectors corresponding to different segments in the text are different.

[0186] Then, perform position embedding processing on the elements in the token sequence to obtain the position embedding vectors corresponding to the token sequence. Among them, the segment embedding vectors corresponding to different texts are different.

[0187] Finally, determine the sum of the symbol embedding vector, the role embedding vector, the segment embedding vector, and the position embedding vector of each element, and combine the sums corresponding to each element into an embedding vector sequence.

[0188] Exemplarily, refer to Figure 4C , Figure 4C is a schematic diagram of generating an embedding vector sequence provided by an embodiment of the present application. Take Figure 4C the sum of the symbol embedding vector, the segment embedding vector, the position embedding vector, and the role embedding vector in the first column shown as the embedding vector of the first element, and so on, and combine the sums corresponding to each element into an embedding vector sequence.

[0189] Based on text elements of multiple description types related to the text classification task extracted from the text to be classified, in the stage of performing embedding processing in the related technology, the present application embodiment adds role embedding vectors generated based on text elements of multiple description types related to the text classification task. Compared with the embedding vector sequence generated by the symbol embedding vector, the segment embedding vector, and the position embedding vector in the related technology, it is equivalent to introducing prior information related to the classification task in the classification process. Compared with the related technology that simply classifies based on the text itself, the accuracy is improved.

[0190] Continue to refer to Figure 3A , and the description continues from step 104 above.

[0191] In step 105, the embedded vector sequence is encoded to obtain a text vector representing the text and multiple text element vectors respectively representing multiple text elements of various description types.

[0192] In some embodiments, the embedded vector sequence can be encoded by a semantic understanding model to obtain a text vector representing the text and multiple text element vectors respectively representing multiple text elements of various description types.

[0193] Exemplarily, the embedded vector sequence can be encoded by a Bidirectional Encoder Representations from Transformer (BERT) to obtain a text vector representing the text and multiple text element vectors respectively representing multiple text elements of various description types.

[0194] In some embodiments, refer to Figure 3F , Figure 3F which is a schematic flowchart of the method for obtaining a text vector representing the text and text element vectors provided in the embodiments of the present application. Figure 3A Step 105 of Figure 3F can be implemented by steps 1051 to 1053 of

[0195] In step 1051, a pre-trained semantic understanding model is called to encode the embedded vector sequence to obtain an encoded sequence, where the encoded sequence includes the embedded vectors corresponding to the text elements in the text sequence and the embedded vector corresponding to the start token.

[0196] In some embodiments, the encoded sequence includes the embedded vectors corresponding to the text elements in the text sequence, the embedded vector corresponding to the start token, and the embedded vector corresponding to the end token.

[0197] In step 1052, the embedded vector corresponding to the start token is used as the text vector.

[0198] In some embodiments, the embedded vectors corresponding to the start token and the end token are used as the text vector.

[0199] In step 1053, the embedded vector corresponding to the text element with a description type token is used as the text element vector representing the description type corresponding to the description type token.

[0200] Continue to refer to Figure 3A , and the description continues from step 105 above.

[0201] In step 106, the text vector and multiple text element vectors are fused to obtain a text fusion vector.

[0202] In some embodiments, referring to Figure 3G , Figure 3G is the first process schematic diagram of the method for obtaining a text fusion vector provided by the embodiments of the present application. Figure 3A Step 106 of Figure 3G can be implemented by steps 1061 to 1063 of

[0203] In step 1061, average pooling is performed on multiple text element vectors to obtain an average pooling vector.

[0204] For example, average pooling is performed on the type-A text elements in the above example to obtain the average pooling vector corresponding to the type-A text elements; average pooling is performed on the type-B text elements to obtain the average pooling vector corresponding to the type-B text elements; average pooling is performed on the type-C text elements to obtain the average pooling vector corresponding to the type-C text elements.

[0205] In step 1062, the average pooling vector is weighted to obtain the weighted vectors corresponding to multiple text element vectors.

[0206] For example, the average pooling vector corresponding to the type-A text elements in the above example is weighted to obtain the weighted vector corresponding to the type-A text elements; the average pooling vector corresponding to the type-B text elements is weighted to obtain the weighted vector corresponding to the type-B text elements; the average pooling vector corresponding to the type-C text elements is weighted to obtain the weighted vector corresponding to the type-C text elements.

[0207] In step 1063, the text vector and the weighted vectors corresponding to multiple text element vectors are concatenated to obtain a text fusion vector.

[0208] By performing average pooling and weighting on the text element vectors, the embodiments of the present application assign different weights to the text elements related to the text classification task, and can fully consider the proportion of text elements of different description types during the process of classifying the text to be classified. Compared with the related art that treats each text element without distinction, it can make the classification more accurate.

[0209] Continuing to refer to Figure 3A , the description continues with step 106 above.

[0210] In step 107, the text fusion vector is mapped to the classification result of the text type related to the text classification task.

[0211] In some embodiments, a classifier maps the text fusion vector to a classification result of a text type related to the text classification task.

[0212] In some embodiments, referring to Figure 3H , Figure 3H is a schematic flowchart of a process for obtaining a text classification result provided by an embodiment of the present application. Figure 3A Step 107 of Figure 3H can be implemented by steps 1071 to 1072 of

[0213] In step 1071, the text fusion vector is mapped to a probability distribution of multiple text types related to the text classification task.

[0214] In some embodiments, a classifier maps the text fusion vector to a probability distribution of multiple text types related to the text classification task.

[0215] For example, if the text classification task is business operation classification, the probability distribution result of mapping the text fusion vector to multiple text types related to the text classification task by a classifier may be that the probability value of operating clothing is 70% and the probability value of operating catering is 20%.

[0216] In step 1072, the text type corresponding to the maximum probability in the probability distribution is used as the classification result of the text type related to the text classification task.

[0217] For example, in the above example, the probability distribution result shows that the probability value of operating clothing is 70% and the probability value of operating catering is 20%. Then, clothing is used as the classification result of the text type related to the text classification task.

[0218] Next, a text classification model training method provided by an embodiment of the present application will be described. The electronic device for implementing the text classification model training method of the embodiment of the present application can be a terminal or a server, or a combination of both. Therefore, the execution subject of each step will not be repeated hereinafter.

[0219] Referring to Figure 3I , Figure 3I is a schematic flowchart of the text classification model training method provided by an embodiment of the present application, and will be described in conjunction with the steps shown in Figure 3I shown below.

[0220] In some embodiments, the text classification model includes a semantic understanding model and a classifier. Among them, the semantic understanding model can be a pre-trained one or an untrained one, which can convert human language into a form that can be understood by a computer; the classifier is a trained classification model that uses given categories and known training data to learn classification rules and the classifier, and then classifies (or predicts) unknown data.

[0221] In step 201, a text sample and the actual classification result of the text sample are obtained.

[0222] In some embodiments, the text sample can be text collected from any scenario. For example, text related to business operations, text related to sentiment classification, text related to intent recognition, text related to news content, etc.

[0223] In step 202, text elements of multiple description types related to the text classification task are extracted from the text sample.

[0224] In some embodiments, if the text sample is text related to business operations, the text classification task is to classify the business operations of the merchant; if the text sample is text related to sentiment classification, the text classification task is to classify the sentiment tendency, for example, like or dislike.

[0225] Exemplarily, if the text sample is text related to business operations, the text classification task is to classify the business operations of the merchant, and the text elements of multiple description types can be "description of business content", "risk description", "confirmation description", etc.

[0226] In some embodiments, refer to Figure 3J , Figure 3J is the third process schematic diagram of the method for extracting text elements provided by the embodiments of the present application. Figure 3I Step 202 of Figure 3J can be implemented by steps 2021A to 2023A of

[0227] In step 2021A, multiple keyword libraries respectively associated with multiple description types are obtained.

[0228] Exemplarily, for the text elements "description of business content", "risk description", "confirmation description" in the above example, a keyword library 1 associated with the description type "description of business content", a keyword library 2 associated with the description type "risk description", and a keyword library 3 associated with the description type "confirmation description" are respectively obtained.

[0229] In step 2022A, the keywords in the keyword library are compared with the text sample to obtain a comparison result.

[0230] In some embodiments, the keywords in the keyword library can be compared with the text sample through regular expressions to obtain a comparison result.

[0231] Exemplarily, the keywords in the keyword library 1 associated with the description type "business content description" in the above example are compared with the text to obtain a comparison result.

[0232] In step 2023A, in response to the comparison result indicating that the text sample contains keywords, the part of the text sample containing the keywords is used as the text element of the description type associated with the keyword library.

[0233] Exemplarily, if the comparison result in the above example is that the text contains the keywords in keyword library 1, then the part of the text containing the keywords is used as the text element of the description type "business content description" associated with keyword library 1.

[0234] In some embodiments, referring to Figure 3K , Figure 3K is the fourth process schematic diagram of the method for extracting text elements provided by the embodiments of the present application. Figure 3I Step 202 of Figure 3K can also be implemented through steps 2021B to 2022B of

[0235] In step 2021B, a prompt word is obtained, where the prompt word includes a text classification task and various description types to which the text elements related to the text classification task belong.

[0236] In some embodiments, the prompt word includes the text classification task corresponding to the text to be classified, and the description types to which the text elements related to the text classification task belong.

[0237] Exemplarily, if the text to be classified obtained is text related to business operations, the prompt word includes business operation classification and the description types to which the text elements related to the text classification task belong, for example, "business content description", "risk description", "confirmation description", etc.

[0238] In step 2022B, according to the prompt word, a pre-trained semantic understanding model is called to extract text elements of various description types from the text sample.

[0239] By extracting text elements of various description types related to the text classification task from the text sample, the embodiments of the present application fully explore the reference value of the text elements of various description types related to the text classification task in the text sample, and can improve the accuracy of text classification.

[0240] Continuing to refer to Figure 3I , the description continues with step 202 above.

[0241] In step 203, at the positions of the text elements corresponding in the text sample, description type tags corresponding to the description types of the text elements are added to obtain a text sample sequence.

[0242] In some embodiments, at the start position and the end position of the text element corresponding in the text sample respectively, a start tag and an end tag of the description type corresponding to the description type of the text element are added to obtain a text sample sequence.

[0243] For example, an original text sample is: TTTTAAATTBBTTTTTCCC, and the text elements of multiple description types related to the text classification task extracted from the text sample are A, B, and C respectively. Then, add the tag # before and after the text element of type A; add the tag & before and after the text element of type B; add the tag $ before and after the text element of type C; the obtained text sample sequence is: TTTT#AAA#TT&BB&TTTTT$CCC$.

[0244] In step 204, an embedding vector sequence of the text sample sequence is obtained.

[0245] In some embodiments, refer to Figure 3L , Figure 3L which is a schematic flowchart of the method for obtaining the embedding vector sequence of the text sample sequence provided by the embodiments of the present application. Figure 3I Step 204 of Figure 3L can be implemented by steps 2041 to 2042 of

[0246] In step 2041, a start tag is added at the start position of the text sample sequence, and an end tag is added at the end position of the text sample sequence to form a tag sequence.

[0247] For example, refer to Figure 4A , the tag sequence corresponding to the text sample sequence in the above example is "[CLS]T1 T2T3…#Ta…Tb#…&Ti…Tj&…$Tm…Tn$…[SEP]"; where Ta…Tb refers to the text element of type A, Ti…Tj refers to the text element of type B, and Tm…Tn refers to the text element of type C.

[0248] In step 2042, the tag sequence is embedded to obtain an embedding vector sequence of the text sample sequence.

[0249] In some embodiments, refer to Figure 3M , Figure 3M which is a second schematic flowchart of the method for generating the embedding vector sequence of the text sample sequence provided by the embodiments of the present application. Figure 3L Step 2042 of Figure 3MIt is implemented by steps 20421 to 20423, which are specifically described below.

[0250] In step 20421, semantic embedding processing is performed on the elements in the token sequence to obtain symbol embedding vectors corresponding to each element in the token sequence, where the elements include text elements, start tokens, and end tokens.

[0251] In some embodiments, the symbol embedding vectors corresponding to each element in the obtained token sequence are the vector representations of the symbols corresponding to each element in the token sequence.

[0252] In step 20422, role embedding processing is performed on the elements in the token sequence to obtain role embedding vectors corresponding to each element in the token sequence, where the role embedding vectors of text elements with different description type tokens are different.

[0253] In some embodiments, the role embedding vectors corresponding to each element in the obtained token sequence are the vector representations of different types corresponding to each element in the token sequence.

[0254] In some embodiments, the role embedding of text elements without description type tokens is different from the role embedding vectors of text elements with description type tokens.

[0255] In step 20423, an embedding vector sequence is generated based on the symbol embedding vectors and role embedding vectors of each element.

[0256] In some embodiments, the sum of the symbol embedding and role embedding of each element is determined, and the sums of each element are used to form an embedding vector sequence.

[0257] In some embodiments, generating an embedding vector sequence based on the symbol embedding vectors and role embedding vectors of each element can be achieved in the following way:

[0258] First, segment embedding processing is performed on the elements in the token sequence to obtain segment embedding vectors corresponding to the token sequence, where the segment embedding vectors represent the embedding vectors of the segments to which the elements belong in the text, and the segment embedding vectors corresponding to different segments in the text are different.

[0259] Then, position embedding processing is performed on the elements in the token sequence to obtain position embedding vectors corresponding to the token sequence, where the segment embedding vectors corresponding to different texts are different.

[0260] Finally, the sum of the symbol embedding vector, role embedding vector, segment embedding vector, and position embedding vector of each element is determined, and the sums corresponding to each element are used to form an embedding vector sequence.

[0261] Exemplarily, refer to Figure 4C ,Figure 4C This is a schematic diagram of generating an embedded vector sequence provided by an embodiment of the present application. Add the symbol embedded vector, segment embedded vector, position embedded vector, and role embedded vector in the first column shown in Figure 4C as the embedded vector of the first element, and so on. The sum corresponding to each element forms an embedded vector sequence.

[0262] Based on text elements of multiple description types related to the text classification task extracted from the text to be classified, in the stage of performing embedding processing in the related technology, the embodiment of the present application adds a role embedded vector generated based on text elements of multiple description types related to the text classification task. Compared with the embedded vector sequence generated by the symbol embedded vector, segment embedded vector, and position embedded vector in the related technology, it is equivalent to introducing prior information related to the classification task during the classification process. Compared with the related technology that simply classifies based on the text itself, the accuracy is improved.

[0263] Continue to refer to Figure 3I and continue to describe based on step 204 above.

[0264] In step 205, the semantic understanding model encodes the embedded vector sequence to obtain a text vector representing the text sample and multiple text element vectors respectively representing text elements of multiple description types.

[0265] In some embodiments, the semantic understanding model may be a Bidirectional Encoder Representations from Transformer (BERT) based on Transformer.

[0266] In some embodiments, refer to Figure 3N , Figure 3N This is a schematic flowchart of a method for obtaining a text vector representing a text sample and text element vectors provided by an embodiment of the present application. Figure 3I Step 205 of Figure 3N can be implemented by steps 2051 to 2053 of

[0267] In step 2051, call the semantic understanding model to encode the embedded vector sequence to obtain an encoded sequence, where the encoded sequence includes the embedded vectors corresponding to the text elements in the text sample sequence and the embedded vector corresponding to the start token.

[0268] In some embodiments, the encoded sequence includes the embedded vectors corresponding to the text elements in the text sample sequence, the embedded vector corresponding to the start token, and the embedded vector corresponding to the end token.

[0269] In step 2052, the embedding vector corresponding to the start token is used as the text vector.

[0270] In some embodiments, the embedding vector corresponding to the start token and the embedding vector corresponding to the end token are used as the text vector.

[0271] In step 2053, the embedding vector corresponding to the text element with the description type token is used as the text element vector representing the description type corresponding to the description type token.

[0272] Continue to refer to Figure 3I , and the steps of 205 above are continued for description.

[0273] In step 206, the text vector and multiple text element vectors are fused to obtain a text fusion vector.

[0274] In some embodiments, refer to Figure 3O , Figure 3O is the second process schematic diagram of the method for obtaining the text fusion vector provided by the embodiments of the present application. Figure 3I Step 206 of Figure 3O can be implemented by steps 2061 to 2063 of

[0275] In step 2061, average pooling is performed on multiple text element vectors to obtain an average pooling vector.

[0276] For example, average pooling is performed on the type A text elements in the above example to obtain the average pooling vector corresponding to the type A text elements; average pooling is performed on the type B text elements to obtain the average pooling vector corresponding to the type B text elements; average pooling is performed on the type C text elements to obtain the average pooling vector corresponding to the type C text elements.

[0277] In step 2062, the average pooling vector is weighted to obtain the weighted vectors corresponding to multiple text element vectors.

[0278] For example, the average pooling vector corresponding to the type A text elements in the above example is weighted to obtain the weighted vector corresponding to the type A text elements; the average pooling vector corresponding to the type B text elements is weighted to obtain the weighted vector corresponding to the type B text elements; the average pooling vector corresponding to the type C text elements is weighted to obtain the weighted vector corresponding to the type C text elements.

[0279] In step 2063, the text vector and the weighted vectors corresponding to multiple text element vectors are concatenated to obtain a text fusion vector.

[0280] In the embodiments of the present application, by performing average pooling and weighting on the text element vectors, different weights are assigned to the text elements related to the text classification task, so that the proportion of text elements of different description types can be fully considered during the classification of the text to be classified. Compared with the related art where all text elements are processed without distinction, the classification can be made more accurate.

[0281] Continue to refer to Figure 3I , and the description continues with step 206 above.

[0282] In step 207, the text fusion vector is mapped by a classifier to a predicted classification result of a text type related to the text classification task.

[0283] In some embodiments, refer to Figure 3P , Figure 3P is a schematic flowchart of obtaining the predicted classification result of the text sample provided by the embodiments of the present application. Figure 3I Step 207 of Figure 3P can be implemented through step 2071 to step 2072 of

[0284] In step 2071, the text fusion vector is mapped by a classifier to a probability distribution of multiple text types related to the text classification task.

[0285] For example, if the text classification task is business operation classification, the probability distribution result of mapping the text fusion vector by a classifier to multiple text types related to the text classification task may be that the probability value of operating clothing is 70% and the probability value of operating catering is 20%.

[0286] In step 2072, the text type corresponding to the maximum probability in the probability distribution is used as the predicted classification result of the text type related to the text classification task.

[0287] For example, in the above example, the probability distribution result shows that the probability value of operating clothing is 70% and the probability value of operating catering is 20%. Then, clothing is used as the classification result of the text type related to the text classification task.

[0288] Continue to refer to Figure 3I , and the description continues with step 207 above.

[0289] In step 208, according to the difference between the predicted classification result and the actual classification result, the parameters of the semantic understanding model and the classifier are updated reversely.

[0290] Next, the exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0291] The text classification method provided by the embodiments of this application can be specifically applied to the text classification for detecting transaction authenticity. In the field of risk control, comprehensively identifying the business content from various information sources of merchants is an important way to detect transaction authenticity. On the one hand, by obtaining the text submitted by merchants, the business content of the merchants is identified; on the other hand, by verifying the electronic payment transactions of customers; comparing the identified business content of the merchants with the business content involved in the transaction content of electronic payments as a reference for transaction authenticity.

[0292] However, due to the large number of merchants, the cost of all manual verification is too high. It is a common practice to train a model based on the text of some audited merchants as seed data. Therefore, it is necessary to further extract the audited text to predict the actual business content of the corresponding merchants.

[0293] Since there are many dimensions in the audited text and there is no label for the business scope, it is necessary to infer the business scope based on relevant descriptions. The embodiments of this application design an inference model for audited text combined with element prompts. The effect is significantly improved compared with the original basic model, and the effects of the overall downstream tasks are improved to varying degrees.

[0294] The text classification scheme provided by the embodiments of this application is as follows: (1) Retrieve the text of the business content description, risk description, and confirmation description in the audited text; (2) Distinguish and mark the three different elements, that is, attach prompt words when inputting the original text; (3) In the embedding process of the BERT model, introduce role embedding and train it with the fine-tuning of the model; combine the output of the entire hidden layer and the average feature output of the three elements; and set different relationship weights (self-learning variables) for the three elements to output the classification result. The following is a specific description.

[0295] First, for the text of the three elements of "business content description", "risk description", and "confirmation description", the text elements are extracted by combining the methods of regular expressions and keyword matching.

[0296] Then, mark the text corresponding to the three elements in the original input text. The "business content description", "risk description", and "confirmation description" are respectively denoted as A, B, and C, and the other original text is denoted as T.

[0297] For example, an original text is: TTTTAAATTBBTTTTTCCC; add marks #, &, $ before and after A, B, and C respectively; then the processed original text is: TTTT#AAA#TT&BB&TTTTT$CCC$.

[0298] Finally, execute the main model process to output the classification result. See Figure 5 , Figure 5It is a model framework diagram of the business content scenario provided by the embodiments of the present application. Token processing is performed on the processed marked text. In the embedding link, role embedding processing is introduced for the processed Token. After the Token is input into the bidirectional transformer model, the representation CLS of the entire input is output. At the same time, the average pooling of the representations of the three types of element texts is concatenated with CLS, and the concatenated representation is input into the softmax layer, and finally the inferred category is output. Next, the key steps involved in the model of the transaction authenticity detection scenario provided by the embodiments of the present application are specifically described.

[0299] First, obtain the marked sequence.

[0300] Processing result: "[CLS]T1 T2 T3…#Ta…Tb#…&Ti…Tj&…$Tm…Tn$…[SEP]"; where Ta…Tb refers to "business content description", Ti…Tj refers to "risk description", and Tm…Tn refers to "confirmation description".

[0301] Second, generate role embeddings.

[0302] Each Token has a corresponding role representation, thus forming the role matrix of the entire input text (dimension: max_length*hidden_size). This matrix is a learnable variable, which is randomly initialized and trained together with the downstream task later. See Figure 4B , Figure 4B It is a schematic diagram of role embedding provided by the embodiments of the present application. Figure 4B The E shown in O represents the role of ordinary Token, and E I represents the role of "business content description", and E R represents the role of "risk description", and E C represents the role of "confirmation description".

[0303] Then, generate the sequence of embedding vectors.

[0304] See Figure 4C , Figure 4C It is a schematic diagram of the input of the BERT model with role embedding provided by the embodiments of the present application. Figure 4C The input of the BERT model shown includes the symbol embedding vector sequence, the segment embedding vector sequence, the position embedding vector sequence, and the role embedding vector sequence. The symbol embedding vector, segment embedding vector, position embedding vector, and role embedding vector corresponding to each element are respectively summed up to obtain the output result as the embedding vector sequence.

[0305] Next, perform feature learning.

[0306] Input the embedding vector sequence obtained in the previous step into a bidirectional Transformer for feature learning. Refer to Figure 6 , Figure 6 which is the Transformer model framework diagram provided by an embodiment of this application.

[0307] Again, perform fusion encoding.

[0308] After all Tokens are input into the ​ bidirectional Transformer model shown, the representation CLS of the overall input is output. At the same time, average pooling is performed on the representations of "business content description", "risk description", and "confirmation description" respectively, and different weights are set. Finally, they are concatenated with CLS.

[0309] H^ = concat(CLS, WI*AvgI, WR*AvgR, WC*AvgC)

[0310] where AvgI, AvgR, and AvgC are the average pooling representations corresponding to "business content description", "risk description", and "confirmation description" respectively, and WI, WR, and WC are the weights corresponding to the three representations. This weight is a variable and self-learns as the model is trained.

[0311] Finally, perform softmax output.

[0312] The concatenated representation is input into the softmax layer to output the inferred category.

[0313] Softmax is a common multi-classifier used to predict the probabilities of an object belonging to each category. The formula is as follows:

[0314]

[0315] In an embodiment of this application, by extracting text elements of multiple description types related to the text classification task from the text to be classified; and adding description type tags corresponding to the description types of the text elements at the positions of the text elements in the text to obtain a text sequence; obtaining an embedding vector sequence of the text sequence; encoding the embedding vector sequence to obtain a text vector representing the text and multiple text element vectors respectively representing the text elements of multiple description types; performing fusion processing on the text vector and the multiple text element vectors to obtain a text fusion vector; mapping the text fusion vector to a classification result of a text type related to the text classification task. By combining text elements of different description types to classify the text, the accuracy of text classification can be improved.

[0316] Next, continue to describe the exemplary structure of the implementation of the text classification device 455 provided by an embodiment of this application as software modules. In some embodiments, such as ​As shown, the software modules stored in the text classification device 455 of the memory 450 may include:

[0317] A first acquisition module 4551, configured to acquire the text to be classified; acquire the embedding vector sequence of the text sequence.

[0318] A first processing module 4552, configured to extract text elements of multiple description types related to the text classification task from the text; add description type tags corresponding to the description types of the text elements at the positions of the text elements in the text to obtain a text sequence.

[0319] A first encoding module 4553, configured to encode the embedding vector sequence to obtain a text vector representing the text and multiple text element vectors respectively representing the text elements of multiple description types.

[0320] A first fusion module 4554, configured to perform a fusion process on the text vector and the multiple text element vectors to obtain a text fusion vector.

[0321] A first mapping module 4555, configured to map the text fusion vector to a classification result of a text type related to the text classification task.

[0322] In some embodiments, the first processing module 4552 is further configured to acquire multiple keyword libraries respectively associated with multiple description types; compare the keywords in the keyword libraries with the text to obtain a comparison result; in response to the comparison result indicating that the text contains a keyword, use the part of the text containing the keyword as the text element of the description type associated with the keyword library.

[0323] In some embodiments, the first module 4552 is further configured to acquire a prompt word, where the prompt word includes the text classification task and multiple description types to which the text elements related to the text classification task belong; call a pre-trained semantic understanding model according to the prompt word to extract text elements of multiple description types from the text.

[0324] In some embodiments, the first fusion module 4554 is further configured to perform average pooling processing on the multiple text element vectors to obtain an average pooling vector; perform weighting processing on the average pooling vector to obtain a weighted vector corresponding to the multiple text element vectors; splice the text vector and the weighted vector corresponding to the multiple text element vectors to obtain a text fusion vector.

[0325] In some embodiments, the first processing module 4552 is further configured to add a start tag at the start position of the text sequence and add an end tag at the end position of the text sequence to form a tagged sequence; perform embedding processing on the tagged sequence to obtain the embedding vector sequence of the text sequence.

[0326] In some embodiments, the first processing module 4552 is further configured to perform semantic embedding processing on the elements in the token sequence to obtain a symbol embedding vector corresponding to each element in the token sequence, where the elements include text elements, a start token, and an end token; perform role embedding processing on the elements in the token sequence to obtain a role embedding vector corresponding to each element in the token sequence, where the role embedding vectors of text elements with different description type tokens are different; and generate an embedding vector sequence based on the symbol embedding vector and the role embedding vector of each element.

[0327] In some embodiments, the first processing module 4552 is further configured to perform segment embedding processing on the elements in the token sequence to obtain a segment embedding vector corresponding to the token sequence, where the segment embedding vector represents the embedding vector of the segment to which the element belongs in the text, and the segment embedding vectors corresponding to different segments in the text are different; perform position embedding processing on the elements in the token sequence to obtain a position embedding vector corresponding to the token sequence, where the segment embedding vectors corresponding to different texts are different; determine the sum of the symbol embedding vector, the role embedding vector, the segment embedding vector, and the position embedding vector of each element, and form an embedding vector sequence with the sum corresponding to each element.

[0328] In some embodiments, the first encoding module 4553 is further configured to call a pre-trained semantic understanding model to encode the embedding vector sequence to obtain an encoded sequence, where the encoded sequence includes the embedding vector corresponding to the text element in the text sequence and the embedding vector corresponding to the start token; use the embedding vector corresponding to the start token as the text vector; and use the embedding vector corresponding to the text element with a description type token as the text element vector representing the description type corresponding to the description type token.

[0329] In some embodiments, the first mapping module 4555 is further configured to map the text fusion vector to a probability distribution of multiple text types related to the text classification task; and use the text type corresponding to the maximum probability in the probability distribution as the classification result of the text type related to the text classification task.

[0330] Next, the implementation of the text classification model training device 253 provided in the embodiments of the present application as an exemplary structure of software modules will be continued. In some embodiments, as ​ shown, the software modules stored in the text classification device 253 in the memory 250 may include:

[0331] The second acquisition module 2531 is configured to acquire a text sample and the actual classification result of the text sample; and acquire an embedding vector sequence of the text sample sequence.

[0332] The second processing module 2532 is configured to extract text elements of multiple description types related to the text classification task from the text sample; add description type tags corresponding to the description types of the text elements at the positions of the text elements in the text sample to obtain a text sample sequence.

[0333] The second encoding module 2533 is configured to encode the embedded vector sequence through a semantic understanding model to obtain a text vector representing the text sample and multiple text element vectors respectively representing the text elements of multiple description types.

[0334] The second fusion module 2534 is configured to perform a fusion process on the text vector and the multiple text element vectors to obtain a text fusion vector.

[0335] The second mapping module 2535 is configured to map the text fusion vector to a predicted classification result of a text type related to the text classification task through a classifier.

[0336] The update module 2536 is configured to reversely update the parameters of the semantic understanding model and the classifier according to the difference between the predicted classification result and the actual classification result.

[0337] In some embodiments, the second processing module 2532 is further configured to obtain multiple keyword libraries respectively associated with multiple description types; compare the keywords in the keyword libraries with the text sample to obtain a comparison result; in response to the comparison result indicating that the text sample contains a keyword, use the part of the text sample containing the keyword as the text element of the description type associated with the keyword library.

[0338] In some embodiments, the second processing module 2532 is further configured to obtain a prompt word, where the prompt word includes the text classification task and multiple description types to which the text elements related to the text classification task belong; call the semantic understanding model according to the prompt word to extract text elements of multiple description types from the text sample.

[0339] In some embodiments, the second fusion module 2534 is further configured to perform average pooling processing on the multiple text element vectors to obtain an average pooling vector; perform a weighting process on the average pooling vector to obtain a weighted vector corresponding to the multiple text element vectors; splice the text vector and the weighted vector corresponding to the multiple text element vectors to obtain a text fusion vector.

[0340] In some embodiments, the second processing module 2532 is further configured to add a start tag at the start position of the text sample sequence and add an end tag at the end position of the text sample sequence to form a tagged sequence; perform an embedding process on the tagged sequence to obtain an embedded vector sequence of the text sample sequence.

[0341] In some embodiments, the second processing module 2532 is further configured to perform semantic embedding processing on the elements in the token sequence to obtain a symbol embedding vector corresponding to each element in the token sequence, where the elements include text elements, start tokens, and end tokens; perform role embedding processing on the elements in the token sequence to obtain a role embedding vector corresponding to each element in the token sequence, where the role embedding vectors of text elements with different description type tokens are different; and generate an embedding vector sequence based on the symbol embedding vector and the role embedding vector of each element.

[0342] In some embodiments, the second processing module 2532 is further configured to perform segment embedding processing on the elements in the token sequence to obtain a segment embedding vector corresponding to the token sequence, where the segment embedding vector represents the embedding vector of the segment to which the element belongs in the text, and the segment embedding vectors corresponding to different segments in the text are different; perform position embedding processing on the elements in the token sequence to obtain a position embedding vector corresponding to the token sequence, where the segment embedding vectors corresponding to different texts are different; determine the sum of the symbol embedding vector, the role embedding vector, the segment embedding vector, and the position embedding vector of each element, and form an embedding vector sequence with the sum corresponding to each element.

[0343] In some embodiments, the second encoding module 2533 is further configured to call a semantic understanding model to encode the embedding vector sequence to obtain an encoded sequence, where the encoded sequence includes the embedding vector corresponding to the text element in the text sample sequence and the embedding vector corresponding to the start token; use the embedding vector corresponding to the start token as the text vector; and use the embedding vector corresponding to the text element with a description type token as the text element vector representing the description type corresponding to the description type token.

[0344] In some embodiments, the second mapping module 2535 is further configured to map the text fusion vector to a probability distribution of text types related to the text classification task through a classifier; and use the text type corresponding to the maximum probability in the probability distribution as the predicted classification result of the text type related to the text classification task.

[0345] An embodiment of the present application provides a computer program product, which includes a computer program or computer executable instructions, and the computer program or computer executable instructions are stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instructions from the computer-readable storage medium, and the processor executes the computer executable instructions, so that the electronic device executes the above text classification method or text classification model training method of the embodiments of the present application.

[0346] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, in which computer-executable instructions or computer programs are stored. When the computer-executable instructions or computer programs are executed by a processor, the processor will be caused to execute the text classification method or the text classification model training method provided by the embodiments of the present application. For example, ​ the text classification method shown.

[0347] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0348] In some embodiments, the computer-executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as an independent program or being deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0349] As an example, the computer-executable instructions may or may not correspond to a file in the file system, may be stored as part of a file that stores other programs or data. For example, stored in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program being discussed, or stored in multiple cooperating files (for example, files that store one or more modules, subroutines, or code portions).

[0350] As an example, the computer-executable instructions may be deployed to execute on one electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed at multiple locations and interconnected by a communication network.

[0351] In summary, in the embodiments of the present application, text elements of multiple description types related to the text classification task are extracted from the text to be classified; and description type tags corresponding to the description types of the text elements are added at the positions of the text elements in the text to obtain a text sequence; an embedding vector sequence of the text sequence is obtained; the embedding vector sequence is encoded to obtain a text vector representing the text and multiple text element vectors respectively representing the text elements of multiple description types; the text vector and the multiple text element vectors are fused to obtain a text fusion vector; and the text fusion vector is mapped to a classification result of a text type related to the text classification task. By combining text elements of different description types to classify the text, the accuracy of text classification can be improved.

[0352] As described above, these are only embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and scope of the present application are all included within the protection scope of the present application.

Claims

1. A text classification method, characterized in that, The method includes: Obtaining the text to be classified; Extracting text elements of multiple description types related to the text classification task from the text; Adding a description type tag of the description type corresponding to the text element at the position corresponding to the text element in the text to obtain a text sequence; Obtaining an embedding vector sequence of the text sequence; Encoding the embedding vector sequence to obtain a text vector representing the text and multiple text element vectors respectively representing the text elements of the multiple description types; Performing a fusion process on the text vector and the multiple text element vectors to obtain a text fusion vector; Mapping the text fusion vector to a classification result of a text type related to the text classification task.

2. The method according to claim 1, wherein The extracting text elements of multiple description types related to the text classification task from the text includes: Obtaining multiple keyword libraries respectively associated with the multiple description types; Comparing the keywords in the keyword library with the text to obtain a comparison result; In response to the comparison result indicating that the text contains the keyword, taking the part of the text containing the keyword as the text element of the description type associated with the keyword library.

3. The method according to claim 1, wherein The extracting text elements of multiple description types related to the text classification task from the text includes: Obtaining a prompt word, where the prompt word includes the text classification task and multiple description types to which the text elements related to the text classification task belong; Invoking a pre-trained semantic understanding model according to the prompt word to extract the text elements of the multiple description types from the text.

4. The method according to claim 1, wherein The performing a fusion process on the text vector and the multiple text element vectors to obtain a text fusion vector includes: Performing average pooling on the multiple text element vectors to obtain an average pooling vector; Performing a weighting process on the average pooling vector to obtain a weighted vector corresponding to the multiple text element vectors; Concatenating the text vector and the weighted vector corresponding to the multiple text element vectors to obtain a text fusion vector.

5. The method according to any one of claims 1 to 4, characterized in that The obtaining an embedding vector sequence of the text sequence includes: Adding a start tag at the start position of the text sequence and adding an end tag at the end position of the text sequence to form a tagged sequence; Performing an embedding process on the tagged sequence to obtain an embedding vector sequence of the text sequence.

6. The method according to claim 5, wherein The performing an embedding process on the tagged sequence to obtain an embedding vector sequence of the text sequence includes: Performing semantic embedding on the elements in the tagged sequence to obtain a symbol embedding vector corresponding to each element in the tagged sequence, where the elements include the text element, the start tag, and the end tag; Performing role embedding on the elements in the tagged sequence to obtain a role embedding vector corresponding to each element in the tagged sequence, where the role embedding vectors of the text elements with different description type tags are different; Generating the embedding vector sequence based on the symbol embedding vector and the role embedding vector of each element.

7. The method according to claim 6, characterized in that, Generating the embedding vector sequence based on the symbol embedding vectors and the role embedding vectors of each of the elements includes: Performing segment embedding processing on the elements in the token sequence to obtain segment embedding vectors corresponding to the token sequence, where the segment embedding vectors represent the embedding vectors of the segments to which the elements belong in the text, and the segment embedding vectors corresponding to different segments in the text are different; Performing position embedding processing on the elements in the token sequence to obtain position embedding vectors corresponding to the token sequence, where the segment embedding vectors corresponding to different texts are different; Determining the sum of the symbol embedding vector, the role embedding vector, the segment embedding vector, and the position embedding vector of each element, and forming an embedding vector sequence with the sums corresponding to each element.

8. The method according to claim 6 or 7, characterized in that Encoding the embedding vector sequence to obtain a text vector representing the text and multiple text element vectors respectively representing the text elements of the multiple description types includes: Invoking a pre-trained semantic understanding model to encode the embedding vector sequence to obtain an encoded sequence, where the encoded sequence includes the embedding vectors corresponding to the text elements in the text sequence and the embedding vector corresponding to the start token; Using the embedding vector corresponding to the start token as the text vector; Using the embedding vectors corresponding to the text elements with the description type tokens as the text element vectors representing the description types corresponding to the description type tokens.

9. The method according to any one of claims 1 to 7, characterized in that Mapping the text fusion vector to a classification result of a text type related to the text classification task includes: Mapping the text fusion vector to a probability distribution of multiple text types related to the text classification task; Using the text type corresponding to the maximum probability in the probability distribution as the classification result of the text type related to the text classification task.

10. A method for training a text classification model, characterized in that, The text classification model includes a semantic understanding model and a classifier; The method includes: Obtaining a text sample and the actual classification result of the text sample; Extracting text elements of multiple description types related to the text classification task from the text sample; Adding description type tokens corresponding to the description types of the text elements at the positions corresponding to the text elements in the text sample to obtain a text sample sequence; Obtaining an embedding vector sequence of the text sample sequence; Encoding the embedding vector sequence through the semantic understanding model to obtain a text vector representing the text sample and multiple text element vectors respectively representing the text elements of the multiple description types; Performing a fusion process on the text vector and the multiple text element vectors to obtain a text fusion vector; Mapping the text fusion vector to a predicted classification result of a text type related to the text classification task through the classifier; According to the difference between the predicted classification result and the actual classification result, reversely updating the parameters of the semantic understanding model and the classifier.

11. A text classification device, characterized in that, The device includes: A first acquisition module, configured to acquire the text to be classified; acquire the embedding vector sequence of the text sequence; A first processing module, configured to extract text elements of multiple description types related to the text classification task from the text; add description type tags of the description types corresponding to the text elements at positions corresponding to the text elements in the text to obtain a text sequence; A first encoding module, configured to encode the embedding vector sequence to obtain a text vector representing the text and multiple text element vectors respectively representing the text elements of the multiple description types; A first fusion module, configured to perform a fusion process on the text vector and the multiple text element vectors to obtain a text fusion vector; A first mapping module, configured to map the text fusion vector to a classification result of a text type related to the text classification task.

12. A text classification model training device, characterized in that, The text classification model includes a semantic understanding model and a classifier; The apparatus includes: A second acquisition module, configured to acquire a text sample and the actual classification result of the text sample; acquire the embedding vector sequence of the text sample sequence; A second processing module, configured to extract text elements of multiple description types related to the text classification task from the text sample; add description type tags of the description types corresponding to the text elements at positions corresponding to the text elements in the text sample to obtain a text sample sequence; A second encoding module, configured to encode the embedding vector sequence through the semantic understanding model to obtain a text vector representing the text sample and multiple text element vectors respectively representing the text elements of the multiple description types; A second fusion module, configured to perform a fusion process on the text vector and the multiple text element vectors to obtain a text fusion vector; A second mapping module, configured to map the text fusion vector to a predicted classification result of a text type related to the text classification task through the classifier; An update module, configured to reversely update the parameters of the semantic understanding model and the classifier according to the difference between the predicted classification result and the actual classification result.

13. An electronic device, characterized in that, The electronic device includes: A memory, configured to store computer-executable instructions; A processor, configured to implement the text classification method according to any one of claims 1 to 9, or implement the text classification model training method according to claim 10 when executing the computer-executable instructions stored in the memory.

14. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, The computer-executable instructions or the computer program, when executed by the processor, implement the text classification method according to any one of claims 1 to 9, or implement the text classification model training method according to claim 10.

15. A computer program product comprising computer-executable instructions or a computer program, characterized in that, The computer-executable instructions or the computer program, when executed by the processor, implement the text classification method according to any one of claims 1 to 9, or implement the text classification model training method according to claim 10.

Images