Text classification model training method, text classification method and device

Abstract

The present disclosure provides a text classification model training method, a text classification method and a device, and relates to the technical field of artificial intelligence, in particular to the technical field of natural language processing, machine learning and the like. The implementation scheme is as follows: a sample set is obtained; the parameters of a first text classification model are adjusted at least once based on the sample set to obtain a trained second text classification model, each adjustment comprising: adjusting the parameters of the current first text classification model using a first subset of the current sample set to obtain an adjusted text classification model; determining a first evaluation value of each first output category of the adjusted text classification model using a second subset of the current sample set; in response to the first evaluation value of any first output category being less than a threshold value, deleting samples with a category label of the first output category from the current sample set; and in response to the first evaluation values of all first output categories being greater than or equal to the threshold value, determining the adjusted text classification model as the second text classification model.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, particularly to the fields of natural language processing and machine learning, and specifically to a training method and apparatus for a text classification model, a text classification method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] Artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0003] Text classification is a common natural language processing task.

[0004] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0005] This disclosure provides a method and apparatus for training a text classification model, a text classification method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0006] According to one aspect of this disclosure, a method for training a text classification model is provided, comprising: acquiring a sample set, wherein the sample set includes a plurality of samples, each of the plurality of samples including sample text and a category label of the sample text; and adjusting the parameters of a first text classification model at least once based on the sample set to obtain a trained second text classification model, wherein each of the at least one adjustment includes: adjusting the parameters of the current first text classification model using a first subset of the current sample set to obtain an adjusted first text classification model; determining a first evaluation value for each of a plurality of first output categories of the adjusted first text classification model using a second subset of the current sample set, wherein the second subset is disjoint from the first subset, and the first evaluation value indicates the classification effect of the adjusted first text classification model on the corresponding first output category; in response to a first evaluation value of any of the plurality of first output categories being less than a threshold, deleting samples with the category label of the first output category from the current sample set; or in response to a first evaluation value of each of the plurality of first output categories being greater than or equal to the threshold, determining the adjusted first text classification model as the second text classification model.

[0007] According to one aspect of this disclosure, a text classification method is provided, comprising: acquiring text to be classified; segmenting the text using multiple segmentation granularities to obtain a word set of the text; and for each word in the word set, determining the category to which the word belongs using a text classification model, wherein the text classification model is trained according to the training method of the aforementioned text classification model.

[0008] According to one aspect of this disclosure, a training apparatus for a text classification model is provided, comprising: a first acquisition module configured to acquire a sample set, wherein the sample set includes a plurality of samples, each of the plurality of samples including sample text and a category label of the sample text; and a first adjustment module configured to adjust the parameters of a first text classification model at least once based on the sample set to obtain a trained second text classification model, wherein the first adjustment module includes: an adjustment unit configured to adjust the parameters of a current first text classification model using a first subset of the current sample set to obtain an adjusted first text classification model; and a first evaluation unit configured to evaluate the parameters of a current first text classification model using a first subset of the current sample set. The second subset determines the first evaluation value of each of the multiple first output categories of the adjusted first text classification model, wherein the second subset is disjoint from the first subset, and the first evaluation value indicates the classification effect of the adjusted first text classification model on the corresponding first output category; the deletion unit is configured to delete samples with the category label of the first output category from the current sample set in response to the first evaluation value of any of the multiple first output categories being less than a threshold; or the first determination unit is configured to determine the adjusted first text classification model as the second text classification model in response to the first evaluation value of each of the multiple first output categories being greater than or equal to the threshold.

[0009] According to one aspect of this disclosure, a text classification apparatus is provided, comprising: an acquisition module configured to acquire text to be classified; a word segmentation module configured to segment the text using multiple word segmentation granularities to obtain a word set of the text; and a classification module configured to determine the category to which each word in the word set belongs using a text classification model, wherein the text classification model is trained using a training device for the aforementioned text classification model.

[0010] According to one aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any of the above aspects.

[0011] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the methods of any of the above aspects.

[0012] According to one aspect of this disclosure, a computer program product is provided, including computer program instructions that, when executed by a processor, implement the method of any of the above aspects.

[0013] According to one or more embodiments of this disclosure, the accuracy of text classification can be improved.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0016] Figure 1 A schematic diagram of an exemplary system in which various methods described herein may be implemented according to embodiments of the present disclosure is shown;

[0017] Figure 2 A flowchart illustrating a training method for a text classification model according to an embodiment of the present disclosure is shown;

[0018] Figure 3 A flowchart of a text classification method according to an embodiment of the present disclosure is shown;

[0019] Figure 4 A structural block diagram of a training apparatus for a text classification model according to an embodiment of the present disclosure is shown;

[0020] Figure 5 A structural block diagram of a text classification apparatus according to an embodiment of the present disclosure is shown; and

[0021] Figure 6 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0022] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0023] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0024] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof. "More than" means two or more.

[0025] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0026] With the development of information technology, enterprises have accumulated a large amount of data. Taking financial institutions as an example, users generate behavioral data when using the financial institution's applications, and may send request data to the application's server and receive response data from the server. User behavioral data, request data, response data, and other data will be stored as the financial institution's data assets. Data that enterprises obtain through the network is usually unstructured data in formats such as JSON (JavaScript Object Notation) and HTML (Hyper Text Markup Language).

[0027] To achieve effective data construction, application, and security for enterprises, it is necessary to classify and grade data. For example, from a regulatory perspective, laws and regulations such as the *Data Security Law* and the *Personal Information Protection Law* require data classification and grading. From a value perspective, most of the data collected by enterprises is "dark data," whose value is not yet clear and cannot be used for other purposes. In the era of big data, data is a new factor of production, and assetization is gradually becoming a trend. Classifying and grading data can effectively identify important and sensitive data for enterprises, and classify and grade it according to industry standards and requirements; therefore, data classification and grading are particularly important. From a data governance perspective, data classification and grading are the foundation of all data applications. Only by properly classifying and grading data can data security be better guaranteed and data applications be realized more quickly. For example, the classification of data sensitivity levels can be directly applied to data security protection work, quickly formulating different data security strategies for data with different sensitivity levels.

[0028] Data classification refers to categorizing data into a specific class from a business perspective. This class can be a field in a company's database table. For example, the data "Zhang San" and "Li Si" might be categorized as "Name," and the data "No. 001, Street C, District B, City A" might be categorized as "Address," and so on. The number of data classes is usually large, sometimes reaching hundreds or even thousands. Data grading refers to classifying data into a specific level based on factors such as regulatory oversight, security, and sensitivity, such as Level 1, Level 2, Level 3, etc. The number of data grades is usually smaller, typically in the single digits.

[0029] Data is typically stored as text. Related technologies usually employ sample data to train a text classification model, which is then used to classify the real data to be classified. However, the quality of sample data varies greatly; for example, there may be too few samples for a particular category, an imbalance in the number of samples across categories, or a discrepancy between the distribution of sample data and real data. These issues lead to lower classification accuracy in text classification models.

[0030] To address the aforementioned issues, this disclosure provides a method for training a text classification model that can improve the classification accuracy of the text classification model and avoid misclassification of text data.

[0031] Figure 1 A schematic diagram of an exemplary system 100 in which the various methods and apparatus described herein can be implemented according to embodiments of this disclosure is shown. Reference Figure 1 The system 100 includes one or more client devices 101, 102, 103, 104, 105 and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105 and 106 can be configured to execute one or more applications.

[0032] In embodiments of this disclosure, client devices 101, 102, 103, 104, 105, and 106, as well as server 120, may run one or more services or software applications that enable the execution of training methods for text classification methods or text classification models.

[0033] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtual and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as to users of client devices 101, 102, 103, 104, 105, and / or 106 under a Software as a Service (SaaS) model.

[0034] exist Figure 1 In the configuration shown, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or combinations thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 can sequentially interact with server 120 using one or more client applications to utilize the services provided by these components. It should be understood that various different system configurations are possible and may differ from system 100. Therefore, Figure 1 This is an example of a system used to implement the various methods described herein, and is not intended to be limiting.

[0035] Client devices 101, 102, 103, 104, 105, and / or 106 can provide interfaces that enable users of the client devices to interact with them. The client devices can also output information to the user through these interfaces. Although Figure 1 Only six client devices are described, but those skilled in the art will understand that this disclosure can support any number of client devices.

[0036] Client devices 101, 102, 103, 104, 105, and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, in-vehicle devices, gaming systems, thin clients, various messaging devices, sensors, or other sensing devices. These computer devices can run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux, or Linux-like operating systems; or include various mobile operating systems, such as Microsoft Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include cellular phones, smartphones, tablets, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Gaming systems may include various handheld gaming devices, internet-enabled gaming devices, etc. Client devices are capable of executing various applications, such as various internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and can use various communication protocols.

[0037] Network 110 can be any type of network well known to those skilled in the art, and can use any of a variety of available protocols (including but not limited to TCP / IP, SNA, IPX, etc.) to support data communication. By way of example only, one or more networks 110 can be a local area network (LAN), an Ethernet-based network, a token ring network, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, Wi-Fi), and / or any combination of these and / or other networks.

[0038] Server 120 may include one or more general-purpose computers, special-purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0039] The computing unit in server 120 can run one or more operating systems, including any of the aforementioned operating systems and any commercially available server operating system. Server 120 can also run any of a variety of additional server applications and / or middleware applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0040] In some implementations, server 120 may include one or more applications to analyze and merge data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105 and / or 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105 and / or 106.

[0041] In some implementations, server 120 can be a server for a distributed system or a server integrated with blockchain. Server 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0042] System 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. Databases 130 may reside in various locations. For example, a database used by server 120 may be local to server 120, or it may be located away from server 120 and may communicate with server 120 via a network-based or dedicated connection. Databases 130 may be of different types. In some embodiments, the database used by server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data from and from the databases in response to commands.

[0043] In some embodiments, one or more of the databases 130 may also be used by an application to store application data. The databases used by the application may be of different types, such as key-value stores, object stores, or regular stores supported by a file system.

[0044] Figure 1 The system 100 can be configured and operated in various ways to enable the application of the various methods and apparatus described in this disclosure.

[0045] Figure 2 A flowchart of a training method 200 for a text classification model according to an embodiment of the present disclosure is shown. The execution entity for each step of method 200 is typically a server, for example... Figure 1 The server 120 shown. In some embodiments, the entity executing method 200 may also be a client device, such as... Figure 1 The client devices shown are 101-106.

[0046] like Figure 2 As shown, method 200 includes steps S210 and S220.

[0047] In step S210, a sample set is obtained. The sample set includes multiple samples, and each sample includes sample text and a category label for that sample text.

[0048] In step S220, based on the above sample set, the parameters of the first text classification model are adjusted at least once to obtain a trained second text classification model.

[0049] Each adjustment in at least one of the adjustments in step S220 above includes steps S221-S224.

[0050] In step S221, the parameters of the current first text classification model are adjusted using the first subset of the current sample set to obtain the adjusted first text classification model.

[0051] In step S222, a first evaluation value is determined for each of the multiple first output categories of the adjusted first text classification model using a second subset of the current sample set. The second subset does not overlap with the first subset. The first evaluation value indicates the classification performance of the adjusted first text classification model for the corresponding first output category.

[0052] In step S223, in response to the first evaluation value of any of the multiple first output categories being less than a threshold, samples with the category label of that first output category are deleted from the current sample set.

[0053] In step S224, in response to the fact that the first evaluation values ​​of each of the multiple first output categories are all greater than or equal to the threshold, the adjusted first text classification model is determined as the second text classification model.

[0054] According to embodiments of this disclosure, the classification performance of each category is dynamically evaluated during the training process of the text classification model. If the classification performance of a certain category is poor, the sample data of that category is deleted, and the model is retrained until the classification performance of all categories reaches the threshold. This ensures the classification accuracy of the text classification model and avoids misclassification of text.

[0055] The following details each step of method 200.

[0056] In step S210, a sample set is obtained. The sample set includes multiple samples, each of which includes sample text and a category label for that sample text.

[0057] According to some embodiments, to achieve data classification for a specific industry (e.g., the financial industry), a database classification template for that industry can be pre-set. The database classification template includes multiple fields under the specific industry and exemplary data items (i.e., demo data) under each field. Each field in the database corresponds to a category of data classification. For example, the database classification template includes a field "Name," under which multiple exemplary data items such as Zhang San, Li Si, and Wang Wu are included.

[0058] According to some embodiments, the sample set in step S210 can be exemplary data items under each field in the database classification template. That is, the identifier of the field in the database is used as the category label, and the exemplary data item under that field is used as sample text with that category label.

[0059] It is understandable that, in order to achieve standardized data classification for a specific industry, it is generally necessary to construct a database classification template for that industry. According to the above embodiment, the structured data in the database classification template is directly used to construct a sample set for training the text classification model, eliminating the need to label unstructured real text (e.g., JSON text, HTML text, etc. obtained from the internet), thus improving the efficiency of sample acquisition and consequently improving model training efficiency.

[0060] It is understood that in the above embodiments, the sample text (exemplary data in the database classification template) is structured virtual text, and each sample text corresponds to one and only one category label. However, the real text to be classified is unstructured network data, such as JSON text or HTML text collected or returned via API. Each real text to be classified may correspond to one category or multiple categories. For example, the real text "Zhang San" corresponds to the category "Name", and the fragments "D Company, 1234567, No. 001, C Street, B District, A City" of the real text "D Company, 1234567, No. 001, C Street, B District, A City" correspond to the categories "Company Name", "Company Phone Number", and "Company Address", respectively. Therefore, the data distribution of sample text and real text is different. Step S220 of this embodiment is used to ensure the classification accuracy of the text classification model when the data distribution of sample text and real text is different.

[0061] According to some embodiments, database classification templates can be preprocessed, and the processed database classification templates can be used as a sample set. Preprocessing may include, for example, removing dirty data, such as data containing identifiers like "pending," "test," or "test," thereby ensuring model accuracy and preventing low-quality data from interfering with model performance. Preprocessing may also include data filtering. For example, if the number of exemplary data items under a certain field (category) is less than a threshold (e.g., 100), the model abandons prediction for that category. This is because if there are too few sample texts under a certain category, the model's understanding of the text in that category will be insufficient, resulting in poor classification performance. Even if good classification performance is achieved during the training phase, the lack of generalization and reliability due to insufficient sample texts cannot guarantee classification performance on real text, therefore training and prediction for that category are abandoned.

[0062] As described above, in step S220, the parameters of the first text classification model are adjusted at least once based on the sample set to obtain a trained second text classification model. Each of the above-mentioned adjustments includes steps S221-S224.

[0063] In step S221, the parameters of the current first text classification model are adjusted using the first subset of the current sample set to obtain the adjusted first text classification model.

[0064] It should be noted that the current sample set in the first adjustment is the initial value of the sample set, that is, the sample set obtained through step S210. The current sample set in the second and subsequent adjustments is the sample set updated in the previous adjustment.

[0065] The first text classification model used in the first adjustment is the initial text classification model. The parameters of the initial text classification model can be all zeros or random numbers. The first text classification model used in the second and subsequent adjustments is the text classification model after the previous adjustment.

[0066] According to some embodiments, the first text classification model is a lightweight model, such as a random forest, which ensures the efficiency of model training, model update iteration, and especially model prediction. Common data processing tasks in business systems, such as de-identification, recognition, and protection, often have QPS (Queries Per Second) in the tens of thousands. To avoid affecting the normal operation of the business system, it is necessary to ensure a high QPS for model prediction. Furthermore, according to some embodiments, to avoid labeling, the sample set uses a database classification template, which typically contains only a small amount of exemplary data, such as dozens or hundreds of records. Models that can achieve good results with limited data are often lightweight models.

[0067] According to some embodiments, a first subset of the current sample set may include more than half of the samples in the current sample set, for example, 70% of the samples in the sample set. The first subset may be obtained, for example, by randomly selecting samples from the current sample set.

[0068] According to some embodiments, step S221 may include steps S2211-S2214.

[0069] In step S2211, a keyword list for extracting text features is obtained.

[0070] In step S2212, for any sample text in the first subset: based on the keyword list, extract the text features of the sample text; and input the text features into the current first text classification model to obtain the predicted category of the sample text output by the current first text classification model.

[0071] In step S2213, the loss value of the current first text classification model is determined based on the predicted category and category label of each sample text in the first subset.

[0072] In step S2214, the parameters of the current first text classification model are adjusted based on the loss value.

[0073] According to the above embodiments, extracting text features using a keyword list can improve the relevance and accuracy of text features. Furthermore, the text classification model uses these text features to determine the predicted category of the sample text, thereby improving the model's classification accuracy.

[0074] According to some embodiments, the keyword list in step S2211 is determined by statistical analysis of sample texts in the sample set obtained in step S210. This improves the efficiency and relevance of keyword list acquisition.

[0075] According to some implementation methods, the sample text under each category label can be segmented into words, and the occurrence frequency or number of occurrences of each word under that category label can be counted. One or more words with the highest occurrence frequency or number of occurrences can be selected as keywords. For example, if a word ranks within the top 30 in the occurrence frequency of a certain category label, and its occurrence frequency is greater than 10% of the total number of sample texts under that category label, then that word can be identified as a keyword.

[0076] According to some embodiments, in step S2212, for any sample text in the first subset, the keywords included in the sample text and the number of times (or frequency) of the keywords are determined by querying a keyword table, and then text features of the sample text are generated based on the keywords. The text features may, for example, be a keyword vector composed of the number of times (or frequency) of each keyword included in the sample text.

[0077] According to some embodiments, text features can also be generated by combining statistical information of each character in the sample text or semantic information of the sample text. Character statistical information includes, for example, the number of Chinese characters, the number of English characters, the number of numbers, and the number of special symbols. Concatenating these statistical information items yields a character statistical vector of the sample text. Semantic information of the sample text can be obtained, for example, through a text representation model. Specifically, by inputting the sample text into a text representation model, the semantic vector of the sample text output by the text representation model can be obtained.

[0078] According to some embodiments, the keyword vector can be concatenated with at least one of the above-mentioned character statistical vector and semantic vector to obtain the text features of the sample text.

[0079] The text features of the sample text are input into the current first text classification model, which then outputs the predicted category of the sample text. The predicted category can be represented as a probability distribution, that is, the probability that the sample text belongs to each category. The category with the highest probability is the predicted category of the sample text.

[0080] According to some embodiments, in step S2213, the loss value of the current first text classification model may be, for example, the cross-entropy between the predicted category and the category label.

[0081] In step S2214, the parameters of the current first text classification model are adjusted in the direction of reducing the loss value.

[0082] It is understandable that steps S2212-S2214 can be executed repeatedly until a preset termination condition is reached, resulting in the adjusted first text classification model. The termination condition could be, for example, the number of iterations reaching a threshold, the model's classification accuracy reaching a threshold, or accuracy convergence.

[0083] After obtaining the adjusted first text classification model through step S221, step S222 is executed to determine the first evaluation value of each of the multiple first output categories of the adjusted first text classification model using the second subset of the current sample set.

[0084] The second subset does not intersect with the first subset used to adjust model parameters. According to some embodiments, the second subset can be the difference between the current sample set and the first subset, that is, the second subset includes all samples in the current sample set that do not belong to the first subset.

[0085] The first output category is the category that the adjusted first text classification model can output.

[0086] According to some embodiments, the first evaluation value may be the precision of the first output category. Specifically, the precision of the first output category is the ratio of the number of sample texts whose predicted category and category label are both the first output category to the number of sample texts whose predicted category is the first output category.

[0087] According to some embodiments, the first evaluation value may also be the recall rate (also known as the full count) or F1 score of the first output category.

[0088] The first evaluation value typically ranges from 0 to 1. To ensure the classification performance of the model, the threshold for the first evaluation value is usually set to a relatively large value, such as 0.9 or 0.95.

[0089] If the first evaluation value of the first output category is greater than or equal to the threshold, it indicates that the adjusted first text classification model has a good classification effect on this category and can achieve accurate prediction.

[0090] If the first evaluation value of the first output category is less than the threshold, it indicates that the adjusted first text classification model performs poorly in classifying this category and it is difficult to accurately predict this category. Therefore, the prediction for this category is abandoned. Accordingly, in step S223, samples with the category label of this category are deleted from the current sample set, and the updated sample set is used for the next model parameter adjustment.

[0091] For example, in practice, it's possible to skip predicting the "nickname" category. This is because it's difficult to predict accurately, anything can happen, and it's easily confused with other categories.

[0092] The fact that the first evaluation value of all first output categories is greater than or equal to the threshold indicates that the adjusted first classification model can accurately predict all first output categories and achieves good overall classification performance. Accordingly, in step S224, the adjusted first text classification model is used as the trained second text classification model, and the model training process ends.

[0093] The trained second text classification model can classify the real text to be classified.

[0094] According to some embodiments, method 200 further includes step S230.

[0095] In step S230, prediction thresholds are determined for each of the multiple second output categories of the second text classification model. These prediction thresholds are used when the probability of the text to be classified belonging to a corresponding second output category is greater than the prediction threshold, thus determining the predicted category of the text to be classified as that second output category. For example, the prediction threshold for the "name" category is 0.9. When the text to be classified, "Xiaoming," is input into the second text classification model, the model outputs a probability of 0.93 for "Xiaoming" belonging to the "name" category. Since 0.93 is greater than the prediction threshold of 0.9 for the "name" category, the predicted category for "Xiaoming" is determined to be "name".

[0096] It's understandable that if the prediction threshold for a certain category is set too low (e.g., 0.3), it will affect the precision of that category, resulting in a large number of misclassifications—that is, text that does not belong to that category will be incorrectly classified as belonging to it. If the prediction threshold for a certain category is set too high (e.g., 0.9), it will affect the recall of that category, resulting in some missed classifications—that is, text that actually belongs to that category will be incorrectly classified as belonging to other categories.

[0097] According to the above embodiments, by setting a prediction threshold for each category, it is possible to ensure that the prediction threshold for each category is appropriate for that category, reduce misclassification or omission, and thus ensure the classification effect of each category.

[0098] The first output category is the category that the second text classification model can output. The prediction threshold can be, for example, 0.8, 0.9, etc.

[0099] According to some embodiments, the prediction threshold for each second output category can be determined according to steps S231 and S232.

[0100] In step S231, for any candidate prediction threshold among multiple candidate prediction thresholds:

[0101] Based on the candidate prediction threshold, the predicted category of each sample text in the second subset of the current sample set is determined using the second text classification model;

[0102] Based on the predicted category and category label of each sample text in the second subset of the current sample set, a second evaluation value is determined for each of the multiple second output categories, wherein the second evaluation value indicates the classification performance of the second text classification model for the corresponding second output category.

[0103] In step S232, for any second output category among a plurality of second output categories, the candidate prediction threshold that maximizes the second evaluation value of the second output category among a plurality of candidate prediction thresholds is determined as the prediction threshold of the second output category.

[0104] According to the above embodiments, the candidate prediction threshold that best achieves the classification effect for each category is used as its prediction threshold, which can ensure the classification effect for each category and reduce misclassification or omission.

[0105] According to the above embodiments, the candidate prediction threshold can be obtained by sampling a preset sampling interval according to a preset sampling interval. For example, if the sampling interval is 0.6-0.9 and the sampling interval is 0.05, then the candidate prediction thresholds are 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, and 0.9.

[0106] In step S231, the current sample set is the sample set from which the second text classification model was trained. It can be understood that this current sample set is a subset of the sample set obtained in step S210.

[0107] According to some embodiments, by inputting the text features of sample texts from a second subset of the current sample set into a second text classification model, the probability that the sample text belongs to each second output category can be obtained. If the maximum probability is greater than the current candidate prediction threshold, the second output category corresponding to the maximum probability is determined as the predicted category of the sample text. If the maximum probability is less than or equal to the current candidate prediction threshold, the predicted category of the sample text is set to empty, that is, the sample text is considered not to belong to any category.

[0108] Once the predicted category of each sample text in the second subset is determined, a second evaluation value for each second output category can be determined based on the predicted category and category label of each sample text.

[0109] The second evaluation metric could be, for example, the F1 score of the second output category. The F1 score is the harmonic mean of precision and recall, i.e., F1 score = 2 * precision * recall / (precision + recall), where precision and recall represent precision and recall, respectively.

[0110] The precision of the second output category is the ratio of the number of sample texts with both predicted category and category label belonging to that second output category to the number of sample texts with predicted category belonging to that second output category.

[0111] The recall of the second output category is the ratio of the number of sample texts whose predicted category and category label are both the second output category to the number of sample texts whose category label is the second output category.

[0112] According to the above embodiment, the F1 score is determined based on the precision and recall of the second output class. Using the F1 score as a second evaluation metric can evaluate both the precision and recall of the second output class, thereby improving the accuracy of the prediction threshold and reducing misclassification and missed classification.

[0113] According to some embodiments, the second evaluation value may also be the precision or recall of the second output category.

[0114] According to some embodiments, method 200 further includes steps S240-S260.

[0115] In step S240, the text to be classified is classified using the second text classification model to obtain the predicted category of the text to be classified.

[0116] In step S250, the category labeling results of the text to be classified are obtained.

[0117] In step S260, in response to the difference between the predicted category and the category labeling result, the parameters of the second text classification model are adjusted based on the text to be classified and the category labeling result to obtain the trained third text classification model.

[0118] According to the above embodiments, the classification effect of the second text classification model can be dynamically detected during its application. The model can be updated using misclassified real text, thus correcting its biases. Furthermore, by updating the model using real text, the distribution of training text and real text becomes increasingly consistent, thereby improving the model's accuracy in classifying real text.

[0119] According to some embodiments, in step S250, the category labeling results of the text to be classified can be obtained by manual labeling or machine labeling.

[0120] In step S260, the text to be classified and its category labeling results are used as samples to adjust the parameters of the second text classification model, that is, the second text classification model is iteratively updated. This yields the updated third text classification model. It can be understood that the accuracy of the third text classification model is better than that of the second text classification model.

[0121] The text classification model (including the second text classification model and the third text classification model) trained according to method 200 can be used to implement the text classification method 300 of this disclosure embodiment to classify the real text to be classified.

[0122] Figure 3 A flowchart of a text classification method 300 according to an embodiment of the present disclosure is shown. The execution entity of method 300 may be a server (e.g., Figure 1 The server 120 shown can also be a client device (e.g., Figure 1 Client devices 101-106 shown in the diagram. Figure 3 As shown, method 300 includes steps S310-S330.

[0123] In step S310, the text to be classified is obtained.

[0124] In step S320, the text is segmented using multiple segmentation granularities to obtain the word set of the text.

[0125] In step S330, for each word in the word set, the category to which the word belongs is determined using a text classification model. The text classification model is trained according to the text classification model training method of this disclosure embodiment. For example, the text classification model can be the second text classification model or the third text classification model in method 200.

[0126] According to embodiments of this disclosure, the text to be classified is segmented into words at different granularities, and a trained text classification model with high accuracy is used to determine the category of each word, which can improve the accuracy of text classification.

[0127] In some implementations, the text to be categorized is value data from API (API) data or HTML (HTML) data. API data is typically in JSON format.

[0128] For example, after obtaining API data in JSON format, the JSON is parsed, and the values ​​are extracted as the text to be classified. This text is then segmented using word segmentation algorithms of different granularities, including coarse-grained and fine-grained algorithms. For instance, for the text "Zhang Daming works at Baidu in Beijing," different granularity word segmentation results in the word set {Zhang Daming works at Baidu in Beijing, Zhang Daming, Beijing, Baidu, Beijing Baidu, working}. For each word, a text classification model is called to predict its category, and the category is determined by combining the prediction threshold for each category. For example, if the model predicts that "Beijing" belongs to the "capital" category with a probability of 0.95, which is greater than the prediction threshold of 0.9 for the "capital" category, then "capital" is classified as belonging to the "Beijing" category. If the model predicts that "Baidu" belongs to the "city" category with a probability of 0.3, which is less than the prediction threshold of 0.8 for the "city" category, then "Baidu" is not classified as belonging to the "city" category, and this prediction result is discarded.

[0129] In the embodiments of this disclosure, the text to be classified is unstructured and may correspond to one category or multiple categories. By segmenting the text to be classified at different granularities, it can be ensured that the words in the word set can fully cover all possible categories, thereby making the distribution of real text consistent with that of training text and avoiding the omission of text classification.

[0130] According to some embodiments, a correspondence between categories and levels can be pre-defined. Accordingly, after determining the category to which the text belongs, the level to which the text belongs can be further determined based on the correspondence between categories and levels. A level indicates the degree to which data possesses a certain attribute (such as sensitivity, security, etc.), and is usually represented numerically, such as level one, level two, level three, etc.

[0131] According to embodiments of this disclosure, a training apparatus for a text classification model is also provided. Figure 4 A structural block diagram of a training apparatus 400 for a text classification model according to an embodiment of the present disclosure is shown. Figure 4 As shown, the device 400 includes a first acquisition module 410 and a first adjustment module 420.

[0132] The first acquisition module 410 is configured to acquire a sample set. The sample set includes multiple samples, each of which includes sample text and a category label for the sample text.

[0133] The first adjustment module 420 is configured to adjust the parameters of the first text classification model at least once based on the sample set to obtain a trained second text classification model.

[0134] The first adjustment module 420 includes an adjustment unit 421, a first evaluation unit 422, a deletion unit 423, and a first determination unit 424.

[0135] The adjustment unit 421 is configured to adjust the parameters of the current first text classification model using a first subset of the current sample set to obtain the adjusted first text classification model.

[0136] The first evaluation unit 422 is configured to determine a first evaluation value for each of the multiple first output categories of the adjusted first text classification model using a second subset of the current sample set, wherein the second subset does not intersect with the first subset, and the first evaluation value indicates the classification effect of the adjusted first text classification model on the corresponding first output category.

[0137] The deletion unit 423 is configured to delete samples with the category label of the first output category from the current sample set in response to a first evaluation value of any of the plurality of first output categories being less than a threshold.

[0138] The first determining unit 424 is configured to determine the adjusted first text classification model as the second text classification model in response to the first evaluation value of each of the plurality of first output categories being greater than or equal to the threshold.

[0139] According to embodiments of this disclosure, the classification performance of each category is dynamically evaluated during the training process of the text classification model. If the classification performance of a certain category is poor, the sample data of that category is deleted, and the model is retrained until the classification performance of all categories reaches the threshold. This ensures the classification accuracy of the text classification model and avoids misclassification of text.

[0140] According to some embodiments, the category label is an identifier of a field in a database, and the sample text with the category label is an exemplary data item under the field.

[0141] According to some embodiments, the apparatus 400 further includes: a classification module configured to classify the text to be classified using the second text classification model to obtain a predicted category of the text to be classified; a second acquisition module configured to acquire a category labeling result of the text to be classified; and a second adjustment module configured to adjust the parameters of the second text classification model based on the text to be classified and the category labeling result in response to a difference between the predicted category and the category labeling result, to obtain a trained third text classification model.

[0142] According to some embodiments, the apparatus 400 further includes: a determining module configured to determine a prediction threshold for each of a plurality of second output categories of the second text classification model, wherein the prediction threshold is used to determine the predicted category of the text to be classified as the second output category in response to a probability that the probability of the text to be classified belonging to the corresponding second output category output by the second text classification model is greater than the prediction threshold.

[0143] According to some embodiments, the determining module includes: a second evaluation unit configured to, for any one of a plurality of candidate prediction thresholds: determine the predicted category of each sample text in a second subset of the current sample set using the second text classification model based on the candidate prediction threshold; and determine a second evaluation value for each of the plurality of second output categories based on the predicted category and category label of each sample text in the second subset of the current sample set, wherein the second evaluation value indicates the classification effect of the second text classification model on the corresponding second output category; and a second determining unit configured to, for any one of the plurality of second output categories, determine the candidate prediction threshold that maximizes the second evaluation value of the second output category among the plurality of candidate prediction thresholds as the prediction threshold for the second output category.

[0144] According to some embodiments, the adjustment unit 421 includes: an acquisition subunit configured to acquire a keyword list for extracting text features; a prediction subunit configured to, for any sample text in the first subset,: extract text features of the sample text based on the keyword list; and input the text features into the current first text classification model to obtain the predicted category of the sample text output by the current first text classification model; a determination subunit configured to determine the loss value of the current first text classification model based on the predicted category and category label of each sample text in the first subset; and an adjustment subunit configured to adjust the parameters of the current first text classification model based on the loss value.

[0145] According to some embodiments, the keyword list is determined by statistical analysis of sample texts in the sample set.

[0146] It should be understood that Figure 4 The various modules and units of the apparatus 400 shown can be used with reference to Figure 2 The steps in method 200 described correspond to each other. Therefore, the operations, features, and advantages described above for method 200 also apply to apparatus 400 and its included modules and units. For the sake of brevity, some operations, features, and advantages will not be repeated here.

[0147] According to embodiments of this disclosure, a text classification device is also provided. Figure 5A structural block diagram of a text classification apparatus 500 according to an embodiment of the present disclosure is shown. Figure 5 As shown, the device 500 includes an acquisition module 510, a word segmentation module 520, and a classification module 530.

[0148] The acquisition module 510 is configured to acquire the text to be classified.

[0149] The word segmentation module 520 is configured to segment the text using multiple word segmentation granularities to obtain a word set of the text.

[0150] The classification module 530 is configured to determine the category to which each word belongs using a text classification model for each word in the word set, wherein the text classification model is trained by the text classification model training device of the present disclosure embodiment.

[0151] According to embodiments of this disclosure, the text to be classified is segmented into words at different granularities, and a trained text classification model with high accuracy is used to determine the category of each word, which can improve the accuracy of text classification.

[0152] In the embodiments of this disclosure, the text to be classified is unstructured and may correspond to one category or multiple categories. By segmenting the text to be classified at different granularities, it can be ensured that the words in the word set can fully cover all possible categories, thereby making the distribution of real text consistent with that of training text and avoiding the omission of text classification.

[0153] It should be understood that Figure 5 The various modules and units of the apparatus 500 shown can be used with reference to Figure 3 The steps in method 300 described correspond to each other. Therefore, the operations, features, and advantages described above for method 300 also apply to apparatus 500 and its included modules and units. For the sake of brevity, some operations, features, and advantages will not be repeated here.

[0154] Although specific functions have been discussed with reference to specific modules above, it should be noted that the functions of the various modules discussed in this article can be divided into multiple modules, and / or at least some functions of multiple modules can be combined into a single module.

[0155] It should also be understood that this article can describe various technologies in the general context of software and hardware components or program modules. The above regarding... Figure 4 , Figure 5The described units can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these units can be implemented as computer program code / instructions configured to execute in one or more processors and stored in a computer-readable storage medium. Alternatively, these units can be implemented as hardware logic / circuit. For example, in some embodiments, one or more of modules 410-530 can be implemented together in a System on Chip (SoC). The SoC may include an integrated circuit chip (which includes a processor (e.g., a Central Processing Unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and / or one or more components of other circuitry) and may optionally execute received program code and / or include embedded firmware to perform functions.

[0156] According to embodiments of the present disclosure, an electronic device is also provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a training method and / or a text classification method of an embodiment of the present disclosure.

[0157] According to embodiments of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute a training method and / or a text classification method of an embodiment of the present disclosure.

[0158] According to embodiments of the present disclosure, a computer program product is also provided, including computer program instructions that, when executed by a processor, implement the training method and / or text classification method of the text classification model of the present disclosure embodiments.

[0159] refer to Figure 6The present invention describes a structural block diagram of an electronic device 600 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0160] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0161] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, output unit 607, storage unit 608, and communication unit 609. Input unit 606 can be any type of device capable of inputting information to electronic device 600. Input unit 606 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device, and can include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 607 can be any type of device capable of presenting information, and can include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 608 can include, but is not limited to, disk and optical disk. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and can include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, 802.11 devices, Wi-Fi devices, WiMAX devices, cellular communication devices, and / or the like.

[0162] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as method 200 or method 300. For example, in some embodiments, method 200 and method 300 may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of method 200 and method 300 described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform method 200 or method 300 by any other suitable means (e.g., by means of firmware).

[0163] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0164] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0165] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0166] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0167] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0168] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0169] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0170] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of this disclosure is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A method for training a text classification model, comprising: obtaining a sample set, wherein the sample set comprises a plurality of samples, each sample in the plurality of samples comprises a sample text and a category label of the sample text, wherein the category label is an identification of a field in a database, and the sample text with the category label is an exemplary data item under the field; and adjusting parameters of a first text classification model based on the sample set to obtain a trained second text classification model, wherein each of the at least one adjustment comprises: adjusting the parameters of a current first text classification model using a first subset of the current sample set to obtain an adjusted first text classification model; determining a first evaluation value of each of a plurality of first output categories of the adjusted first text classification model based on a second subset of the current sample set, wherein the second subset is disjoint with the first subset, and the first evaluation value indicates a classification effect of the adjusted first text classification model on the corresponding first output category; in response to the first evaluation value of any first output category in the plurality of first output categories being less than a threshold value, deleting, from the current sample set, a sample with a category label of the first output category and the first evaluation value of the first output category being less than the threshold value, and performing a next adjustment on the parameters of the first text classification model using the sample set after the deletion, so that the first text classification model does not output the first output category with the first evaluation value less than the threshold value; and in response to the first evaluation value of each of the plurality of first output categories being greater than or equal to the threshold value, determining the adjusted first text classification model as the second text classification model.

2. The method of claim 1, further comprising: classifying a text to be classified using the second text classification model to obtain a predicted category of the text to be classified; obtaining a category annotation result of the text to be classified; and in response to the predicted category being different from the category annotation result, adjusting the parameters of the second text classification model based on the text to be classified and the category annotation result to obtain a trained third text classification model.

3. The method of claim 1, further comprising: determining a predicted threshold value of each of a plurality of second output categories of the second text classification model, wherein the predicted threshold value is used to determine a predicted category of a text to be classified as a second output category in response to a probability that the text to be classified belongs to the second output category output by the second text classification model being greater than the predicted threshold value. The determining the predicted threshold value of each of the plurality of second output categories of the second text classification model comprises: for any candidate predicted threshold value in a plurality of candidate predicted threshold values: determining, based on the candidate predicted threshold value, a predicted category of each sample text in the second subset of the current sample set using the second text classification model; and ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 4. The method of claim 3, wherein, ​ ​ ​ determine a second evaluation value of each of the plurality of second output categories based on the predicted category and the category label of each sample text in a second subset of the current sample set, wherein the second evaluation value indicates a classification effect of the second text classification model on the corresponding second output category; and for any second output category in the plurality of second output categories, determine a candidate prediction threshold in the plurality of candidate prediction thresholds that maximizes the second evaluation value of the second output category as the prediction threshold of the second output category.

5. The method of any one of claims 1-4, wherein, The adjusting the parameters of the current first text classification model using the first subset of the current sample set to obtain an adjusted first text classification model comprises: obtaining a keyword table used for extracting text features; for any sample text in the first subset: extracting a text feature of the sample text based on the keyword table; and inputting the text feature into the current first text classification model to obtain a predicted category of the sample text output by the current first text classification model; determining a loss value of the current first text classification model based on the predicted category and the category label of each sample text in the first subset; and adjusting the parameters of the current first text classification model based on the loss value.

6. The method of claim 5, wherein, The keyword table is determined by statistical analysis on sample texts in the sample set.

7. A text classification method, comprising: obtaining a text to be classified; performing word segmentation on the text using a plurality of word segmentation granularities to obtain a word set of the text; and for each word in the word set, determining a category to which the word belongs using a text classification model, wherein the text classification model is trained according to the method of any one of claims 1-6. The text is value data in interface data or web page data.

8. The method of claim 7, wherein, 9. A training device of a text classification model, comprising: a first obtaining module configured to obtain a sample set, wherein the sample set comprises a plurality of samples, each sample in the plurality of samples comprises a sample text and a category label of the sample text, wherein the category label is an identification of a field in a database, and the sample text with the category label is an exemplary data item under the field; and a first adjusting module configured to adjust parameters of a first text classification model based on the sample set at least once to obtain a trained second text classification model, wherein the first adjusting module comprises: an adjusting unit configured to adjust parameters of a current first text classification model using a first subset of a current sample set to obtain an adjusted first text classification model; a first evaluation unit configured to determine a first evaluation value of each of a plurality of first output categories of the adjusted first text classification model using a second subset of the current sample set, wherein the second subset is disjoint from the first subset, and the first evaluation value indicates a classification effect of the adjusted first text classification model on the corresponding first output category. ​ The deleting unit is configured to, in response to a first evaluation value of any first output category in the plurality of first output categories being less than a threshold value, delete, from the current sample set, a sample whose category label is the first output category and whose first evaluation value of the first output category is less than the threshold value, and perform next adjustment of parameters of the first text classification model using the sample set after the deletion, so that the first text classification model does not output the first output category whose first evaluation value is less than the threshold value. The first determining unit is configured to, in response to the first evaluation value of each of the plurality of first output categories being greater than or equal to the threshold value, determine the adjusted first text classification model as the second text classification model.

10. The apparatus of claim 9, further comprising: a classification module configured to classify a text to be classified using the second text classification model to obtain a predicted category of the text to be classified; a second obtaining module configured to obtain a category label result of the text to be classified; and a second adjusting module configured to, in response to the predicted category being different from the category label result, adjust parameters of the second text classification model based on the text to be classified and the category label result to obtain a third text classification model trained.

11. The apparatus of claim 9, further comprising: a determining module configured to determine a plurality of second evaluation values of each of a plurality of second output categories of the second text classification model, wherein the predicted threshold value is used to determine that a predicted category of a text to be classified is a second output category in response to a probability that the text to be classified output by the second text classification model belongs to the second output category being greater than the predicted threshold value. The determining module comprises:

12. The apparatus of claim 11, wherein, a second evaluating unit configured to, for any candidate predicted threshold value in a plurality of candidate predicted threshold values: determine, based on the candidate predicted threshold value, a predicted category of each sample text in a second subset of the current sample set using the second text classification model; and determine, based on the predicted category and the category label of each sample text in the second subset of the current sample set, a second evaluation value of each of the plurality of second output categories, wherein the second evaluation value indicates a classification effect of the second text classification model on the corresponding second output category; and a second determining unit configured to, for any second output category in the plurality of second output categories, determine, as a predicted threshold value of the second output category, a candidate predicted threshold value in the plurality of candidate predicted threshold values that makes a second evaluation value of the second output category maximum. The adjusting unit comprises: an obtaining subunit configured to obtain a keyword table used to extract text features; 13. The apparatus of any of claims 9-12, wherein, a predicting subunit configured to, for any sample text in the first subset: extract, based on the keyword table, a text feature of the sample text; and input the text feature into the current first text classification model to obtain a predicted category of the sample text output by the current first text classification model. ​ ​ A determining sub-unit, configured to determine a loss value of the current first text classification model based on the predicted category of each sample text in the first subset and the category label. An adjusting sub-unit, configured to adjust parameters of the current first text classification model based on the loss value.

14. The apparatus of claim 13, wherein, The keyword table is determined by statistical analysis on sample texts in the sample set.

15. A text classification apparatus, comprising: an obtaining module configured to obtain a text to be classified; a word segmentation module configured to segment the text using a plurality of word segmentation granularities to obtain a word set of the text; and a classification module configured to determine, for each word in the word set, a category to which the word belongs using a text classification model, wherein the text classification model is trained by the apparatus of any one of claims 9-14.

16. An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8. The computer instructions are used to enable a computer to perform the method of any one of claims 1-8.

17. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer program instructions, when executed by a processor, implement the method of any one of claims 1-8.

18. A computer program product comprising computer program instructions, wherein, The computer program instructions, when executed by a processor, implement the method of any one of claims 1-8.

Images