Text classification method, device and electronic device using machine learning

Abstract

The disclosed embodiments relate to a text classification method, device, electronic device, and storage medium using machine learning. The method includes: in response to a text classification application creation instruction for a text classification task, obtaining text classification application configuration information and text annotation data; creating a text classification application based on the text classification application configuration information, wherein the text classification application is at least used to explore a model solution based on the text classification application configuration information and text annotation data to obtain a model solution; in response to an instruction to start the text classification application, deploying the text classification application online, and generating a text classification service address, so that the text classification application provides an online estimation service for the text classification task based on the text classification service address. It can be seen that for people who do not have professional knowledge of machine learning, they can automatically create a text classification application by specifying a text classification scenario, a text classification task, and a text classification application configuration information, thereby reducing the cost of implementing text classification capabilities.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of machine learning technology and specifically to a text classification method, device, electronic device and non-transitory computer-readable storage medium applying machine learning. Background Art

[0002] Natural Language Processing (NLP) text classification is increasingly used in all walks of life, but NLP text classification is still a field that requires strong professional skills. In many scenarios, in addition to classifying Chinese text, there are also many demands for classifying English text, and there is a high demand for products that can simultaneously meet the needs of Chinese and English classification.

[0003] At present, the construction of NLP text classification models is mainly achieved by professional modelers through manual coding or using graphical interfaces (such as DAG, Directed Acyclic Graph). However, both of these implementation methods require high manpower and time costs to obtain a relatively satisfactory model, which is not conducive to large-scale exploration and application of the model. In addition, not only the software cost, but also the procurement of graphics processing units (GPUs) for servers for many small scenarios is a large expense. Therefore, it is necessary to provide a text classification solution that applies machine learning. Summary of the invention

[0004] One purpose of the embodiments of the present disclosure is to provide a new text classification solution using machine learning.

[0005] In a first aspect, the present disclosure provides a text classification method using machine learning, including:

[0006] In response to a text classification application creation instruction of a text classification task, obtaining text classification application configuration information and text annotation data; the text annotation data includes a text column and a label column;

[0007] Based on the text classification application configuration information, create a text classification application; wherein the text classification application is a first service program instance, and is at least used to perform model solution exploration based on the text classification application configuration information and the text annotation data to obtain a model solution;

[0008] In response to an instruction to start the text classification application, the text classification application is deployed online, and a text classification service address is generated, so that the text classification application provides an online estimation service for the text classification task based on the text classification service address; wherein the online estimation service is performed based on online related data of the text classification task.

[0009] In some embodiments, each line of text in the text column corresponds to one or more tags.

[0010] In some embodiments, the text classification application is also used to perform model self-learning based on the online relevant data and the model scheme to obtain an online text classification model.

[0011] In some embodiments, the method further comprises:

[0012] Returning the online related data to the first database;

[0013] Returning the labeled intermediate data generated by the model solution based on the online related data to the first database;

[0014] Providing a reflow data annotation interface to obtain reflow annotation data, and returning the reflow annotation data to the first database again, wherein the reflow annotation data is obtained by annotating the data that flows back to the first database;

[0015] Correspondingly, the text classification application performs model self-learning based on the online related data, the labeled intermediate data, the reflow annotation data and the model scheme to obtain an online text classification model.

[0016] In some embodiments, the method further includes: deploying the online text classification model online to provide batch estimation services for the text classification task.

[0017] In some embodiments, the batch estimation service includes: providing a batch estimation service interface, the online text classification model obtains the data set to be batch estimated for the text classification task based on the batch estimation service interface, and outputs the batch estimation result based on the data set to be batch estimated.

[0018] In some embodiments, the data set to be batch estimated includes a plurality of data columns, and the plurality of data columns includes a text column;

[0019] The outputting a batch estimation result based on the data set to be batch estimated includes:

[0020] Perform batch estimation based on the text column in the data set to be batch estimated to obtain an estimated label column;

[0021] The estimated label column is concatenated with the data set to be batch estimated to obtain a batch estimation result and output it.

[0022] In some embodiments, deploying the online text classification model online includes: replacing a machine learning model that has been deployed online with the online text classification model.

[0023] In some embodiments, before the text classification application creation instruction in response to the text classification task, the method further includes:

[0024] A user interface is provided, based on which a text classification scenario and a text classification task input by a user are received, and based on which a text classification application creation instruction triggered by a user is received, wherein the text classification application creation instruction corresponds to the text classification scenario and the text classification task input by the user.

[0025] In some embodiments, the creating a text classification application based on the text classification application configuration information includes:

[0026] Packaging the text classification application configuration information, the second service program instance and the third service program instance into a text classification application;

[0027] The second service program instance is used to perform model solution exploration based on the text classification application configuration information and the text annotation data to obtain a model solution;

[0028] The third service program instance is used to perform model self-learning based on the online related data and the model scheme to obtain an online text classification model.

[0029] In some embodiments, the text classification application configuration information includes one or more of the following:

[0030] How often the model learns itself;

[0031] Evaluation indicators for model self-learning;

[0032] Dataset language;

[0033] Use GPU acceleration;

[0034] The percentage of data used to evaluate the model.

[0035] In some embodiments, the text classification application is used to perform model solution exploration based on the data in the first database, the text classification application configuration information and the text annotation data to obtain a model solution; wherein the model solution includes the following solution sub-items: feature engineering solution, model algorithm and model hyperparameters;

[0036] Correspondingly, deploying the text classification application online includes: deploying the explored model solution online.

[0037] In some embodiments, the online text classification model is obtained by training an offline model; wherein the offline model is a model generated in the process of exploring the model solution, and when the explored model solution is deployed online, the offline model is also deployed online.

[0038] In some embodiments, the online text classification model is a model generated based on the model algorithm and model hyperparameters in the model scheme; and when the explored model scheme is deployed online, the offline model is not deployed online.

[0039] In a second aspect, the present disclosure also provides a text classification device using machine learning, including:

[0040] A text classification application creation module, used to obtain text classification application configuration information and text annotation data in response to a text classification application creation instruction of a text classification task; the text annotation data includes a text column and a label column; based on the text classification application configuration information, a text classification application is created; wherein the text classification application is a first service program instance, and is at least used to perform model solution exploration based on the text classification application configuration information and the text annotation data to obtain a model solution;

[0041] A text classification application startup module is used to deploy the text classification application online in response to an instruction to start the text classification application, and generate a text classification service address so that the text classification application provides an online estimation service for the text classification task based on the text classification service address; wherein the online estimation service is performed based on online related data of the text classification task.

[0042] In some embodiments, each line of text in the text column corresponds to one or more tags.

[0043] In some embodiments, the text classification application is also used to perform model self-learning based on the online relevant data and the model scheme to obtain an online text classification model.

[0044] In some embodiments, the text classification application is further used to return the online relevant data to the first database, and return the labeled intermediate data generated by the model scheme based on the online relevant data to the first database;

[0045] The text classification device using machine learning further includes a labeling interface module, which is used to provide a reflow data labeling interface to obtain reflow labeling data, and reflow the reflow labeling data to the first database again, wherein the reflow labeling data is obtained by labeling the data reflowing to the first database;

[0046] Correspondingly, the text classification application performs model self-learning based on the online related data, the labeled intermediate data, the reflow annotation data and the model scheme to obtain an online text classification model.

[0047] In some embodiments, the text classification application startup module is further used to deploy the online text classification model online to provide batch estimation services for the text classification task.

[0048] In some embodiments, the text classification application startup module is also used to provide a batch estimation service interface, and the online text classification model obtains the data set to be batch estimated for the text classification task based on the batch estimation service interface, and outputs the batch estimation results based on the data set to be batch estimated.

[0049] In some embodiments, the data set to be batch estimated includes a plurality of data columns, and the plurality of data columns includes a text column;

[0050] The online text classification model outputs batch estimation results based on the data set to be batch estimated, including: performing batch estimation based on the text column in the data set to be batch estimated to obtain an estimated label column, splicing the estimated label column with the data set to be batch estimated, obtaining batch estimation results and outputting them.

[0051] In some embodiments, the text classification application startup module deploys the online text classification model online, including: replacing the deployed machine learning model with the online text classification model.

[0052] In some embodiments, the text classification application creation module is further used to:

[0053] Before responding to a text classification application creation instruction for a text classification task, a user interface is provided, based on which a text classification scenario and a text classification task input by a user are received, and based on the user interface, a text classification application creation instruction triggered by a user is received, wherein the text classification application creation instruction corresponds to the text classification scenario and the text classification task input by the user.

[0054] In some embodiments, the text classification application creation module creates a text classification application based on the text classification application configuration information, including:

[0055] Packaging the text classification application configuration information, the second service program instance and the third service program instance into a text classification application;

[0056] The second service program instance is used to perform model solution exploration based on the text classification application configuration information and the text annotation data to obtain a model solution;

[0057] The third service program instance is used to perform model self-learning based on the online related data and the model scheme to obtain an online text classification model.

[0058] In some embodiments, the text classification application configuration information includes one or more of the following:

[0059] How often the model learns itself;

[0060] Evaluation indicators for model self-learning;

[0061] Dataset language;

[0062] Use GPU acceleration;

[0063] The percentage of data used to evaluate the model.

[0064] In some embodiments, the text classification application is used to perform model solution exploration based on the data in the first database, the text classification application configuration information and the text annotation data to obtain a model solution; wherein the model solution includes the following solution sub-items: feature engineering solution, model algorithm and model hyperparameters;

[0065] Correspondingly, the text classification application startup module deploys the text classification application online, including: deploying the explored model solution online.

[0066] In some embodiments, the online text classification model is obtained by training an offline model; wherein the offline model is a model generated in the process of exploring the model solution, and when the explored model solution is deployed online, the offline model is also deployed online.

[0067] In some embodiments, the online text classification model is a model generated based on the model algorithm and model hyperparameters in the model scheme; and when the explored model scheme is deployed online, the offline model is not deployed online.

[0068] In a third aspect, an embodiment of the present disclosure further proposes an electronic device, comprising: a processor and a memory; the processor is used to execute the method steps of any embodiment of the first aspect by calling a program or instruction stored in the memory.

[0069] In a fourth aspect, the embodiments of the present disclosure further provide a non-transitory computer-readable storage medium for storing programs or instructions, wherein the programs or instructions enable a computer to execute the method steps of any embodiment of the first aspect.

[0070] It can be seen that in at least one embodiment of the present disclosure, even those who do not have professional knowledge of machine learning can create a text classification application by specifying text classification scenarios, text classification tasks, and text classification application configuration information, thereby reducing the cost of implementing NLP text classification capabilities.

[0071] In some embodiments, model self-learning can be performed based on online relevant data and model solutions to obtain an online text classification model, thereby realizing automatic model construction and reducing model construction costs.

[0072] In some embodiments, by managing the data of the text classification scene (including but not limited to scene splicing, etc.), reusable data for model construction is obtained.

[0073] In some embodiments, by deploying the constructed model online, a batch estimation service for text classification tasks can be provided. In addition, using the acquired online data, the model solution obtained through exploration, and the intermediate data generated by the batch estimation service, the model can be self-learned to achieve automatic iterative updates of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure, and a person skilled in the art can also obtain other drawings based on these drawings.

[0075] Figure 1 An exemplary application scenario diagram of text classification using machine learning provided in an embodiment of the present disclosure;

[0076] Figure 2 An exemplary block diagram of a scene module provided in an embodiment of the present disclosure;

[0077] Figure 3 An exemplary block diagram of a text classification device using machine learning provided in an embodiment of the present disclosure;

[0078] Figure 4 An exemplary architecture diagram of providing online estimation service or batch estimation service for a text classification application provided in an embodiment of the present disclosure;

[0079] Figure 5 An exemplary block diagram of an electronic device provided for an embodiment of the present disclosure;

[0080] Figure 6 An exemplary flow chart of a text classification method using machine learning provided in an embodiment of the present disclosure;

[0081] Figures 7 to 15 A schematic diagram of an interface related to a text classification process using machine learning provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0082] In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It is understood that the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. The specific embodiments described herein are only used to explain the present disclosure, rather than to limit the present disclosure. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in the field belong to the scope of protection of the present disclosure.

[0083] It should be noted that, in this document, relational terms such as “first” and “second” are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0084] Since the construction of NLP text classification models is currently mainly completed by professional modelers, and the training cost of professional modelers is relatively high, the gap in professional modelers cannot be quickly filled in the short term, which is not conducive to the large-scale exploration and application of NLP text classification models.

[0085] In addition, after the NLP text classification model has been online for a period of time, the estimated effect of the NLP text classification model will decay, so professional modelers are needed to re-model and tune it, that is, repeated personnel investment is required every certain period of time, so the more modeling is done, the higher the personnel cost. Therefore, how to achieve automatic iterative updates of the model to ensure the estimated effect of the model is also an urgent problem to be solved.

[0086] To this end, the disclosed embodiment provides a text classification solution using machine learning, which can form a learning closed loop for NLP text classification applications from problem definition, to modeling, to model online service, offline batch prediction, and subsequent feedback collection and iterative update of the model. By automatically training the NLP text classification model and automatically launching the NLP text classification application, the difficult and costly implementation of NLP text classification capabilities is fundamentally solved. Even people without NLP-related experience can complete the entire process of text classification scenario implementation. Both CPU and GPU modes are supported, and small scenarios without GPUs can also complete text classification exploration. It also supports the classification of Chinese and English texts, further expanding the scope of scenario coverage.

[0087] In some embodiments, non-professional modelers or personnel without NLP-related experience can specify text classification scenarios, text classification tasks, text classification application configuration information, and text annotation data, and then the text classification scheme using machine learning can automatically create a text classification application based on the text classification application configuration information, wherein the text classification application can explore the model scheme based on the text classification application configuration information and the text annotation data to obtain the model scheme; and then the created text classification application can be deployed online, and a text classification service address can be generated, so that the text classification application can provide an online estimation service for the text classification task based on the text classification service address; wherein the online estimation service is based on the online related data of the text classification task. It can be seen that in the disclosed embodiments, the created text classification application can automatically explore the model scheme, realize the full automation of the model building process, and thus reduce the modeling cost.

[0088] In some embodiments, the created text classification application can also perform model self-learning based on the acquired online relevant data of the text classification task and the model solution obtained through exploration to obtain an online text classification model. It can be seen that in the embodiments of the present disclosure, the created text classification application can perform model self-learning to achieve automatic iterative update of the model and ensure the estimated effect of the model. Even without professional modeling personnel and machine learning knowledge reserves, models and applications can be built with a low threshold.

[0089] Figure 1 An exemplary application scenario diagram of text classification using machine learning provided in the embodiment of the present disclosure. Figure 1 As shown, the scenario module 11 and the text classification device 12 using machine learning can be connected to the text classification scenario. The text classification scenario can be specified by the user. Furthermore, the user can also specify the text classification task in the text classification scenario. There can be multiple text classification tasks. For each text classification task, the text classification device 12 using machine learning can create a corresponding text classification application. For example, one task creates a text classification application; different tasks create different text classification applications. The text classification application is used to process the corresponding text classification task, such as a real-time estimation task or a batch estimation task. Among them, real-time estimation is to perform estimation (i.e. text classification) upon receiving an estimation request (i.e. text classification request); while batch estimation is non-real-time estimation, which is performed in batches by timing or event triggering. For example, when the preset batch estimation conditions are met, batch estimation is performed on multiple estimation requests. For example, when the estimation requests accumulate a preset number of items, batch estimation is performed.

[0090] The scenario module 11 is used to implement the text classification scenario definition. In some embodiments, the text classification scenario definition can be completed by the user, for example, the user can define a news classification scenario. Accordingly, the scenario module 11 receives the scenario definition information input by the user. In some embodiments, the scenario module 11 can provide a user interface, and the user inputs the scenario definition information through the user interface to specify the text classification scenario and the text classification task. In some embodiments, the scenario definition information may include but is not limited to one or more of the following: scenario name, task name, task ID, task-related data definition, etc. The related data definitions of different tasks are different. In some embodiments, the related data definition can be a data table schema definition. In some embodiments, the Schema definition includes but is not limited to one or more of the following: the names of one or more data tables, the fields included in each data table, and the data relationships between multiple data tables.

[0091] In some embodiments, the scenario module 11 is also used for data access. For example, the scenario module 11 obtains relevant data of the text classification task based on the text classification task of the text classification scenario. In some embodiments, the relevant data may include but is not limited to: request data, exposure data, and feedback data.

[0092] Scenario definition information may include but is not limited to:

[0093] 1) Basic information of the scene, such as name and notes. Used to identify and distinguish scenes.

[0094] 2) Define relevant data. Relevant data includes but is not limited to: request data, exposure data, feedback data, and business data. Request data refers to information sent to a text classification application. For example, if there are 10,000 news items, and you want to classify them through a text classification application in combination with other information, these 10,000 news items are request data. After being estimated by the text classification application, when you actually read the news, you will not read all 10,000 news items. The business party (or customer) will only choose the news that they are interested in, and may only read 100 of them. These 100 items are exposure data. Finally, the category of the news you actually read is feedback data. In addition to request data, exposure data, and feedback data, the scenario may also contain business data. Business data is other information that may help improve the estimated effect of the text classification application, such as basic information of customers, customer notes, and other BO (BusinessObject) data. Among them, there may be no business data or there may be multiple business data.

[0095] a) Define the schema of each related data stream (request data stream, exposure data stream, feedback data stream, and business data stream). For example, what fields are included in each data stream? The specific information that needs to be configured includes field name, field type, and field remarks (optional).

[0096] b) It should be pointed out that after obtaining the request data and exposure data, the behavior data will be constructed by inner join based on the request data and exposure data. The behavior data can be used for subsequent model solution exploration and model self-learning.

[0097] 3) Define the relationship between data description information and each data table. Specifically:

[0098] a) Define the time field of the behavior data. Select a time type field in the behavior data as the primary time field. This time field should be the actual time when the behavior occurs.

[0099] b) Define the feedback field (label) and type of the feedback data. Select the label field in the feedback data. For binary classification scenarios, the label is 1 or 0, which indicates positive and negative samples. In regression scenarios, the label is a continuous value that indicates the actual situation, such as the value of PM2.5. After selecting the label field, you also need to select the type of the label, whether it is binary classification, regression, or multi-classification.

[0100] c) Define the concatenation fields of behavior data and feedback data. Define which fields of behavior data and feedback data are used as the association key for concatenation. The concatenation key can also support multiple groups of keys, that is, a piece of behavior data is considered to correspond to a piece of feedback data only when multiple fields are equal.

[0101] d) Define the tag type of the fields in each data and whether to use it. For a certain field type, in order for automatic modeling to correctly identify the business meaning to achieve better results, the user needs to specify the specific tag type. For example, for an int type field, whether it is a continuous type or a discrete type, which will determine what data change strategy the automatic modeling algorithm uses for this column of data. In addition, it is also necessary to mark whether each column of data is used in model solution exploration, because in text classification scenarios, there may be some meaningless fields that can be excluded from learning, or some fields are strongly correlated with labels and should not be included in learning, so they need to be labeled during use.

[0102] e) Define the relationship between data tables. It is necessary to define the data relationship between behavioral data and business data to facilitate the automatic modeling of multiple tables during automatic modeling. Relationship types include but are not limited to 1:1 and 1:N. Among them, in the 1:N relationship type, it is also necessary to specify the table type of the secondary table (assuming table A and table B, the data of table B will be spliced ​​into table A, then table B is called the secondary table of table A), whether it is an event table or a slice table. Data table splicing not only supports the connection between behavioral tables and business data tables, but also supports direct connection between business data tables.

[0103] After completing the above definition, a text classification scenario can be formally created. After the text classification scenario is created, the scenario module 11 will automatically start a data splicing task to splice the request data and exposure data into behavior data for subsequent model solution exploration and model self-learning.

[0104] The text classification device 12 using machine learning can be used to implement text classification application configuration. In some embodiments, the text classification application configuration can be completed by the user, for example, which business data is used to participate in model solution exploration and model self-learning, and for example, the data range used for model self-learning. Accordingly, the text classification device 12 using machine learning can receive text classification application configuration information and text annotation data input by the user, and the text annotation data is used for model solution exploration, wherein the text annotation data includes at least one text column and one label column, the text column stores text data, the label column stores annotation data, and each line of text in the text column corresponds to one or more labels. In some embodiments, the text classification device 12 using machine learning can provide a user interface, and the user inputs the text classification application configuration information through the user interface and uploads the text annotation data.

[0105] In some embodiments, from the user's perspective, after the user defines the scenario, he or she will want to create a corresponding text classification application. Therefore, the text classification device 12 using machine learning can provide a user interface, and based on the user interface, receive a text classification application creation instruction triggered by the user, and the text classification application creation instruction corresponds to the text classification scenario and text classification task input by the user. After the user triggers the instruction to create the text classification application, for example, the user clicks the "Text Classification Application Creation" button, the text classification device 12 using machine learning can respond to the text classification application creation instruction and display a user interface to obtain the text classification application configuration information and text annotation data input by the user through the user interface.

[0106] In some embodiments, the text classification application configuration information may include but is not limited to one or more of the following:

[0107] How often the model learns itself;

[0108] Evaluation indicators for model self-learning;

[0109] Dataset language;

[0110] Use GPU acceleration;

[0111] The percentage of data used to evaluate the model.

[0112] The frequency of model self-learning can be configured to trigger learning every time new data arrives or to trigger learning every time "N" new data are added.

[0113] Among them, the evaluation indicators of model self-learning include one or more of the four indicators: P (precision), R (recall), F1, and ACC (accuracy).

[0114] The data set language includes two language models: Chinese and English. Different models are automatically matched for training according to the language type selected by the user.

[0115] Among them, use GPU acceleration (switch): after turning on, the GPU will be used to train the model. If there are no GPU resources, you can turn off this switch and use CPU resources to train the model.

[0116] The model evaluation data ratio is the ratio of data used for model evaluation each time learning is performed. In this embodiment, the default value is 8%.

[0117] In some embodiments, the text classification application configuration information may also include but is not limited to one or more of the following:

[0118] Business data used for model solution exploration and model self-learning;

[0119] The scope of data used for model solution exploration and model self-learning;

[0120] The computing power level for model solution exploration and model self-learning;

[0121] The data range for which the model is evaluated;

[0122] Whether the model is automatically launched;

[0123] Whether to use the offline model obtained by model solution exploration.

[0124] The computing power level can be understood as the complexity of model solution exploration and model self-learning. The higher the computing power level, the wider the search space for model solution exploration and model self-learning will be, and the better the estimated effect of the model obtained by model self-learning.

[0125] The model's evaluation data range specifies the data range of the model used to evaluate the model's self-learning output.

[0126] Among them, whether the model is automatically launched online specifies whether the model generated by the continuous iteration and update of the model self-learning is automatically launched online. If the model is set to be automatically launched online, the model generated by the model self-learning will be automatically launched online when the model effect is better than the model that has been deployed and launched. If the model is not set to be automatically launched online, the model generated by the model self-learning can only be launched manually.

[0127] Among them, whether to use the offline model obtained by model solution exploration specifies whether to put the offline model online. If the offline model is not used and only the model solution is put online, the model solution will not output the estimated result. What is output to the text classification scenario is a default estimated result (for example, a default prediction value). The default prediction value is, for example, 0.5. It is necessary to wait for the model to self-learn and produce a model and go online before the model can output the estimated result. If the offline model is used, that is, the model solution is put online at the same time as the offline model, the offline model can output the estimated result, but because the data used in the model solution exploration may be different from the online data, the estimation effect of the offline model may be poor.

[0128] In some embodiments, the text classification device 12 using machine learning can also be used to create a text classification application. For example, the text classification device 12 using machine learning creates a text classification application based on the text classification application configuration information; wherein the text classification application is a first service program instance, which is at least used to explore a model solution based on the text classification application configuration information and the text annotation data uploaded by the user to obtain a model solution. The model solution is a collection of multiple strategies for modeling, including but not limited to: how to filter data, how to construct features, how to tune model hyperparameters, how to select a model, how to train a model, and other strategies. In some embodiments, after the text classification device 12 using machine learning creates a text classification application, a user interface can be displayed to prompt the user that the text classification application has been created, and the user can trigger the instruction to start the text classification application, for example, the user can click the "text classification application start" button on the user interface.

[0129] In some embodiments, the text classification device 12 using machine learning can also be used to deploy text classification applications online. For example, the text classification device 12 using machine learning can respond to an instruction to start a text classification application, deploy the text classification application online, and generate a text classification service address, so that the text classification application provides an online estimation service for the text classification task based on the text classification service address; wherein the online estimation service is based on online relevant data of the text classification task. Among them, the text classification application can also be used to perform model self-learning based on online relevant data and the model scheme obtained by exploration to obtain an online text classification model. Among them, model self-learning can use online relevant data to perform model automatic learning on a scheduled or event-triggered basis, so that the latest data information and business changes can also be learned by the model, ensuring that the self-learning model effect continues to be good.

[0130] In some embodiments, the functionality of the scene module 11 may be integrated into a text classification device 12 that applies machine learning.

[0131] Figure 2An exemplary block diagram of a scene module 20 provided in an embodiment of the present disclosure. In some embodiments, the scene module 20 can be implemented as Figure 1 The scene module 11 or a part of the scene module 11 in the embodiment.

[0132] like Figure 2 As shown, the scene module 20 can be divided into multiple units, for example including but not limited to: a data access unit 21, a scene puzzle unit 22 and a data management unit 23.

[0133] The data access unit 21 is used to perform data connection with the text classification scenario. In some embodiments, the data access unit 21 can obtain relevant data of the text classification task based on the text classification task of the text classification scenario. In some embodiments, the data access unit 21 can obtain the relevant data definition of the text classification task, and then perform data connection with the text classification scenario based on the relevant data definition to obtain the relevant data of the text classification task.

[0134] In some embodiments, the data access unit 21 can create a data interface corresponding to the relevant data definition based on the relevant data definition of the text classification task, and then obtain the relevant data of the text classification task through the data interface. The data interface uses a dynamic data table or a data group as an interface, or the data interface is an encapsulation interface, which is a unified interface obtained by encapsulating the dynamic data table and the data group.

[0135] In some embodiments, the data interface is an interface with a dynamic data table or a data group. The data access unit 21 uses a dynamic data table or a data group as a data storage carrier. A dynamic data table refers to a data table to which data can still be added (append) after the data table is created. A data group refers to a combination of a series of isomorphic (same data fields) data slices. When adding new data, the append of data is realized by adding a new data slice to the data group. In this embodiment, when the user needs to use more data for training or estimation, the corresponding dynamic data table or data group is used as an interface to import data, such as text annotation data. The way to import data includes but is not limited to one or more of single import, timed import and streaming import. Among them, streaming import is, for example, Kafka (distributed publish-subscribe message system) import, etc. From the data source, support local import, database import, FTP (File Transfer Protocol, File Transfer Protocol) import, HDFS (Hadoop Distributed File System, Hadoop Distributed File System) import, hive (Hadoop-based data warehouse tool) import and other methods to meet the data import requirements of different text classification scenarios.

[0136] In some embodiments, the data interface is an encapsulation interface, which is a unified interface obtained by encapsulating dynamic data tables and data groups. In this embodiment, by encapsulating dynamic data tables and data groups into a unified data interface, the underlying data storage implementation is not exposed to the user, thereby improving the user experience. For example, through the encapsulation interface, only four types of data interfaces, namely request data, impression data, feedback data, and business data, need to be exposed to the user. The user only needs to perceive these four data interfaces, and no longer needs to perceive what the specific corresponding data groups are.

[0137] The scene splicing unit 22 is used to splice the request data and the exposure data in the related data to obtain behavior data, such as text classification behavior. In some embodiments, the scene splicing unit 22 constructs the behavior data (also called sample data) according to the request data and the exposure data by inner joining.

[0138] In some embodiments, the scene stitching unit 22 may use a filter to process and flatten the request data and the exposure data to construct the behavior data.

[0139] For example, the scene splicing unit 22 can use a filter to filter the request data based on the exposure data to obtain intersection data, and then flatten the intersection data to obtain behavior data. For example, the exposure data has 10 data items, the request data has 12 data items, and the exposure data and the request data have 10 identical data items. The scene splicing unit 22 filters through the filter to obtain these 10 identical data items as the intersection data, and filters out the different data, and then flattens the intersection data (these 10 identical data items) to obtain behavior data.

[0140] The data management unit 23 is used to manage the data in the first database and the data in the second database. In some embodiments, the first database is an offline database. For example, the offline database can be a distributed file storage system (HDFS, Hadoop Distributed File System), or other offline databases. In some embodiments, the second database is an online database, such as a real-time feature storage engine (RtiDB), or other online databases.

[0141] In some embodiments, the data management unit 23 may accumulate the online related data of the text classification task obtained by the data access unit 21 into the first database. In some embodiments, the data management unit 23 may accumulate the behavior data obtained by the scene puzzle unit 22 into the first database. In some embodiments, the data management unit 23 may store the online related data into the second database.

[0142] In some embodiments, the division of each unit in the scene module 20 is only a logical function division. There may be other division methods in actual implementation. For example, at least two units of the data access unit 21, the scene puzzle unit 22 and the data management unit 23 can be implemented as one unit; the data access unit 21, the scene puzzle unit 22 or the data management unit 23 can also be divided into multiple sub-units. It is understandable that each unit or sub-unit can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application.

[0143] Figure 3 An exemplary block diagram of a text classification device 30 applying machine learning provided in an embodiment of the present disclosure. In some embodiments, the text classification device 30 applying machine learning can be implemented as Figure 1 A text classification device 12 or a part thereof using machine learning.

[0144] like Figure 3 As shown, the text classification device 30 using machine learning can be divided into multiple units, for example including but not limited to: a text classification application creation module 31 and a text classification application startup module 32.

[0145] The text classification application creation module 31 is used to implement the text classification application configuration and create the text classification application. In some embodiments, the text classification application creation module 31 responds to the text classification application creation instruction of the text classification task to obtain the text classification application configuration information and text annotation data. In some embodiments, the text classification application creation module 31 provides a user interface to receive the text classification application creation instruction triggered by the user through the user interface. When the user triggers the text classification application creation instruction, the text classification application creation module 31 responds to the text classification application creation instruction of the text classification task to display the user interface to obtain the text classification application configuration information and text annotation data input by the user through the user interface.

[0146] In some embodiments, the text classification application creation module 31 creates a text classification application based on the text classification application configuration information; wherein the text classification application is a first service program instance, which is at least used to explore a model solution based on the text classification application configuration information and the text annotation data to obtain a model solution.

[0147] In some embodiments, the text classification application can explore the model scheme based on the data in the first database (such as one or more of request data, sample data, feedback data, business data, and exposure data) and text annotation data to obtain a model scheme. The model scheme includes the following scheme sub-items: feature engineering scheme, model algorithm, and model hyperparameters. The feature engineering scheme has at least a table-splitting function. The feature engineering scheme may also have other functions, such as extracting features from data for use by the model algorithm or model. The model algorithm may be a commonly used machine learning algorithm, such as a supervised learning algorithm, including but not limited to: LR (Logistic Regression), GBDT (Gradient Boosting Decision Tree), DeepNN (Deep Neural Network), etc. The hyperparameters of the model are parameters pre-set before machine learning to assist model training, such as the number of categories in the clustering algorithm, the step size of the gradient descent method, the number of layers of the neural network, the learning rate of training the neural network, etc.

[0148] In some embodiments, when exploring model solutions, the text classification application may generate at least two model solutions, wherein at least one solution sub-item is different between different model solutions. In some embodiments, the text classification application uses at least two model solutions to perform model training based on the data in the first database, and can obtain the parameters of the model itself, wherein the parameters of the model itself are, for example, weights in a neural network, support vectors in a support vector machine, coefficients in a linear regression or a logistic regression, etc. In some embodiments, the text classification application may evaluate the models trained by at least two model solutions based on machine learning model evaluation indicators, and then select from at least two model solutions based on the evaluation results to obtain the explored model solution. The machine learning model evaluation indicators are, for example, AUC (Area Under Curve) values, etc.

[0149] In some embodiments, the text classification application may return the online relevant data of the text classification task to the first database, and return the labeled intermediate data generated by the model scheme based on the online relevant data to the first database. Among them, the intermediate data can be the wide table feature data of the estimated sample (which can be understood as behavioral data). In some embodiments, the text classification device 30 using machine learning provides a reflow data annotation interface to obtain the reflow annotation data, and reflow the reflow annotation data to the first database again, wherein the reflow annotation data is obtained by annotating the data reflowed to the first database. In some embodiments, the reflow data annotation interface can be provided by a special annotation platform, which can call and annotate the data from the data reflowed to the first database according to the dimensions of time and data volume, and the annotated data will flow back to the first database again, further improving the self-learning effect of the model.

[0150] In some embodiments, the text classification application performs model self-learning based on online related data, labeled intermediate data, reflow annotation data and model scheme to obtain an online text classification model.

[0151] In some embodiments, the text classification application creation module 31 can package the text classification application configuration information, the second service program instance and the third service program instance into a text classification application. The second service program instance is used to perform model solution exploration based on the text classification application configuration information and the text annotation data to obtain a model solution. The third service program instance is used to perform model self-learning based on the online relevant data of the text classification task and the explored model solution to obtain an online text classification model.

[0152] The text classification application startup module 32 is used to deploy the text classification application online. In some embodiments, the text classification application startup module 32 responds to the instruction to start the text classification application, deploys the text classification application online, and generates a text classification service address, so that the text classification application provides an online estimation service for the text classification task based on the text classification service address; wherein the online estimation service is performed based on the online relevant data of the text classification task.

[0153] In some embodiments, the text classification application startup module 32 can deploy the model solution explored by the second service program instance of the text classification application online. Accordingly, the model solution deployed online can be based on the labeled intermediate data generated by the online relevant data of the text classification task. In some embodiments, the third service program instance of the text classification application can perform model self-learning based on the online relevant data of the text classification task, the model solution explored by the second service program instance, and the intermediate data generated by the model solution to obtain an online text classification model.

[0154] In some embodiments, when the text classification application startup module 32 deploys the model solution online, the offline model obtained in the model solution exploration process is also deployed online. The offline model is trained based on the relevant data of the text classification task and the text annotation data accumulated in the first database (i.e., the offline database), and after the offline model is deployed online, it can provide estimation services for the relevant data of the text classification scenario. Therefore, although the data obtained by calculating the online and offline features may be inconsistent, the same source of online and offline data is still achieved.

[0155] In some embodiments, the third service program instance of the text classification application obtains an online text classification model by training an offline model; wherein the offline model is a model generated in the process of exploring the model solution by the second service program instance of the text classification application, and when the text classification application startup module 32 deploys the model solution online, the offline model is also deployed online. In some embodiments, the third service program instance of the text classification application trains the offline model through the model algorithm and the hyperparameters of the model in the model solution, updates the parameter values ​​of the offline model itself, and obtains the online text classification model.

[0156] In some embodiments, the text classification application startup module 32 only deploys the model solution online, but does not deploy the offline model obtained in the model solution exploration process online, which can avoid the problem that the estimated effect of the deployed offline model is poor due to the inconsistency of the data obtained by online feature calculation and offline feature calculation after the offline model is directly deployed online. In addition, since only the model solution is deployed online and the offline model is not deployed online, the estimated result will not be generated. When the request data is received, the default estimated result is output to the text classification scene, and the text classification scene ignores the default estimated result after receiving it.

[0157] In some embodiments, the third service program instance of the text classification application can perform model self-learning to generate an online text classification model based on online relevant data of the text classification task, the model algorithm and model hyperparameters in the model solution obtained by exploring the second service program instance, and the intermediate data generated by the model solution; and when the text classification application startup module 32 deploys the model solution online, the offline model is not deployed online.

[0158] In some embodiments, the text classification application startup module 32 can deploy the online text classification model online so that the online text classification model provides batch estimation services for text classification tasks. In some embodiments, the text classification application startup module 32 can provide a batch estimation service interface, which is used to obtain the data set to be batch estimated for the text classification task. Accordingly, the deployed online text classification model can obtain the data set to be batch estimated through the batch estimation service interface, and output the batch estimation results based on the data set to be batch estimated.

[0159] In some embodiments, the data set to be batch estimated includes multiple data columns, the multiple data columns include a text column, and may also include other columns, such as an ID column and a remark column, etc. The online text classification model outputs batch estimation results based on the data set to be batch estimated, including: performing batch estimation based on the text column in the data set to be batch estimated to obtain an estimated label column, concatenating the estimated label column with the data set to be batch estimated, obtaining batch estimation results and outputting them.

[0160] In some embodiments, taking a piece of request data as an example, when the online text classification model receives a piece of request data, based on the feature engineering solution in the model solution deployed online, the data in the second database and the received request data are used to perform online real-time feature calculations to obtain feature data of the estimated sample. In some embodiments, when the online text classification model receives request data, based on the feature engineering solution in the model solution deployed online, the data in the second database and the received request data are pieced together and online real-time feature calculations are performed to obtain wide table feature data, and the feature data of the estimated sample obtained is wide table feature data.

[0161] In some embodiments, the online text classification model can obtain feature data (or wide table feature data) of the estimated sample based on the model scheme deployed online, and splice the feature data and feedback data to generate sample data with features and feedback. The sample data may also include other data, such as timestamp data, etc. In some embodiments, before the online text classification model splices the feature data and feedback data, it splices the feature data and exposure data to obtain feature data with exposure data; and then splices the feature data and feedback data with exposure data to generate sample data with exposure, features and feedback. In some embodiments, the online text classification model returns the sample data with features and feedback to the first database for model self-learning. The online text classification model obtained by model self-learning can be deployed online to ensure that the data and feature engineering scheme used for model self-learning are respectively consistent with the data and feature engineering scheme used by the model online estimation service, so as to achieve consistency between the model self-learning effect and the model estimation effect.

[0162] In some embodiments, the process of model self-learning of the third service program instance of the text classification application is: based on sample data with features and feedback, training is performed through the model algorithm and model hyperparameters in the model scheme to obtain an online text classification model.

[0163] In some embodiments, the text classification application deploys the explored model solution online, including: replacing the deployed model solution with the explored model solution.

[0164] In some embodiments, the text classification application startup module 32 may replace the deployed machine learning model with the online text classification model; or, deploy the online text classification model online and provide batch estimation services for text classification tasks together with the deployed machine learning model.

[0165] In some embodiments, the division of each unit in the text classification device 30 using machine learning is only a logical function division. There may be other division methods in actual implementation. For example, the text classification application creation module 31 and the text classification application startup module 32 can be implemented as one unit; the text classification application creation module 31 or the text classification application startup module 32 can also be divided into multiple sub-units. It is understandable that each unit or sub-unit can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application.

[0166] Figure 4 An exemplary architecture diagram of a text classification application providing online estimation service or batch estimation service provided by an embodiment of the present disclosure. Figure 4 As shown, the text classification application has at least two functions: model solution exploration and model self-learning. In some embodiments, the text classification application can be Figure 1 A text classification application is created by the text classification device 12 using machine learning, and after the text classification application is deployed online, the model solution explored by the text classification application is also deployed online, and the online text classification model obtained by the text classification application through model self-learning is also deployed online.

[0167] Combination Figure 4 ,The process of providing online estimation service or batch estimation service for text classification application is described as follows:

[0168] After defining the text classification scenario, you can connect data with the text classification scenario to implement data management. For example, Figure 2The function of the data management unit 23 shown. When the text classification application is online, the second service program instance of the text classification application can explore the model solution based on the text classification application configuration information and the text annotation data to obtain the model solution, and then the model solution can be deployed online to provide online estimation services or batch estimation services (in fact, the estimation results will not be output, and the default estimation results are output, so they are indicated by dotted lines in the figure), and the model solution will reflow the intermediate data. The third service program instance of the text classification application can perform model self-learning based on the reflowed intermediate data and the model solution to produce an online text classification model, and then the online text classification model can be deployed online to provide online estimation services or batch estimation services.

[0169] Can be, Figure 4 In the process, data management, model self-learning, and online estimation service (or batch estimation service) form a small closed loop; data management, model solution exploration, and online estimation service (or batch estimation service) form a large closed loop. Among them, the small closed loop ensures that the data and feature engineering solutions used for model self-learning are the same as the data and feature engineering solutions used for batch estimation services, respectively, to achieve consistency between model self-learning effects and model estimation effects. The large closed loop ensures that the data used for model solution exploration (referred to as offline data) and the data used for batch estimation services (referred to as online data) are of the same source, achieving the same source of offline and online data.

[0170] Figure 5 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. In some embodiments, Figure 1 The text classification device using machine learning can be arranged in an electronic device or implemented as an electronic device.

[0171] like Figure 5 As shown, the electronic device includes: at least one processor 51, at least one memory 52 and at least one communication interface 53. The various components in the electronic device are coupled together through a bus system 54. The communication interface 53 is used to transmit information between external devices. It can be understood that the bus system 54 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 54 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 5 Various buses are labeled as bus system 54 .

[0172] It can be understood that the memory 52 in this embodiment can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0173] In some embodiments, the memory 52 stores the following elements, executable units or data structures, or a subset or an extended set thereof: an operating system and application programs.

[0174] The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic tasks and process hardware-based tasks. The application includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application tasks. The program that implements the text classification method using machine learning provided in the embodiment of the present disclosure can be included in the application.

[0175] In the embodiment of the present disclosure, the processor 51 calls the program or instructions stored in the memory 52, specifically, the program or instructions stored in the application program, and the processor 51 is used to execute the steps of each embodiment of the text classification method using machine learning provided in the embodiment of the present disclosure.

[0176] The text classification method using machine learning provided in the embodiment of the present disclosure can be applied to the processor 51, or implemented by the processor 51. The processor 51 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 51 or an instruction in the form of software. The above processor 51 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0177] The steps of the text classification method using machine learning provided in the embodiment of the present disclosure can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software units in the decoding processor. The software unit can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 52, and the processor 51 reads the information in the memory 52 and completes the steps of the method in combination with its hardware.

[0178] Figure 6 This is an exemplary flow chart of a text classification method using machine learning provided in an embodiment of the present disclosure. The execution subject of the method is an electronic device. For ease of description, the following embodiments illustrate the process of the text classification method using machine learning using an electronic device as the execution subject.

[0179] like Figure 6 As shown, in step 601, in response to a text classification application creation instruction of a text classification task, text classification application configuration information and text annotation data are obtained; wherein the text annotation data includes a text column and a label column. In some embodiments, each line of text in the text column corresponds to one or more labels.

[0180] In step 602, a text classification application is created based on the text classification application configuration information; wherein the text classification application is a first service program instance, which is at least used to explore a model solution based on the text classification application configuration information and text annotation data to obtain a model solution.

[0181] In step 603, in response to the instruction to start the text classification application, the text classification application is deployed online, and a text classification service address is generated, so that the text classification application provides an online estimation service for the text classification task based on the text classification service address; wherein the online estimation service is performed based on online relevant data of the text classification task. In some embodiments, the text classification application is also used to perform model self-learning based on online relevant data and model solutions to obtain an online text classification model.

[0182] In some embodiments, the method further comprises:

[0183] Returning relevant online data to the first database;

[0184] Returning the labeled intermediate data generated by the model scheme based on the online related data to the first database;

[0185] Providing a reflow data annotation interface to obtain reflow annotation data, and returning the reflow annotation data to the first database again, wherein the reflow annotation data is obtained by annotating the data that flows back to the first database;

[0186] Correspondingly, the text classification application performs model self-learning based on online relevant data, labeled intermediate data, reflux annotation data and model scheme to obtain an online text classification model.

[0187] In some embodiments, the method further includes: deploying an online text classification model online to provide batch estimation services for text classification tasks.

[0188] In some embodiments, the batch estimation service includes: providing a batch estimation service interface, the online text classification model obtains a data set to be batch estimated for the text classification task based on the batch estimation service interface, and outputs a batch estimation result based on the data set to be batch estimated.

[0189] In some embodiments, the data set to be batch estimated includes multiple data columns, and the multiple data columns include a text column;

[0190] The online text classification model outputs batch estimation results based on the data set to be batch estimated, including:

[0191] Batch estimation is performed based on the text column in the data set to be batch estimated to obtain an estimated label column; the estimated label column is concatenated with the data set to be batch estimated to obtain and output the batch estimation result.

[0192] In some embodiments, deploying the online text classification model online includes: replacing the deployed machine learning model with the online text classification model.

[0193] In some embodiments, before responding to a text classification application creation instruction for a text classification task, the method further includes:

[0194] A user interface is provided, and a text classification scenario and a text classification task input by a user are received based on the user interface, and a text classification application creation instruction triggered by a user is received based on the user interface, wherein the text classification application creation instruction corresponds to the text classification scenario and the text classification task input by the user.

[0195] In some embodiments, based on the text classification application configuration information, creating a text classification application includes:

[0196] Packing the text classification application configuration information, the second service program instance, and the third service program instance into a text classification application;

[0197] The second service program instance is used to perform model solution exploration based on the text classification application configuration information and the text annotation data to obtain a model solution;

[0198] Among them, the third service program instance is used to perform model self-learning based on online relevant data and model solutions to obtain an online text classification model.

[0199] In some embodiments, the text classification application configuration information includes one or more of the following:

[0200] How often the model learns itself;

[0201] Evaluation indicators for model self-learning;

[0202] Dataset language;

[0203] Use GPU acceleration;

[0204] The percentage of data used to evaluate the model.

[0205] In some embodiments, a text classification application is used to explore model solutions based on data in a first database, text classification application configuration information, and text annotation data to obtain a model solution; wherein the model solution includes the following solution sub-items: a feature engineering solution, a model algorithm, and model hyperparameters; accordingly, deploying the text classification application online includes: deploying the explored model solution online.

[0206] In some embodiments, the online text classification model is obtained by training an offline model; wherein the offline model is a model generated in the process of model solution exploration, and when the explored model solution is deployed online, the offline model is also deployed online.

[0207] In some embodiments, the online text classification model is a model generated based on the model algorithm and the hyperparameters of the model in the model scheme; and when the explored model scheme is deployed online, the offline model is not deployed online.

[0208] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all described as a series of action combinations, but those skilled in the art can understand that the embodiments of the present disclosure are not limited by the described action sequence, because according to the embodiments of the present disclosure, some steps can be performed in other sequences or simultaneously. In addition, those skilled in the art can understand that the embodiments described in the specification are all optional embodiments.

[0209] The embodiments of the present disclosure also provide a non-transitory computer-readable storage medium, which stores programs or instructions. The programs or instructions enable a computer to execute the steps of various embodiments of text classification using machine learning. To avoid repeated description, they are not repeated here.

[0210] In some embodiments, the text classification application can be regarded as a "learning circle". The "learning circle" is deployed online to provide online estimation services, which can be understood as an online application. The online text classification model obtained by self-learning in the "learning circle" is deployed online to provide batch estimation services, which can be understood as a batch application. Users only need to upload the data set that needs to be estimated and indicate the text column that needs to be estimated. The batch application will automatically write the estimation results into the predict label column behind the data set. Figures 7 to 15 This is a schematic diagram of the interface related to the text classification process of applying machine learning, which will not be described in detail.

[0211] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0212] Those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present disclosure and to form different embodiments.

[0213] Those skilled in the art will appreciate that the description of each embodiment has its own emphasis, and for parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0214] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A text classification method using machine learning, comprising: In response to a text classification application creation instruction of a text classification task, obtaining text classification application configuration information and text annotation data; The text annotation data includes a text column and a label column; Based on the text classification application configuration information, a text classification application is created; wherein the text classification application is a first service program instance, and is at least used to perform model solution exploration based on the text classification application configuration information and the text annotation data to obtain a model solution, wherein the model solution is a collection of multiple strategies for modeling, and the model solution includes how to screen data, how to construct features, how to tune model hyperparameters, how to select a model, and how to train a model; In response to an instruction to start the text classification application, the text classification application is deployed online, and a text classification service address is generated, so that the text classification application provides an online estimation service for the text classification task based on the text classification service address; wherein the online estimation service is performed based on online related data of the text classification task; Wherein, deploying the text classification application online includes: deploying the model solution obtained through exploration online; The text classification application is also used to perform model self-learning based on the online relevant data and the model scheme to obtain an online text classification model.

2. The method according to claim 1, wherein: Each line of text in the text column corresponds to one or more labels.

3. The method according to claim 1, wherein: The method further comprises: Returning the online related data to the first database; Returning the labeled intermediate data generated by the model solution based on the online related data to the first database; Providing a reflow data annotation interface to obtain reflow annotation data, and returning the reflow annotation data to the first database again, wherein the reflow annotation data is obtained by annotating the data that flows back to the first database; Correspondingly, the text classification application performs model self-learning based on the online related data, the labeled intermediate data, the reflow annotation data and the model scheme to obtain an online text classification model.

4. The method according to claim 1, wherein: The method also includes: deploying the online text classification model online to provide batch estimation services for the text classification task.

5. The method according to claim 4, wherein: The batch estimation service includes: providing a batch estimation service interface, the online text classification model obtains the data set to be batch estimated for the text classification task based on the batch estimation service interface, and outputs the batch estimation result based on the data set to be batch estimated.

6. The method according to claim 5, wherein: The data set to be batch estimated includes multiple data columns, and the multiple data columns include a text column; The outputting a batch estimation result based on the data set to be batch estimated includes: Perform batch estimation based on the text column in the data set to be batch estimated to obtain an estimated label column; The estimated label column is concatenated with the data set to be batch estimated to obtain a batch estimation result and output it.

7. The method according to claim 4, wherein: Deploying the online text classification model online includes: replacing the deployed machine learning model with the online text classification model.

8. The method according to claim 1, wherein: Before the text classification application creation instruction in response to the text classification task, the method further includes: A user interface is provided, based on which a text classification scenario and a text classification task input by a user are received, and based on which a text classification application creation instruction triggered by a user is received, wherein the text classification application creation instruction corresponds to the text classification scenario and the text classification task input by the user.

9. The method according to claim 1, wherein: The creating a text classification application based on the text classification application configuration information includes: Packaging the text classification application configuration information, the second service program instance and the third service program instance into a text classification application; The second service program instance is used to perform model solution exploration based on the text classification application configuration information and the text annotation data to obtain a model solution; The third service program instance is used to perform model self-learning based on the online related data and the model scheme to obtain an online text classification model.

10. The method according to claim 1, wherein: The text classification application configuration information includes one or more of the following: How often the model learns itself; Evaluation indicators for model self-learning; Dataset language; Use GPU acceleration; The percentage of data used to evaluate the model.

11. The method according to claim 3, wherein: The text classification application is used to explore model solutions based on the data in the first database, the text classification application configuration information and the text annotation data to obtain a model solution; wherein the model solution includes the following solution sub-items: feature engineering solution, model algorithm and model hyperparameters.

12. The method according to claim 11, wherein: The online text classification model is obtained by training an offline model; wherein the offline model is a model generated in the process of exploring the model solution, and when the explored model solution is deployed online, the offline model is also deployed online.

13. The method according to claim 11, wherein: The online text classification model is a model generated based on the model algorithm and model hyperparameters in the model scheme; and when the model scheme obtained through exploration is deployed online, the offline model is not deployed online.

14. A text classification device using machine learning, comprising: A text classification application creation module, used to obtain text classification application configuration information and text annotation data in response to a text classification application creation instruction of a text classification task; The text annotation data includes a text column and a label column; based on the text classification application configuration information, a text classification application is created; wherein the text classification application is a first service program instance, and is at least used to perform model solution exploration based on the text classification application configuration information and the text annotation data to obtain a model solution, wherein the model solution is a collection of multiple strategies for modeling, and the model solution includes how to screen data, how to construct features, how to tune model hyperparameters, how to select a model, and how to train a model; A text classification application startup module, configured to deploy the text classification application online in response to an instruction to start the text classification application, and generate a text classification service address, so that the text classification application provides an online estimation service for the text classification task based on the text classification service address; wherein the online estimation service is performed based on online related data of the text classification task; The text classification application is also used to perform model self-learning based on the online relevant data and the model scheme to obtain an online text classification model; The text classification application startup module deploys the text classification application online, including: deploying the explored model solution online.

15. The device according to claim 14, wherein: Each line of text in the text column corresponds to one or more labels.

16. The device according to claim 14, wherein: The text classification application is further used to return the online relevant data to the first database, and return the labeled intermediate data generated by the model scheme based on the online relevant data to the first database; The text classification device using machine learning further includes a labeling interface module, which is used to provide a reflow data labeling interface to obtain reflow labeling data, and reflow the reflow labeling data to the first database again, wherein the reflow labeling data is obtained by labeling the data reflowing to the first database; Correspondingly, the text classification application performs model self-learning based on the online related data, the labeled intermediate data, the reflow annotation data and the model scheme to obtain an online text classification model.

17. The device according to claim 14, wherein: The text classification application startup module is also used to deploy the online text classification model online to provide batch estimation services for the text classification task.

18. The device according to claim 17, wherein: The text classification application startup module is also used to provide a batch estimation service interface. The online text classification model obtains the data set to be batch estimated for the text classification task based on the batch estimation service interface, and outputs the batch estimation result based on the data set to be batch estimated.

19. The device according to claim 18, wherein: The data set to be batch estimated includes multiple data columns, and the multiple data columns include a text column; The online text classification model outputs batch estimation results based on the data set to be batch estimated, including: performing batch estimation based on the text column in the data set to be batch estimated to obtain an estimated label column, splicing the estimated label column with the data set to be batch estimated, obtaining batch estimation results and outputting them.

20. The device according to claim 17, wherein: The text classification application startup module deploys the online text classification model online, including: replacing the deployed machine learning model with the online text classification model.

21. The device according to claim 14, wherein: The text classification application creation module is also used for: Before responding to a text classification application creation instruction for a text classification task, a user interface is provided, based on which a text classification scenario and a text classification task input by a user are received, and based on the user interface, a text classification application creation instruction triggered by a user is received, wherein the text classification application creation instruction corresponds to the text classification scenario and the text classification task input by the user.

22. The device according to claim 14, wherein: The text classification application creation module creates a text classification application based on the text classification application configuration information, including: Packaging the text classification application configuration information, the second service program instance and the third service program instance into a text classification application; The second service program instance is used to perform model solution exploration based on the text classification application configuration information and the text annotation data to obtain a model solution; The third service program instance is used to perform model self-learning based on the online related data and the model scheme to obtain an online text classification model.

23. The device according to any one of claims 14, wherein: The text classification application configuration information includes one or more of the following: How often the model learns itself; Evaluation indicators for model self-learning; Dataset language; Use GPU acceleration; The percentage of data used to evaluate the model.

24. The device according to any one of claims 16, wherein: The text classification application is used to explore model solutions based on the data in the first database, the text classification application configuration information and the text annotation data to obtain a model solution; wherein the model solution includes the following solution sub-items: feature engineering solution, model algorithm and model hyperparameters.

25. The device according to claim 24, wherein: The online text classification model is obtained by training an offline model; wherein the offline model is a model generated in the process of exploring the model solution, and when the explored model solution is deployed online, the offline model is also deployed online.

26. The device according to claim 24, wherein: The online text classification model is a model generated based on the model algorithm and model hyperparameters in the model scheme; and when the model scheme obtained through exploration is deployed online, the offline model is not deployed online.

27. An electronic device comprising: Processor and memory; The processor is used to execute the steps of the method according to any one of claims 1 to 13 by calling the program or instruction stored in the memory.

28. A non-transitory computer-readable storage medium storing a program or instructions, wherein the program or instructions enable a computer to execute the steps of the method according to any one of claims 1 to 13.

Images