Information classification model automatic training method, system and device, and storage medium

Abstract

The application discloses an information classification model automatic training method, system, device and storage medium. The training method comprises the following steps: acquiring a first information set; the first information set comprises daily newly added information; determining a second information set according to the first information set; the second information set comprises historical newly added information; determining a first classification label set according to the first information set; the first classification label set comprises new classification labels whose label content quantity is greater than a first preset threshold; reconstructing a model training sample according to the first information set and the second information set to obtain a first sample data set; inputting the first classification label set and the first sample data set into an original model to perform training and obtaining a target classification model. The training method can improve the prediction accuracy of the information classification model. The application can be widely applied in the technical field of information classification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information classification, and in particular to an information classification model automatic training method, system, device and storage medium. BACKGROUND

[0002] One of the core tasks of the current artificial intelligence is the classification model (Classification Model) used to make decisions or assign items to categories. The current industry training and updating of the classification model mainly focuses on fixed classification number and model retraining. The existing classification model mainly has the following problems: Problem one: the problem of fixed classification number model is that it cannot adapt to the scene where new concepts frequently appear. For example, the classification label of information content is a multi-classification model, and the classification number will increase with the emergence of new events. The classification algorithm for understanding the content of information will often fail to capture new labels (such as new film and television variety shows or sudden popular diseases). Problem two: while failing to capture new classifications, the model also fails to capture the distribution changes of different categories over time. In the information field, there will be holidays, seasonal events or occasional emergencies. The model will not be able to capture the distribution changes of related classifications if it is not updated in time, resulting in inaccurate model predictions. The common solution to the above problems is to adjust the model structure and retrain, but the training period is long and resource consumption is large.

[0003] In summary, the problems in the related art need to be solved urgently. SUMMARY

[0004] The present application aims to at least partially solve one of the technical problems in the related art.

[0005] To this end, an object of an embodiment of the present application is to provide an information classification model automatic training method that can improve the prediction accuracy of the information classification model.

[0006] In order to achieve the above technical purpose, the technical solution adopted by the embodiments of the present application comprises: obtaining a first information set; the first information set comprises newly added information on the day; determining a second information set according to the first information set; the second information set comprises historical newly added information; determining a first classification label set according to the first information set; the first classification label set comprises new classification labels whose label content quantity is sorted over a first preset threshold; reconstructing the model training sample according to the first information set and the second information set to obtain a first sample data set; inputting the first classification label set and the first sample data set into an original model for training to obtain a target classification model. The training method can improve the prediction accuracy of the information classification model.

[0007] In addition, the information classification model automatic training method according to the above-mentioned embodiment of the present application can further have the following additional technical features.

[0008] Further, in an embodiment of the present application, the determining the first classification label set according to the first information set comprises: performing label classification on all sub-information of the first information set to determine a second classification label of the sub-information; the second classification label comprises a newly added classification label; extracting information content click volume and information content publishing volume of the sub-information corresponding to each second classification label; performing label content volume sorting on the second classification label according to the information content click volume and the information content publishing volume; and determining the second classification label with a label content volume sorting greater than a first preset threshold as the first classification label set.

[0009] Further, in an embodiment of the present application, the determining the second information set according to the first information set comprises: extracting information publishing dates and information quantities of the first information set; and determining the second information set according to the information publishing dates and the information quantities.

[0010] determining a plurality of second information subsets according to the information publishing dates and the information quantities;

[0011] the second information subset comprises historical newly added information of a single day before the information publishing date;

[0012] wherein the information quantity of the second information subset satisfies the formula:

[0013] n=N*a*dt / e

[0014] wherein n≥1, n is the quantity of the single day historical newly added information; a is a decay coefficient, 0

[0015] performing AND operation on the second information subsets to determine the second information set.

[0016] Further, in an embodiment of the present application, the sample reconstruction of the model training sample according to the first information set and the second information set to obtain the first sample data set comprises: performing data fusion on the first information set and the second information set to obtain first sample data; and obtaining the first sample data set according to the first sample data; wherein the information quantity of the first information set accounts for a fixed value in the first sample data set.

[0017] Further, in an embodiment of the present application, the obtaining the first information set comprises: obtaining all newly added information on the day; and extracting a plurality of newly added information of the all newly added information on the day as the first information set, the plurality of newly added information corresponding to at least one new classification label.

[0018] Further, in an embodiment of the present application, the sorting the second classification label according to the information content click volume and the information content publishing volume comprises: determining a first weight of the information content click volume and a second weight of the information content publishing volume; determining a first heat parameter according to the first weight and the information content click volume; determining a second heat parameter according to the second weight and the information content publishing volume; calculating a heat total sum of the first heat parameter and the second heat parameter, and sorting the second classification label according to the heat total sum.

[0019] In another aspect, an embodiment of the present application provides an information classification model automatic training device, comprising:

[0020] An obtaining module is configured to obtain a first information set.

[0021] A first processing module is configured to determine a second information set according to the first information set.

[0022] A second processing module is configured to determine a first classification label set according to the first information set, the first classification label set comprising a new classification label set whose label content volume exceeds a first preset threshold.

[0023] A third processing module is configured to perform sample reconstruction on a model training sample according to the first information set and the second information set, to obtain a first sample data set.

[0024] A fourth processing module is configured to input the first classification label set and the first sample data set into an original model for training, to obtain a target classification model.

[0025] In another aspect, an embodiment of the present application provides a computer device, comprising:

[0026] At least one processor;

[0027] At least one memory configured to store at least one program;

[0028] When the at least one program is executed by the at least one processor, the at least one processor is caused to implement the above-mentioned information classification model automatic training method.

[0029] In another aspect, the embodiments of the present application also provide a computer readable storage medium, which stores a processor executable program. The processor executable program is used to implement the information classification model automatic training method when executed by a processor.

[0030] The advantages and beneficial effects of the present application will be partially given in the following description, partially will become obvious from the following description, or will be learned by the practice of the present application:

[0031] The information classification model automatic training method disclosed by the embodiments of the present application can determine a new classification label set whose label content quantity exceeds a certain value according to the new information added on the day, train the information classification model by taking the new classification label set whose label content quantity exceeds a certain value, the new information added on the day and the historical information set as training samples. The training method can reduce the data samples for training, and the trained information classification model not only integrates the historical information, but also integrates the new information added on the day and the new classification label set whose label content quantity exceeds a certain value on the day, so that the information classification model can update the relevant classification labels and the distribution change of information in time, and the prediction accuracy of the information classification model can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following introduces the drawings of the related technical solutions in the embodiments of the present application or the prior art. It should be understood that the drawings in the following introduction are only for the convenience of clearly describing part of the embodiments of the technical solutions of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0033] Figure 1 The step schematic diagram of the information classification model automatic training method provided in the embodiments of the present application;

[0034] Figure 2 The step schematic diagram of determining the first classification label set according to the first information set in the information classification model automatic training method provided in the embodiments of the present application;

[0035] Figure 3 The step schematic diagram of determining the second information set according to the first information set in the information classification model automatic training method provided in the embodiments of the present application;

[0036] Figure 4 The step schematic diagram of determining the second information set according to the information publishing date and the information quantity in the information classification model automatic training method provided in the embodiments of the present application;

[0037] Figure 5A step diagram of sample reconstruction of model training samples according to the first information set and the second information set in the information classification model automatic training method provided in the embodiments of the present application, to obtain a first sample data set;

[0038] Figure 6 A step diagram of obtaining a first information set in the information classification model automatic training method provided in the embodiments of the present application;

[0039] Figure 7 A step diagram of label content sorting of the second classification label according to the information content click volume and the information content publishing volume in the information classification model automatic training method provided in the embodiments of the present application;

[0040] Figure 8 A structure diagram of an information classification model automatic training device provided in the embodiments of the present application;

[0041] Figure 9 A computer device structure diagram of an information classification model automatic training system provided in the embodiments of the present application. DETAILED DESCRIPTION

[0042] The present application will be further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0043] In the following description, “some embodiments” are described, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0045] Before the embodiments of the present application are further described in detail, the terms and phrases involved in the embodiments of the present application are explained, and the terms and phrases involved in the embodiments of the present application are applicable to the following explanations.

[0046] 1) Artificial Intelligence (AI) is the theory, method, technology and application system of using digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is the design principle and implementation method of various intelligent machines, so that the machine has the functions of perception, reasoning and decision-making. Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc. several directions.

[0047] 2) Nature Language processing (NLP) is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science and mathematics. This field involves natural languages used in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question and answer, knowledge graph, etc.

[0048] 3) Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is a subject that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning (deep learning) usually includes artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rule-based learning.

[0049] In related technologies, one of the current artificial intelligence core tasks is the classification model (Classification Model) used to make decisions or assign items to categories. At present, the main way to train and update the classification model in the industry is to fix the number of categories and retrain the model.

[0050] Problem one: the problem brought by the fixed number of information classification model is that it cannot adapt to the scene where new concepts appear frequently. For example, the classification label of information content is a multi-classification model, and the number of classification labels will increase with the emergence of new events. The classification algorithm for understanding the content of information will often fail to capture new labels, such as new film and television variety shows or sudden popular diseases.

[0051] Problem two: while failing to capture new label classification, the information classification model also cannot timely capture the distribution changes of information in different classification categories over time. In the information category, there will be holidays, seasonal events or occasional emergencies. If the model is not updated in time, it will not be able to capture the distribution changes of related information classification, resulting in inaccurate model prediction.

[0052] The above two problems currently have no mature solution in the industry. The common solution is to adjust the information model structure and retrain, but this training method has a long training cycle and consumes a lot of resources.

[0053] In order to solve the problems that the information classification model cannot adapt to the scene where new information concepts appear frequently, or the information model cannot timely capture the distribution changes of different categories over time, resulting in inaccurate model prediction in the related technology, the present application provides an information classification model automatic training method.

[0054] Reference Figure 1 , Figure 1 The information classification model automatic training method provided by the embodiments of the present application can determine a new classification label set whose label content amount exceeds a certain value according to the newly added information of the day, use the new classification label set whose label content amount exceeds a certain value, the newly added information of the day and the historical information set as training samples to train the information classification model. The training method can reduce the data samples for training, and the trained information classification model not only integrates historical information, but also integrates new information added on the day and the new classification label set whose label content amount exceeds a certain value on the day. The information classification model can update the related classification labels and the distribution changes of information in time, and can improve the prediction accuracy of the information classification model. The method can include:

[0055] S1, obtaining a first information set; the first information set includes newly added information of the day;

[0056] In this step, the first information set can be a part of the newly added information of all the day, or the total set of all the newly added information of the day; and the newly added information of the day can correspond to the original old classification label in the model and correspond to the newly added classification label; or only correspond to the new classification label. In some embodiments of the present application, the first information set can be obtained by the host computer or other devices as the execution subject, and the execution subject of the information classification model automatic training method can obtain the above-mentioned newly added information of the day through wired connection or wireless connection. It should be pointed out that the above-mentioned limited connection mode can include the connection between the mobile device and the host computer, and can also include the connection between the host computers and the wired connection between other now known or future developed devices and the host computer; and the above-mentioned wireless connection mode can include but not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (Ultra WideBand) connection and other now known or future developed wireless connection mode.

[0057] S2, determining a second information set according to the first information set; the second information set includes historical newly added information;

[0058] In this step, the first information set and the second information set can have a certain information quantity relationship. For example, the first information set is composed of a certain proportion of daily newly added information, and the second information set is composed of a plurality of information subsets, which can contain a certain proportion of historical daily newly added information of the first information set before the release date, or the same number of historical daily newly added information of the first information set before the release date. For example, the number of daily information on July 15 is 2000, at this time, the number of newly added information is small, so 2000 pieces of information can be determined as the first information set. Correspondingly, the information subset of July 14 can be based on the number of 2000 newly added information on July 15, and the same number of samples can be selected, such as 2000 pieces of information on July 14, and the time of the previous days of the newly added information on the day is selected. When the number of sample selection is the same as the number of the day, the time of the previous days of the day can be set in advance, such as the previous 10 days can be set in advance, so when the number of the day is N, 10N historical newly added information is selected from each day in the previous 10 days to form the second information set. The sample can also be attenuated. When the sample is attenuated, a part of the newly added information on July 14 can be selected as the information subset of July 14, such as 1700 pieces. Correspondingly, the information subset of July 13 can also be based on the number of 2000 newly added information on the day, and a part of the newly added information on July 13 can be selected as the information subset of July 13, such as 1350 pieces. It should be noted that the number of information on July 13 selected in combination with the heat characteristics of the information should be less than or equal to the number of information on July 14. In addition, it is conceivable that the number of information in the information subset should be at least 1, and finally the 1700 pieces of information subset and the information subset of the previous date on a single day are collected to form the second information set. The second information set can obtain a second information set containing a large number of historical newly added information.

[0059] S3, determining a first classification label set according to the first information set; the first classification label set includes new classification labels whose label content quantity is greater than a first preset threshold;

[0060] In this step, the first classification label can be a new classification label whose label content quantity is greater than a first preset threshold, and the first information set can be a set of newly added information on the day. According to the first information set, the classification labels corresponding to different newly added information can be determined. The information labels can be determined by manually labeling the newly added information, or by machine labeling the newly added information. The newly added information on the day can have a new classification label or an old classification label corresponding thereto. For the new classification label, different labels can have different label content quantities. For a new classification label with a small label content quantity, the classification model does not need to be predicted. Therefore, the new model does not need to add a new classification label with a small label content quantity. For a new classification label with a large label content quantity, the label content quantity needs to be sorted, and all new classification labels with a label content quantity greater than the first preset threshold are determined as the first classification label set. It should be noted that the first preset threshold in this step can be set according to the length of the output vector of the training model. For example, if the length of the output vector is M, the first preset threshold can be set to M / 2.

[0061] S4, sample reconstruction is performed on the model training sample according to the first information set and the second information set, to obtain a first sample data set;

[0062] In this step, the first information set can include newly added information on the day, and the second information set can include newly added information extracted from each day in the past several days corresponding to the newly added information on the day. In some embodiments of the present application, the model training sample can be reconstructed by the sum of the newly added information on the day and the newly added information extracted from each day in the past several days. The sample reconstruction can make the first sample data set include a certain amount of newly added information and a certain amount of historical information. The model training sample after sample reconstruction can make the training model periodically add new information and perform classification prediction, which can improve the prediction accuracy of the information classification model.

[0063] S5, the first classification label set and the first sample data set are input into an original model for training, to obtain a target classification model;

[0064] In this step, the original model can be a classification model trained using historical information and old labels. In some embodiments of the present application, the original model can use one of commonly used classification models such as a logistic regression model, a decision tree model, a support vector machine model, a random forest model, an XGBoost model, a LightGBM model, or a naive Bayes classifier. The original model can also be a combination of the above-mentioned classification models, or a classification model in which a multilayer perceptron module (MLP) is embedded at the end of each layer of the original model, and finally outputs a classification result using a sigmoid / softmax layer. The type and structure of the original model are not limited herein. Since information is added every day, the original model is also a dynamic classification model that is updated every day based on newly added information and newly added classification labels. The classification labels and newly added information input every day can enable the original model to be trained using samples including newly added information published on the same day, thereby making the original model have good robustness. For training of the target classification model, the newly added first classification label set and the first sample data set can be input into the original model for training. The trained target classification model can predict the classification labels of newly published information based on newly added information every day. Since the target classification model is trained based on the latest information and classification labels of the information, the target classification model can maintain the robustness of the original model and accurately predict the classification of newly published information.

[0065] Further, with reference to Figure 2 , in Figure 2 , the determining of the first classification label set according to the first information set can include:

[0066] S101, extracting information content click volume and information content publishing volume of sub-information corresponding to each second classification label;

[0067] S102, performing label content volume sorting on the second classification label according to the information content click volume and the information content publishing volume;

[0068] S103, determining a second classification label with a label content volume sorting greater than a first preset threshold as a first classification label set.

[0069] In some embodiments of the present application, the second classification label can be a newly added classification label; since the newly added information and the newly added label are not in a one-to-one relationship; one newly added classification label can correspond to multiple pieces of newly added information, or can correspond to one piece of newly added information; in order to reduce the training samples of the classification model, not all newly added classification labels will be used for training of the classification model, and a batch of classification labels with certain information popularity need to be selected as the first classification label set for model training, and the information popularity can include information content click volume and information content publishing volume, the information content click volume can reflect the recent degree of pursuit of the information corresponding to the label on the network, and the information content publishing volume can reflect the recent degree of discussion of the information content on the network; in some embodiments of the present application, the information content click volume and the information content publishing volume of the sub-information corresponding to each second classification label can be extracted, and the newly added classification labels with information popularity ranking before the first preset threshold can be determined as the first classification label according to the information content click volume and the information content publishing volume. The first preset threshold can be set according to the output vector length of the original training model, for example, if the output vector length is 18, the first preset threshold can be set to half of the output vector length, for example, 9, that is, the labels with label content volume in the top 9 among all newly added classification labels can be used as the first classification label, or the first preset threshold can be set to one third of the output vector length, for example, 6, that is, the labels with label content volume in the top 6 among all newly added classification labels can be used as the first classification label.

[0070] Further, with reference to Figure 3 , in Figure 3 , the determining the second information set according to the first information set can include

[0071] S201, extracting information publishing dates and information quantities of the first information set;

[0072] S202, determining a second information set according to the information publishing dates and the information quantities;

[0073] In some embodiments of this application, extracting the information release date can determine the date range for selecting historical new information. For example, if the information release date is July 18th, the historical new information can be new information from several days prior to July 18th. The specific number of days can be determined by the number of new information extracted on that day and a decay formula, or it can be determined by a preset date range. For example, if the preset date range is set to 15 days, the historical new information from the past 15 days can be fixed at a certain value. This value can be the same as the number of new information on that day, or it can be set according to a certain decay rule. Specifically, this application can select new information from that day... The quantity of information and the decay formula determine the number of historically newly added information items for each day in the second information set. After determining the decay formula, when there are many newly added information items on a given day, the date range for decay is also wider. When the quantity of newly added information items on a given day is determined, if the decay coefficient of the decay formula is relatively small, the quantity decays more slowly, and the date range for decay is also wider. Conversely, when the decay coefficient of the decay formula is relatively large, the date range for decay is narrower. After determining the quantity of historically newly added information items for each day prior to the current information release date, a certain number of information items can be extracted from the historically newly added information items as the second sub-information set. Finally, by performing a union operation on the second sub-information sets, the second information set can be obtained.

[0074] Furthermore, referring to Figure 4 ,exist Figure 4 In this context, determining the second information set based on the information release date and the number of information items may include:

[0075] S301. Based on the information release date and the number of information items, determine several second information subsets;

[0076] The second subset of information includes historical new information for a single day prior to the information publication date;

[0077] The number of information items in the second information subset satisfies the formula:

[0078] n=N*

[0079] Where n≥1, n is the number of newly added information in a single day; a is the decay coefficient, 0<a<1; N is the number of information in the first information set; dt is the difference between the information release date and the historical information release date; e is the natural base;

[0080] Perform a union operation on the second subset of information to determine the second subset of information;

[0081] S302. Perform a union operation on the second information subset to determine the second information set.

[0082] In some embodiments of the present application, in order to maintain the stability of the model, the second information set and the first information set are required as training samples, and the second information set can be composed of a plurality of second information subsets; here, the plurality can be calculated by the formula, and the number of information subsets can be calculated by the formula; a fixed number of second information subsets can also be directly set. The second information subset can be an information subset composed of a certain number of information sets extracted from the historical new information every day. In some embodiments of the application, the number of information in the second information subset can be the same as the number of information in the first information set. In other embodiments of the application, the number of information in the second information subset can become smaller and smaller as the date difference between the new information and the current day becomes larger and larger. In order to maximize the weight of the new information on the current day in the information sample, the number of information in the second information subset will become smaller and smaller as the date difference between the new information on the current day and the current day becomes larger and larger. For this case, in some embodiments of the present application, the number of information in the second information subset can satisfy the information calculation formula

[0083] n = N * a dt

[0084] Wherein n≥1, n is the number of information in the second information subset, it can be understood that n needs to be greater than or equal to 1; a is the attenuation coefficient, 0

[0085] Further, with reference to Figure 5 , the method can further include: Figure 5 reconstructing the model training samples according to the first information set and the second information set to obtain a first sample data set,

[0086] S401. The first information set and the second information set are fused to obtain the first sample data;

[0087] S402. Based on the first sample data, a first sample dataset is obtained; wherein, the proportion of the number of information items in the first information set in the first sample dataset is a fixed value.

[0088] In some embodiments of this application, the first sample dataset, used as training samples for the new classification model, needs to contain a certain amount of historical new information and a certain amount of current-day new information. After obtaining the first and second information sets, the two sets need to be merged. For example, the new information in the first information set can be merged with the second information subset corresponding to the date furthest from the date of publication of the new information in the second information set to obtain the first sample data. Alternatively, a certain proportion of information from each set can be used as set elements to form the first sample data, and all the first sample data can be combined into a set. It should be noted that the fusion method is not limited, but the first sample dataset needs to contain a certain amount of current-day new information and historical new information, and the number of current-day new information and historical new information needs to be maintained at a fixed ratio that does not change over time. However, this can change. For example, if the training samples for newly added news on July 15th are 2000, and the training samples for historically added news are 12000 (meaning their proportion in the first sample set is 1 / 6), then when training the model on July 16th, the training samples for newly added news on that day should also be 2000, and the training samples for historically added news should also be 12000. It's important to note that when the number of training samples for newly added news changes, the number of training samples for historically added news will also change. The first sample data can be a dataset including labels and corresponding news items, while the first sample dataset is the set of all data input to the training model; the relationship between the two can be one of inclusion. By obtaining the first sample dataset, the number of news items in the first news set can be kept at a fixed proportion in the first sample dataset. This fixed proportion of the first sample dataset can improve the overall distribution stability of the training model while also giving greater weight to the distribution of newly added news items on that day, allowing the news classification model to quickly learn new classification labels and hotspot distributions.

[0089] Furthermore, referring to Figure 6 ,exist Figure 6 In this context, obtaining the first information set may include:

[0090] S501: Get all new information added on the current day;

[0091] S502, extract a plurality of new information of all new information of the day as a first information set, the plurality of new information corresponding to at least one new classification label;

[0092] In some embodiments of the present application, when the new information of the day is less, all new information of the day can be used as the first information set. However, there can be tens of thousands of new information in many cases. In order to improve the training efficiency of the multi-classification model, a certain proportion of new information can be extracted from the tens of thousands of new information of the day as the first information set. The certain proportion can be 20%, or other values. The specific proportion is not limited. The certain proportion of new information extracted can include new information corresponding to at least one new classification label. Specifically, when there are 10,000 new information, 2,000 new information can be extracted. Among the 2,000 new information, at least one new classification label corresponding information should be maintained. It should be noted that when there is more than one new classification label, the extracted new information should include all new classification labels.

[0093] Further, with reference to Figure 7 , the second classification label can be sorted according to the label content quantity of the information content click quantity and the information content publishing quantity, which can include:

[0094] S601, determine a first weight of information content click quantity and a second weight of information content publishing quantity;

[0095] S602, determine a first heat parameter according to the first weight and the information content click quantity;

[0096] S603, determine a second heat parameter according to the second weight and the information content publishing quantity;

[0097] S604, calculate the heat sum of the first heat parameter and the second heat parameter, and sort the second classification label according to the heat sum.

[0098] In some embodiments of the present application, the weight of the information content click volume can be preset as a first weight, and the weight of the information content publishing volume can be preset as a second weight. The specific values of the weights can be adjusted according to holidays and weekdays, or can be adjusted according to time periods. The first weight and the second weight can be set to be the same or different, such as both being set to 0.5, or being set to 0.7 and 0.3. According to the first weight and the information content click volume, a first heat parameter after combining the weights can be obtained, and according to the second weight and the information content publishing volume, a second heat parameter after combining the weights can be obtained. By adding the second heat parameter and the second heat parameter, a total heat can be obtained. According to the total heat, the second classification label is sorted, and finally the label with a certain ranking of heat can be determined.

[0099] In summary, the information classification model automatic training method of the present application has the following advantages:

[0100] 1. Without retraining the classification model, the original model parameters can be used for further model training, which can greatly accelerate the convergence speed of model training.

[0101] 2. The training samples required by the classification model are less than those of the traditional training method. Since the original model has been trained with a large number of training samples, the goal of new training samples is to learn new classification, and learning new classification only requires 1 / 10 of the initial training samples, and 2 rounds (epochs) of training can be performed each time. The iteration time of the training model can be reduced from days to hours.

[0102] 3. The classification is scalable, and the new information classification is trained and predicted in an appended manner without retraining the model. The server only needs to update the classification mapping dictionary, and the classification order is forward compatible in the backend output. The present classification model can timely synchronize the distribution of large-capacity information content. Since the training time of the classification model is reduced to hours, the training period can be increased to daily training, which can timely synchronize new hotspots and periodic events of information. After the event is over, it can also quickly return to normal distribution.

[0103] 4、The training method can determine a new classification label set with a label content amount exceeding a certain value according to the daily new information, train the information classification model by taking the new classification label set with the label content amount exceeding the certain value, the daily new information, and the historical information set as training samples. The training method adds a large amount of historical information, integrates a certain amount of daily new information, and filters some new classification label sets with a label content amount exceeding a certain value. The information classification model can update the relevant classification labels and the distribution changes of the information in a timely manner, maintain the robustness of the classification model for classification prediction, and accurately predict the classification of the new information, thereby further improving the prediction accuracy of the classification model.

[0104] Reference Figure 8 In some embodiments of the present application, the information classification model automatic training device can comprise: Figure 8 The acquisition module 1001 is configured to acquire a first information set. The first information set comprises daily new information.

[0105] The first processing module 1002 is configured to determine a second information set according to the first information set. The second information set comprises historical new information.

[0106] The second processing module 1003 is configured to determine a first classification label set according to the first information set. The first classification label set comprises a new classification label set with a label content amount exceeding a first preset threshold.

[0107] The third processing module 1004 is configured to reconstruct a model training sample according to the first information set and the second information set, and obtain a first sample data set.

[0108] The fourth processing module 1005 is configured to input the first classification label set and the first sample data set into an original model for training, and obtain a target classification model.

[0109] In some embodiments of the present application, the acquisition module 1001 can be in the same host computer as the acquisition module 1001. The content of the network information is acquired through the module inside the host computer. The acquisition module 1001 can also be any module connected to the host computer. The data is transmitted to the host computer through wired or wireless connection with the host computer. The data is processed through the module inside the host computer. It should be noted that the first processing module 1002, the second processing module 1002, and other processing modules are the same. The specific device connection mode and device setting are not limited.

[0110] It can be understood that

[0111] Figure 1 ​The content of the automatic training method embodiment for information classification models shown is applicable to the automatic training device embodiment for information classification models. The specific functions implemented in this automatic training device embodiment for information classification models are the same as those in the automatic training device embodiment for information classification models. Figure 1 The automatic training method for the information classification model shown is the same as the embodiment, and the beneficial effects achieved are the same. Figure 1 The beneficial effects achieved by the automatic training embodiment of the information classification model shown are also the same.

[0112] Reference Figure 9 This application also discloses a computer device, including:

[0113] At least one processor 1011;

[0114] At least one memory 1012 is used to store at least one program;

[0115] When at least one program is executed by at least one processor 1011, causing at least one processor 1012 to perform as follows: Figure 1 The example shown is an embodiment of an automatic training method for information classification models.

[0116] It is understandable that, such as Figure 1 The embodiments of the automatic training method for information classification models shown are all applicable to the embodiments of this computer device. The specific functions implemented by the embodiments of this computer device are the same as those shown below. Figure 1 The embodiment of the automatic training method for the information classification model shown is the same, and the beneficial effects achieved are the same as those described above. Figure 1 The beneficial effects achieved by the embodiment of the automatic training method for the information classification model shown are also the same.

[0117] This application also discloses a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to implement, for example... Figure 1 The example shown is an embodiment of an automatic training method for information classification models.

[0118] It is understandable that, such as Figure 1 The content of the embodiments of the automatic training method for information classification models shown are all applicable to the embodiments of this computer-readable storage medium. The specific functions implemented by the embodiments of this computer-readable storage medium are the same as those shown below. Figure 1 The automatic training method for the information classification model shown is the same as the embodiment described above, and the beneficial effects achieved are the same as those described above. Figure 1 The beneficial effects achieved by the automatic training method for the information classification model shown in the embodiment are also the same.

[0119] In alternative embodiments, the functions / operations in the flow diagrams can occur in different orders and / or concurrently with each other. For example, two operations shown in succession can in fact be executed substantially concurrently or the operations sometimes can be executed in reverse order, depending upon the functionality / operations involved. Also, the embodiments presented and described in the flowcharts are only examples. A person skilled in the art can easily devise other methods that are equivalent in substance to the methods illustrated by the following figures. Alternative embodiments are within the scope of the following claims in which:

[0120] Moreover, while the present application has been described in the context of functional modules, it is to be understood that one or more of the functions and / or features can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It will also be appreciated that detailed discussion of the actual implementation of each module is not necessary for an understanding of the present application. Rather, the actual implementation is within the routine skill of engineers familiar with the property, function and internal relationships of the various functional modules disclosed herein. Accordingly, the present application is not limited to purely hardware implementations, but also encompasses software implementations, including firmware, resident software, micro-code, etc. It is also to be understood that the disclosed specific concepts are merely illustrative, and not intended to limit the scope of the present application, which is defined by the appended claims and their equivalents.

[0121] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0122] The logic and / or steps represented in the flow diagrams or otherwise described herein, for example, can be embodied in non-transitory computer-readable media in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or in conjunction with which the instructions can be executed. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber (optical), and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via an optical scanner, then compiled, interpreted, or otherwise processed, and stored in a computer memory in a form that can be later executed by a computer. In this context, a "computer-readable medium" can be any means that can store the program for use by or in connection with the instruction execution system, apparatus, or device.

[0123] The foregoing description of various embodiments of the application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise form disclosed, and various modifications and variations are possible in light of the above teachings or can be acquired from practice of the application. For example, while a particular feature of the application can have been described with respect to only one or more embodiments thereof, the feature is not necessarily limited to that one or more embodiments. Rather, candidates for the feature can be found in the other embodiments as well, and the scope of the application is not intended to be limited to the particular embodiment in which the feature was first described.

[0124] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, various steps or methods can be implemented, for example, as software or firmware in a memory and executed by suitable instruction-execution systems. If implemented in hardware, for example, in another embodiment, any of the following technologies, or combinations thereof, can be used: discrete logic circuitry having logic gates for implementing logic functions upon an application of data signals, application-specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field-programmable gate arrays (FPGA), and so forth.

[0125] In the above description of the present application, the description using the terms "one embodiment / one example", "another embodiment / another example" or "some embodiments / some examples" and so forth is intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. The appearances of the above-described terms in various places in the specification are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0126] While the embodiments of the present application have been shown and described, it is to be understood that the embodiments can be changed, modified and further applied by those skilled in the art unless otherwise specified, and the scope of the present application should be defined by the claims and their equivalents.

[0127] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present application, and these equivalent modifications or substitutions are included in the scope of the present application defined by the claims

[0128] In the description of the present specification, the description of the terms "one embodiment", "another embodiment" or "certain embodiments" or the like means that the specific features, structures, materials or characteristics described in connection with the embodiments or examples are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0129] While the embodiments of the present application have been shown and described, it is to be understood that the embodiments can be changed, modified and further applied by those skilled in the art unless otherwise specified, and the scope of the present application should be defined by the claims and their equivalents.

Claims

1. An automatic training method for an information classification model, characterized in that, include: Get the first information set; The first information set includes newly added information for the day; Based on the first information set, determine the second information set; The second information set includes historically added information; Based on the first information set, a first category tag set is determined; the first category tag set includes new category tags whose tag content volume ranking exceeds a first preset threshold. Based on the first information set and the second information set, the model training samples are reconstructed to obtain the first sample dataset; The first classification label set and the first sample dataset are input into the original model for training to obtain the target classification model; The step of determining the first category tag set based on the first information set includes: All sub-information in the first information set are tagged and classified to determine the second category tags for the sub-information; the second category tags include newly added category tags. The first category tag set is determined based on the second category tags whose content size is greater than a first preset threshold; wherein, the content size sorting is obtained by sorting the second category tags. The step of determining the second information set based on the first information set includes: Extract the publication date and number of information items from the first information set; The second information set is determined based on the information release date and the number of information items. The step of determining the second information set based on the information release date and the number of information items includes: Based on the information release date and the number of information items, several second information subsets are determined; the second information subsets include historical newly added information from single days prior to the information release date; Perform a union operation on the second information subset to determine the second information set.

2. The automatic training method for an information classification model according to claim 1, characterized in that, The step of determining the first category tag set based on the first information set further includes: Extract the click count and publication volume of the information content corresponding to each of the second category tags; Based on the number of clicks and publications of the information content, the second category tags are sorted by the amount of tag content.

3. The automatic training method for an information classification model according to claim 1, characterized in that, The number of information items in the second information subset satisfies the formula: n=N* Where n≥1, n is the number of newly added historical information in a single day; a is the decay coefficient, 0<a<1; N is the number of information in the first information set; dt is the difference between the information release date and the historical information release date; e is the natural base.

4. The automatic training method for an information classification model according to claim 1, characterized in that, The step of reconstructing the model training samples based on the first information set and the second information set to obtain the first sample dataset includes: The first information set and the second information set are fused together to obtain the first sample data; Based on the first sample data, a first sample dataset is obtained; wherein, the proportion of information in the first information set in the first sample dataset is a fixed value.

5. The automatic training method for an information classification model according to claim 1, characterized in that, The acquisition of the first information set includes: Get all new information for the day; Extract several new pieces of information from all new information added on the same day as a first information set, and the several new pieces of information correspond to at least one new category tag.

6. The automatic training method for an information classification model according to claim 2, characterized in that, The step of sorting the second category tags by content volume based on the number of clicks and the number of publications of the information content includes: Determine the primary weighting of news content click-through rate and the secondary weighting of news content publication volume; The first popularity parameter is determined based on the first weight and the number of clicks on the information content; The second popularity parameter is determined based on the second weight and the amount of information published. Calculate the sum of the popularity of the first popularity parameter and the second popularity parameter, and sort the second category tags according to the sum of the popularity.

7. An automatic training device for an information classification model, characterized in that, include: The acquisition module is used to acquire the first information set; The first processing module is used to determine the second information set based on the first information set; The second processing module is used to determine a first category tag set based on the first information set; the first category tag set includes a new category tag set whose tag content volume ranking exceeds a first preset threshold. The third processing module is used to reconstruct the model training samples based on the first information set and the second information set to obtain the first sample dataset. The fourth processing module is used to input the first classification label set and the first sample dataset into the original model for training to obtain the target classification model; The second processing module is configured to determine a first category tag set based on the first information set, including: All sub-information in the first information set are tagged and classified to determine the second category tags for the sub-information; the second category tags include newly added category tags. The first category tag set is determined based on the second category tags whose content size is greater than a first preset threshold; wherein, the content size sorting is obtained by sorting the second category tags. The first processing module is configured to determine a second information set based on the first information set, including: Extract the publication date and number of information items from the first information set; The second information set is determined based on the information release date and the number of information items. The step of determining the second information set based on the information release date and the number of information items includes: Based on the information release date and the number of information items, several second information subsets are determined; the second information subsets include historical newly added information from single days prior to the information release date; Perform a union operation on the second information subset to determine the second information set.

8. A computer device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the automatic training method for the information classification model as described in any one of claims 1-6.

9. A computer-readable storage medium storing a processor-executable program, characterized in that: The processor-executable program, when executed by the processor, is used to implement the automatic training method for the information classification model as described in any one of claims 1-6.

Images