Classification model training method and equipment suitable for mass data classification and medium

Abstract

The embodiment of the invention relates to a classification model training method and device suitable for mass data classification and a medium, and the method comprises the steps: obtaining a positive sample carrying a first domain label and a negative sample carrying a second domain label, the first domain label being used for indicating that the positive sample belongs to a target domain, and the second domain label being used for indicating that the negative sample belongs to a target domain; the second field label is used for indicating that the negative sample does not belong to the target field, so that the sample is divided into two categories of belonging to the target field and not belonging to the target field, the initial model is trained by utilizing the samples to obtain the classification model, and the classification model only needs to complete a binary classification task on the input text; according to the method, the calculation cost and resource consumption of text classification are reduced, the processing efficiency of text classification is improved, the method is suitable for classification of mass data, the training data set is divided into the training subset and the verification subset to train the model, the performance of the classification model can be improved, and the accuracy and stability of the classification result are improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of text classification in the field of natural language processing, and in particular to a classification model training method, device and medium suitable for massive data classification. Background Art

[0002] There is a massive amount of text data on the Internet, covering various fields. How to accurately identify and classify content in specific fields from this data is an important technical challenge.

[0003] Currently, related technologies typically use large language models to directly classify text data when performing text classification. However, existing large language models still face limitations in recognizing the diversity of text content, such as limitations in feature extraction and a lack of understanding of context. This approach results in high subjectivity, inaccurate classification results, and poor stability. Summary of the Invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a classification model training method, device and medium suitable for massive data classification.

[0005] In a first aspect, the present disclosure provides a classification model training method suitable for massive data classification, including:

[0006] Obtain a training data set, where the training data set includes positive samples and negative samples, where the positive samples carry a first domain label and the negative samples carry a second domain label, where the first domain label is used to indicate that the positive samples belong to a target domain and the second domain label is used to indicate that the negative samples do not belong to the target domain;

[0007] Dividing the training data set into a training subset and a validation subset, wherein both the training subset and the validation subset include the positive samples and the negative samples, and the training subset and the validation subset have no intersection;

[0008] An initial model is trained based on the training subset and the validation subset to obtain a classification model. The classification model is used to perform domain prediction on the input text and output a first confidence that the input text is predicted to be the first domain label and a second confidence that the second domain label.

[0009] In a second aspect, the present disclosure provides a classification model training device suitable for massive data classification, including:

[0010] an acquisition module, configured to acquire a training data set, the training data set including positive samples and negative samples, the positive samples carrying a first domain label, the negative samples carrying a second domain label, the first domain label being used to indicate that the positive samples belong to a target domain, and the second domain label being used to indicate that the negative samples do not belong to the target domain;

[0011] A partitioning module, configured to divide the training data set into a training subset and a validation subset, wherein both the training subset and the validation subset include the positive samples and the negative samples, and the training subset and the validation subset have no intersection;

[0012] A training module is used to train the initial model based on the training subset and the validation subset to obtain a classification model, wherein the classification model is used to perform domain prediction on the input text and output a first confidence that the input text is predicted to be the first domain label and a second confidence that the second domain label.

[0013] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor for reading the executable instructions from the memory and executing the executable instructions to implement the classification model training method suitable for massive data classification as described in the first aspect.

[0014] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to implement the classification model training method suitable for massive data classification as described in the first aspect.

[0015] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the classification model training method suitable for massive data classification as described in the first aspect.

[0016] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art:

[0017] The embodiments of the present disclosure provide a classification model training scheme suitable for massive data classification, which obtains a training data set, where the training data set includes positive samples and negative samples, the positive samples carry a first domain label, and the negative samples carry a second domain label, the first domain label is used to indicate that the positive sample belongs to a target domain, and the second domain label is used to indicate that the negative sample does not belong to the target domain; the training data set is divided into a training subset and a validation subset, wherein the training subset and the validation subset both include positive samples and negative samples, and the training subset and the validation subset have no intersection; an initial model is trained based on the training subset and the validation subset to obtain a classification model, which is used to perform domain prediction on an input text and output a first confidence level that the input text is predicted to be the first domain label and a second confidence level that the second domain label. By adopting the scheme disclosed in the present invention, by obtaining positive samples carrying a first domain label and negative samples carrying a second domain label, the first domain label is used to indicate that the positive sample belongs to the target domain, and the second domain label is used to indicate that the negative sample does not belong to the target domain, the samples are divided into two categories: belonging to the target domain and not belonging to the target domain, and then these samples are used to train the initial model to obtain a classification model, so that the classification model only needs to complete the binary classification task for the input text, which reduces the computational cost and resource consumption of text classification, improves the processing efficiency of text classification, and is suitable for the classification of massive data. In addition, by dividing the training data set into a training subset and a validation subset to train the model, the performance of the classification model can be improved, and the accuracy and stability of the classification results can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0019] Figure 1 A flow chart of a classification model training method applicable to massive data classification provided by an exemplary embodiment of the present disclosure;

[0020] Figure 2 A schematic diagram showing the effect of testing a classification model according to an exemplary embodiment of the present disclosure is shown;

[0021] Figure 3 A flowchart of a classification model training method applicable to massive data classification provided by another exemplary embodiment of the present disclosure;

[0022] Figure 4 A schematic diagram showing a process of extracting domain content from massive data according to an exemplary embodiment of the present disclosure is shown;

[0023] Figure 5A schematic diagram of the structure of a classification model training device suitable for massive data classification provided by one embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0025] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0026] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0027] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0028] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0029] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0030] Currently, the main text classification methods in the relevant technologies are as follows: rule-based matching, simple machine learning model classification, and direct classification using pre-trained large language models. These methods have some shortcomings. For example, because the rules are manually defined based on experience and have limited coverage, rule-based matching methods are difficult to cope with the needs of processing massive amounts of data. Direct classification using large language models suffers from unstable and highly subjective classification results. Machine learning model classification methods require training the machine learning model, and the training data is manually annotated, which is costly and inefficient. Furthermore, the amount of manually annotated training data is limited, and models trained with limited training data lack interpretability, making them difficult to cope with the needs of processing massive amounts of data.

[0031] In response to the above problems, the present disclosure provides a classification model training method suitable for massive data classification, by obtaining positive samples carrying a first domain label and negative samples carrying a second domain label, the first domain label is used to indicate that the positive sample belongs to the target domain, and the second domain label is used to indicate that the negative sample does not belong to the target domain, thereby achieving the goal of dividing the samples into two categories: belonging to the target domain and not belonging to the target domain, and then using these samples to train the initial model to obtain a classification model, so that the classification model only needs to complete the binary classification task for the input text, reducing the computational cost and resource consumption of text classification, and improving the processing efficiency of text classification, and by dividing the training data set into a training subset and a validation subset to train the model, the performance of the classification model can be improved, and the accuracy and stability of the classification results can be improved. The classification model trained using the scheme of the present disclosure can achieve efficient, stable and well-interpretable large-scale text classification, and is suitable for extracting content in the target domain from massive data, especially for the classification of large-scale web page data in the financial and government fields.

[0032] The following describes in detail the classification model training method, device and medium for massive data classification provided by the present disclosure with reference to the accompanying drawings.

[0033] Figure 1 A flow chart of a classification model training method suitable for massive data classification provided in an exemplary embodiment of the present disclosure is provided. The method can be executed by a classification model training device suitable for massive data classification provided in an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can be integrated into an electronic device.

[0034] like Figure 1 As shown, the classification model training method suitable for massive data classification may include the following steps:

[0035] Step 101: Obtain a training data set. The training data set includes positive samples and negative samples. The positive samples carry a first domain label, and the negative samples carry a second domain label. The first domain label is used to indicate that the positive sample belongs to the target domain, and the second domain label is used to indicate that the negative sample does not belong to the target domain.

[0036] The target domain can be set according to actual business needs. For example, the target domain can be the financial field, medical field, entertainment field, automotive field, etc. The representation of the first domain label and the second domain label can be pre-set according to needs. For example, the number "1" can be used to represent the first domain label, and the number "0" can be used to represent the second domain label; for example, the character "A" can be used to represent the first domain label, and the character "B" can be used to represent the second domain label; for example, the Chinese character "positive" can be used to represent the first domain label, and the Chinese character "negative" can be used to represent the second domain label.

[0037] In this embodiment, for the obtained sample text, positive samples can be annotated using the first domain label or negative samples can be annotated using the second domain label according to the specific content of the sample text, so as to obtain a labeled training data set.

[0038] As an example, sample texts may be annotated manually to obtain a training dataset.

[0039] As another example, the model can be used to annotate sample text and score the training dataset.

[0040] Step 102: Divide the training data set into a training subset and a validation subset, wherein both the training subset and the validation subset include positive samples and negative samples, and the training subset and the validation subset have no intersection.

[0041] In this embodiment, the obtained training dataset can be divided into two parts: a training subset and a validation subset. The number of samples in the training subset is generally greater than the number of samples in the validation subset. To ensure model performance and avoid overfitting, both the training and validation subsets include positive and negative samples, and the training and validation subsets are disjoint. In other words, the training and validation subsets do not contain the same positive or negative samples.

[0042] As an example, when dividing the training data set, it can be divided according to a preset ratio. For example, the preset ratio is 7:3, that is, the sample number ratio of the training subset and the validation subset is 7:3, then the training data set is divided into the training subset and the validation subset according to the ratio of 7:3.

[0043] Step 103: Train the initial model based on the training subset and the validation subset to obtain a classification model. The classification model is used to perform domain prediction on the input text and output a first confidence level that the input text is predicted to be a first domain label and a second confidence level that the input text is predicted to be a second domain label.

[0044] In this embodiment, after obtaining the training subset and the validation subset, the initial model can be trained based on the training subset and the validation subset. After the training is completed, a classification model is obtained.

[0045] Among them, the initial model can adopt the FastText model (an efficient text classification model) or other lightweight models, such as the Text Convolutional Neural Network (TextCNN) and the Long Short-Term Memory Network (LSTM).

[0046] In an optional embodiment of the present disclosure, when training the initial model, the hyperparameter scores of the initial model can be adjusted to obtain multiple initial models, and the multiple initial models can be trained separately using the training subset. After the training is completed, the performance of the multiple trained models can be verified separately using the verification subset, and the model with the best performance among the multiple trained models can be selected as the final classification model.

[0047] In an optional embodiment of the present disclosure, when training the initial model, the initial model can be iteratively trained based on the training subset under the current hyperparameters of the initial model to optimize the model parameters of the initial model and obtain a current model with a training error less than a first preset value; thereafter, the generalization error of the current model is verified based on the verification subset, and when the generalization error is greater than a second preset value, the model hyperparameters of the current model are adjusted, and the adjusted model is iteratively trained as the initial model, and the above training process is repeated until the generalization error is no greater than the second preset value, the training is completed, and a classification model is obtained.

[0048] Among them, the first preset value and the second preset value can be pre-set according to actual needs, and the present disclosure does not limit their specific values.

[0049] It should be noted that, in this embodiment, model parameters refer to common parameters of the model, such as weights and biases; model hyperparameters refer to parameters such as the number of network layers, number of network nodes, number of iterations, learning rate, and optimizer of the model.

[0050] In this embodiment, the model parameters of the initial model are first trained on the training subset using the learning algorithm of the initial model under the current hyperparameters of the initial model, so that the training error of the initial model on the training subset is less than a first preset value. For ease of description, the model with a training error less than the first preset value is called the current model, wherein the training error can be calculated using the loss function of the initial model. When optimizing the model parameters, the model parameters can be updated by gradient backpropagation. After obtaining the current model, the generalization error of the current model is further verified using the validation subset. The larger the generalization error, the weaker the generalization ability of the current model. For example, the generalization error can be represented by the classification error rate of the current model on the validation subset, that is, the generalization error is represented by the number of samples incorrectly classified by the current model on the validation subset divided by the total number of samples in the validation subset. If the generalization error of the current model is greater than the second preset value, the hyperparameters of the current model are adjusted, wherein at least one hyperparameter of the current model can be adjusted. After adjusting the hyperparameters of the current model, the model obtained after adjusting the hyperparameters is used as the initial model for iterative training, and the model parameters are updated on the training subset under the current hyperparameters of the initial model (i.e., the hyperparameters after this adjustment). The above training process is repeated until the generalization error of the current model is no greater than the second preset value, the training is completed, and a trained classification model is obtained.

[0051] Through the above iterative training process, the model parameters are optimized by automatic parameter adjustment technology during the training process, and an efficient, stable and well-interpretable classification model can be trained, which is suitable for the classification of large-scale text data in the target field.

[0052] In this embodiment, the obtained classification model is a binary classification model, which can perform domain prediction (i.e., classification) of the target domain for the input text data, and predict the confidence level of the input text being identified as a first domain label (referred to as the first confidence level for ease of description and distinction) and the confidence level of the input text being identified as a second domain label (referred to as the second confidence level).

[0053] It can be understood that the higher the first confidence, the greater the possibility that the input text belongs to the target domain. Conversely, the higher the second confidence, the greater the possibility that the input text does not belong to the target domain.

[0054] Exemplarily, for the first confidence and second confidence input into the classification model, the sizes of the two can be compared. If the first confidence is greater than the second confidence, it can be determined that the input text belongs to the target domain. If the first confidence is not greater than the second confidence, it can be determined that the input text does not belong to the target domain.

[0055] In the disclosed embodiments, the trained classification model can be used to process massive amounts of data, identifying text data belonging to the target domain. Experimental tests have shown that the classification model trained using the disclosed solution has high processing efficiency, with a daily processing capacity of 80 million data items, low resource consumption, and a processing time of approximately 1.2 milliseconds per data item. It also has a high accuracy rate of 91.84%.

[0056] Figure 2 FIG. 1 shows a schematic diagram showing the effect of testing a classification model according to an exemplary embodiment of the present disclosure. Figure 2 In the example of the target field being finance, the confidence level indicates the confidence level of the data being identified as financial field data. The entire data set used for testing contains 10,000 data items. The proportion of financial data refers to the percentage of data that is actually financial field data in the data identified as financial data by the classification model. The remarks section is a hyperlink to a table of financial field data with corresponding confidence levels. Click the corresponding hyperlink to display the corresponding financial field data. Figure 2 It can be seen that the amount of financial data with a confidence level higher than 0.98 is 283, and the proportion of financial data is 99.6%.

[0057] The embodiments of the present disclosure provide a classification model training method suitable for massive data classification, which obtains a training data set, where the training data set includes positive samples and negative samples, the positive samples carry a first domain label, and the negative samples carry a second domain label, the first domain label is used to indicate that the positive sample belongs to a target domain, and the second domain label is used to indicate that the negative sample does not belong to the target domain; the training data set is divided into a training subset and a validation subset, wherein the training subset and the validation subset both include positive samples and negative samples, and the training subset and the validation subset have no intersection; an initial model is trained based on the training subset and the validation subset to obtain a classification model, which is used to perform domain prediction on an input text and output a first confidence level that the input text is divided into the first domain label and a second confidence level of the second domain label. By adopting the scheme disclosed in the present invention, by obtaining positive samples carrying a first domain label and negative samples carrying a second domain label, the first domain label is used to indicate that the positive sample belongs to the target domain, and the second domain label is used to indicate that the negative sample does not belong to the target domain, the samples are divided into two categories: belonging to the target domain and not belonging to the target domain, and then these samples are used to train the initial model to obtain a classification model, so that the classification model only needs to complete the binary classification task for the input text, which reduces the computational cost and resource consumption of text classification, improves the processing efficiency of text classification, and is suitable for the classification of massive data. In addition, by dividing the training data set into a training subset and a verification subset to train the model, the performance of the classification model can be improved, and the accuracy and stability of the classification results can be improved.

[0058] In order to solve the problem of high cost and low efficiency of manual labeling of training data in related technologies, the present disclosure provides a solution for automatically labeling the original sample data based on the score by scoring the original sample data, thereby realizing automatic labeling of training samples, saving labor costs and improving labeling efficiency. Figure 3 As shown, based on the above embodiment, step 101 may include the following sub-steps:

[0059] Step 201: Obtain the score of each original sample in the original sample data in the target field.

[0060] Among them, the original sample data contains multiple original samples, and the fields involved in these original samples are diverse. The original samples involving the target field obtain higher scores, and the original samples not involving the target field obtain lower scores.

[0061] It is understood that raw sample data can be obtained in a variety of ways. For example, webpage text from different fields can be crawled from the Internet as raw sample data; for another example, partial data can be obtained from existing large-scale datasets as raw sample data. This disclosure does not limit the method of obtaining raw sample data.

[0062] In the embodiments of the present disclosure, different methods may be used to score the original samples.

[0063] As an example, when obtaining the score of each original sample in the target domain, a scoring model can be pre-trained, and the scoring model can be used to obtain the score of each original sample in the original sample data in the target domain.

[0064] As an example, when obtaining the score of each original sample in the target domain, the scoring rules of the target domain can be obtained. The scoring rules include scoring criteria corresponding to different scores; wherein the scoring criteria corresponding to each score can be manually defined according to actual needs. Then, based on the scoring rules, a scoring prompt text for the target domain is generated. The scoring prompt text includes the scoring rules and the output format, wherein the output format is used to indicate the format of the model output results, and the output format can be manually set according to actual needs. Optionally, the scoring prompt text can also include at least one example text and an example description of the corresponding score, so that the model can better learn the scoring rules. Then, the scoring prompt text and the original sample data can be input into a pre-trained scoring model, so that the scoring model scores each original sample in the original sample data based on the scoring prompt text and outputs it according to the output format; wherein the scoring model can be obtained by simply training a large language model. The large language model used can be, but is not limited to, the Tongyi Qianwen 2.0 large language model Qwen2, the Generative Pre-trained Transformer (GPT) model, and the Bidirectional Encoder Representations from Transformers (BERT) model. Furthermore, the score of each original sample in the target field output by the scoring model in the output format can be obtained.

[0065] For example, taking the target field of finance as an example, the generated scoring prompt text can be as follows:

[0066] "

Financial Information Evaluation Standards

[0067] To be considered for financial information scoring, the text must cover the following aspects; otherwise, it will receive a score of 0. Financial information includes: financial reports, financial news, financial contracts, financial market data, financial institution information, financial regulatory policies, financial risk assessments, financial fraud cases, investment strategies and analysis, and financial technology developments.

[0068] The following are the scoring criteria:

[0069] 0 points: If the text contains no financial content at all, the information contained is irrelevant to financial topics and consists primarily of advertisements, promotional materials, commercial content, or other irrelevant information. The content of the text may be completely off the financial track and fail to provide users with any meaningful financial reference or data. This type of text contains no substantive financial information and does not address financial markets, investment analysis, or fintech.

[0070] 1 point: If the text provides some basic financial information, but the content is minimal and superficial. It may mention keywords or overviews of areas such as financial markets and financial reports, but does not provide detailed background or context. This type of text may contain a large amount of non-financial content (such as advertisements and promotional materials). The main information is not related to the financial topic, with only a few relevant points, and fails to provide useful financial guidance or assistance to users.

[0071] 2 points: If the text provides some financial-related content, the quality is low, the structure is loose, and financial information is mixed with other unrelated content, making it difficult to effectively identify key points. The financial information may only be a superficial overview or in-depth citations, lacking in-depth explanation of market analysis, investment strategies, and other aspects. The text lacks coherence, and the information is one-sided or difficult to understand, failing to provide accurate financial analysis or assistance to users. While it may touch on financial topics, the overall quality is insufficient and does not meet the standards for practical application.

[0072] 3 points: If the text provides content applicable to the financial sector, but may only cover basic financial reports, market data, investment analysis, and other areas. This type of text is generally coherent and has some reference value, but the content is relatively simple and may lack comprehensiveness and depth. The text resembles an introductory financial guide. While suitable for ordinary investors, it fails to discuss key financial issues in detail and does not provide in-depth analysis for professionals. This type of content is useful, but its detail and applicability are limited, especially when dealing with complex financial topics.

[0073] 4 points: If the text is highly relevant and substantively helpful in financial matters, particularly with detailed and coherent descriptions of key areas such as financial reporting, market analysis, investment strategies, and financial risk assessment. This type of text has clear logic and a well-organized structure, providing users with reliable financial information. The text is comprehensive enough to be used by financial analysts, investment advisors, or policy researchers, and also provides clear financial explanations for the general public. The text may include relevant data and case studies, but may lack in-depth discussion of certain details or more specialized areas. However, it is still a high-quality financial reference source.

[0074] 5 points: If the text excels in providing financial information, covering a wide range of key financial areas, such as market analysis, investment strategies, and fintech developments. The text is logically coherent, detailed, and insightful, providing users with systematic, accurate, and comprehensive financial analysis and interpretation. This type of text is suitable not only for financial professionals but also for the general public, presenting complex financial issues in a concise and accessible manner. The text is focused, with little irrelevant or distracting content, and contains insightful data, legal interpretations, case studies, and market impact analysis, making it suitable for both professional and everyday financial work.

[0075] You are a domain classification expert with an IQ of 200. Please rate the text in the financial field based on the above criteria.

[0076] Please output the analysis results in the following Chinese output format at the end of the analysis:

[0077]

Final score of the text

[0078] [Rating text]: {content}".

[0079] In the embodiment of the present disclosure, by obtaining the scoring rules of the target domain, a scoring prompt text containing the scoring rules and the output format is generated based on the scoring rules to guide the scoring model to score the input original samples in the target domain and output the results in the output format. This can ensure the accuracy of the scoring results of the original samples and lay the foundation for the subsequent positive and negative sample labeling. Moreover, since the target domain and its corresponding scoring rules can be flexibly set according to the requirements of domain content extraction, the scalability is improved, and an extensible domain text classification system can be realized.

[0080] Step 202: Based on the score of each original sample, the original sample data is divided into original positive samples and original negative samples. The original positive samples carry the first domain label, and the original negative samples carry the second domain label.

[0081] In this embodiment, after determining the score of each original sample, the original sample data can be divided into original positive samples and original negative samples based on the score of each original sample, and a first domain label is labeled for each original positive sample, and a second domain label is labeled for each original negative sample.

[0082] As an example, when dividing raw sample data into raw positive samples and raw negative samples, a preset scoring threshold can be obtained. Then, based on the scoring threshold and the score of each raw sample, the raw sample data is divided into raw positive samples and raw negative samples. Raw samples with scores above the scoring threshold are raw positive samples, and raw samples with scores below the scoring threshold are raw negative samples. This achieves automated sample labeling, reduces the cost of manually labeling training samples, and improves sample labeling efficiency.

[0083] Among them, the scoring threshold can be pre-set according to actual needs.

[0084] For example, assuming the score range of the original samples is 0 to 5, the score threshold can be set to 3. Based on the score threshold and the score corresponding to each original sample, the score of the original sample can be binarized, with scores of 0-3 mapped to 0, and the corresponding original samples are the original negative samples, and 0 is the second domain label carried by these original negative samples. Correspondingly, scores of 4-5 are mapped to 1, and the corresponding original samples are the original positive samples, and 1 is the first domain label carried by these original positive samples.

[0085] Step 203 : According to a preset positive-negative sample ratio, positive samples are determined from the original positive samples and negative samples are determined from the original negative samples to obtain a training data set.

[0086] The ratio of positive and negative samples can be set according to actual training requirements or experience. For example, the ratio of positive and negative samples can be set to 1:2.

[0087] In this embodiment, after the original positive samples and the original negative samples are determined, positive samples can be determined from the original positive samples and negative samples can be determined from the original negative samples to obtain a training data set, and the ratio of positive and negative samples in the training data set meets the positive and negative sample ratio requirements.

[0088] As an example, a certain number of positive samples can be randomly selected from the original positive samples, and then the number of negative samples to be selected is determined based on the positive-negative sample ratio and the number of positive samples selected, and a corresponding number of negative samples are randomly selected from the original negative samples.

[0089] As another example, a preset sample quantity value can be first obtained. This sample quantity value refers to the total number of positive and negative samples required. The specific value can be set according to actual training requirements, and this disclosure does not impose any restrictions on this. Then, based on this sample quantity value and the preset positive-negative sample ratio, the number of positive and negative samples to be obtained is determined. Then, based on the number of positive samples, positive samples are determined from the original positive samples, and based on the number of negative samples, negative samples are determined from the original negative samples. Then, a training dataset is constructed based on the determined positive and negative samples.

[0090] Specifically, when determining positive samples from original positive samples based on the number of positive samples, and determining negative samples from original negative samples based on the number of negative samples, different methods may be used for determination.

[0091] As an example, a number of positive samples may be randomly sampled from the original positive samples, and a number of negative samples may be randomly sampled from the original negative samples.

[0092] As another example, the original positive samples can be divided into multiple positive sample subsets based on the score of each original positive sample in the original positive samples, and the original negative samples can be divided into multiple negative sample subsets based on the score of each original negative sample in the original negative samples, wherein samples with the same score are in the same subset, that is, original positive samples with the same score are in the same positive sample subset, and original negative samples with the same score are in the same negative sample subset; then, based on the number of positive samples, positive samples are evenly obtained from multiple positive sample subsets. For example, if there are two positive sample subsets and the number of positive samples to be extracted is 1,000, 500 positive samples can be extracted from each of the two positive sample subsets; similarly, based on the number of negative samples, negative samples are evenly obtained from multiple negative sample subsets. Thus, by evenly obtaining positive / negative samples from original positive / negative samples with different scores, the training data set finally obtained evenly covers samples of various scores, ensuring the balance of the samples and helping to ensure the training effect of the model.

[0093] The classification model training method suitable for massive data classification of the embodiment of the present disclosure obtains the score of each original sample in the original sample data in the target field, and divides the original sample data into original positive samples and original negative samples based on the score of each original sample. The original positive samples carry the first field label, and the original negative samples carry the second field label. Then, according to the preset positive-negative sample ratio, the positive samples are determined from the original positive samples and the negative samples are determined from the original negative samples to obtain the training data set. In this way, automatic labeling of training samples is achieved, the cost of manual labeling of samples is saved, the efficiency of sample labeling is improved, and the problem of high cost and low efficiency of manual labeling of training data in related technologies is solved.

[0094] In an optional embodiment of the present disclosure, when dividing a training data set into a training subset and a validation subset, the training data set can be first divided into a positive sample set and a negative sample set. For example, based on the domain labels carried by each sample in the training data set, the positive samples carrying the first domain label can be divided into the positive sample set, and the negative samples carrying the second domain label can be divided into the negative sample set; then, the positive sample set can be divided into a positive sample training subset and a positive sample validation subset according to a preset division ratio, and the negative sample set can be divided into a negative sample training subset and a negative sample validation subset according to the division ratio, wherein the division ratio can be pre-set according to actual needs, for example, the division ratio of the training set to the validation set can be set to 7:3, 8:2, etc. It should be noted that if the number of samples contained in the positive sample set and / or the negative sample set cannot be divided exactly according to the division ratio, the number of samples in each subset can be determined by rounding off. For example, assuming that the positive sample set contains 103 positive samples and the division ratio is 7:3, when the positive sample set is divided according to the division ratio, the number of positive sample training subsets is 103*0.7=72.1, and the number of positive sample validation subsets is 103*0.3=30.9. Then, 72 positive samples in the positive sample set can be divided into positive sample training subsets, and the remaining 31 positive samples can be divided into positive sample validation subsets. Similarly, the negative sample set can be divided into negative sample training subsets and negative sample validation subsets. Then, a training subset is constructed based on the positive sample training subset and the negative sample training subset, and a validation subset is constructed based on the positive sample validation subset and the negative sample validation subset. In this way, the number of positive samples and the number of negative samples in the training subset and the validation subset can both meet the division ratio requirements, ensuring the balance of positive and negative samples in the training subset and the validation subset, thereby helping to improve the training effect of the classification model.

[0095] Figure 4 FIG. 1 shows a schematic diagram of a process of extracting domain content from massive data according to an exemplary embodiment of the present disclosure. Figure 4 As shown, taking the scoring model using Qwen2-72B and the classification model using FastText as an example, some data is obtained from the massive data (for example, 115k data is obtained from the 12T dataset) as the original sample data, and the 115k original sample data is scored with a score of 0-5 based on the Qwen2-72B model and the scoring prompt text (Prompt). Then, the samples are balanced and sample data with a preset positive and negative sample ratio is selected, totaling 17,769 data. Then, the scores of these 17,769 data are binarized to obtain 17,769 0-1 classification data, thereby obtaining negative samples carrying 0 labels and positive samples carrying 1. These samples are used to train the FastText model. The trained FastText model can be used to annotate the massive 12T data and complete the domain classification of these data.

[0096] In order to implement the above embodiments, the present disclosure also provides a classification model training device suitable for massive data classification.

[0097] Figure 5 This is a structural diagram of a classification model training device suitable for massive data classification provided by an embodiment of the present disclosure. The device is implemented in software and / or hardware and can be integrated into an electronic device.

[0098] like Figure 5 As shown, the classification model training device 50 suitable for mass data classification may include: an acquisition module 510, a division module 520 and a training module 530.

[0099] The acquisition module 510 is configured to acquire a training data set, where the training data set includes positive samples and negative samples. The positive samples carry a first domain label, and the negative samples carry a second domain label. The first domain label is used to indicate that the positive samples belong to the target domain, and the second domain label is used to indicate that the negative samples do not belong to the target domain.

[0100] A partitioning module 520 is configured to partition the training data set into a training subset and a validation subset, wherein both the training subset and the validation subset include positive samples and negative samples, and the training subset and the validation subset have no intersection;

[0101] The training module 530 is used to train the initial model based on the training subset and the validation subset to obtain a classification model. The classification model is used to perform domain prediction on the input text and output a first confidence level that the input text is predicted to be a first domain label and a second confidence level that the input text is predicted to be a second domain label.

[0102] Optionally, the acquisition module 510 includes:

[0103] A score acquisition unit, used to obtain the score of each original sample in the original sample data in the target field;

[0104] A sample division unit is used to divide the original sample data into original positive samples and original negative samples based on the score of each original sample, where the original positive samples carry the first domain label and the original negative samples carry the second domain label;

[0105] The sample determination unit is used to determine positive samples from the original positive samples and negative samples from the original negative samples according to a preset positive-negative sample ratio to obtain a training data set.

[0106] Further optionally, the score acquisition unit is further configured to:

[0107] Obtain the scoring rules for the target field, including the scoring criteria corresponding to different scores;

[0108] Generate scoring prompt text for the target field based on the scoring rules, the scoring prompt text including the scoring rules and output format;

[0109] Inputting the scoring prompt text and the original sample data into a pre-trained scoring model so that the scoring model scores each original sample in the original sample data based on the scoring prompt text;

[0110] Get the score of each original sample in the target field output by the scoring model in the output format.

[0111] Optionally, the sample division unit is further configured to:

[0112] Get the preset scoring threshold;

[0113] Based on the scoring threshold and the score of each original sample, the original sample data is divided into original positive samples and original negative samples, where the original samples with scores higher than the scoring threshold are original positive samples, and the original samples with scores not higher than the scoring threshold are original negative samples.

[0114] Optionally, the sample determination unit is further configured to:

[0115] Get the preset sample size value;

[0116] Based on the sample size value and the positive-negative sample ratio, determine the number of positive samples and the number of negative samples to be obtained;

[0117] Determine positive samples from the original positive samples based on the number of positive samples, and determine negative samples from the original negative samples based on the number of negative samples;

[0118] Construct a training dataset based on positive samples and negative samples.

[0119] Further optionally, the sample determination unit is further configured to:

[0120] Dividing the original positive samples into multiple positive sample subsets based on the score of each original positive sample in the original positive samples, and dividing the original negative samples into multiple negative sample subsets based on the score of each original negative sample in the original negative samples, wherein samples with the same score are in the same subset;

[0121] Based on the number of positive samples, positive samples are evenly obtained from multiple positive sample subsets;

[0122] Based on the number of negative samples, negative samples are evenly obtained from multiple negative sample subsets.

[0123] Optionally, the dividing module 520 is further configured to:

[0124] Divide the training data set into positive sample set and negative sample set;

[0125] According to the preset division ratio, the positive sample set is divided into a positive sample training subset and a positive sample verification subset;

[0126] According to the division ratio, the negative sample set is divided into a negative sample training subset and a negative sample verification subset;

[0127] A training subset is constructed based on the positive sample training subset and the negative sample training subset, and a validation subset is constructed based on the positive sample validation subset and the negative sample validation subset.

[0128] Optionally, the training module 530 is further configured to:

[0129] Under the current hyperparameters of the initial model, iteratively training the initial model based on the training subset to optimize the model parameters of the initial model to obtain a current model with a training error less than a first preset value;

[0130] The generalization error of the current model is verified based on the validation subset. When the generalization error is greater than the second preset value, the model hyperparameters of the current model are adjusted, and the adjusted model is used as the initial model for iterative training until the generalization error is no greater than the second preset value. The training is completed and a classification model is obtained.

[0131] The classification model training device for mass data classification provided in the embodiments of the present disclosure, applied to an electronic device, can execute the classification model training method for mass data classification provided in the embodiments of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method. For any content not fully described in the embodiments of the present disclosure, reference can be made to the description of any method embodiment of the present disclosure.

[0132] An embodiment of the present disclosure also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the classification model training method suitable for massive data classification provided by any embodiment of the present disclosure.

[0133] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

[0134] processor;

[0135] a memory for storing instructions executable by the processor;

[0136] The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the classification model training method suitable for massive data classification as provided in any embodiment of the present disclosure.

[0137] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium, which stores a computer program, and the computer program is used to implement the classification model training method suitable for massive data classification as provided in any embodiment of the present disclosure.

[0138] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0139] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0140] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0141] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0142] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0143] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0144] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0145] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0146] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A classification model training method suitable for massive data classification, characterized in that: The method comprises: Obtain a training data set, where the training data set includes positive samples and negative samples, where the positive samples carry a first domain label and the negative samples carry a second domain label, where the first domain label is used to indicate that the positive samples belong to a target domain and the second domain label is used to indicate that the negative samples do not belong to the target domain; Dividing the training data set into a training subset and a validation subset, wherein both the training subset and the validation subset include the positive samples and the negative samples, and the training subset and the validation subset have no intersection; An initial model is trained based on the training subset and the validation subset to obtain a classification model. The classification model is used to perform domain prediction on the input text and output a first confidence that the input text is predicted to be the first domain label and a second confidence that the second domain label.

2. The method according to claim 1, characterized in that The obtaining of the training data set includes: Obtaining a score of each original sample in the original sample data in the target field; Based on the score of each original sample, dividing the original sample data into original positive samples and original negative samples, the original positive samples carry the first domain label, and the original negative samples carry the second domain label; According to a preset positive-negative sample ratio, the positive samples are determined from the original positive samples and the negative samples are determined from the original negative samples to obtain the training data set.

3. The method according to claim 2, characterized in that Obtaining a score of each original sample in the original sample data in the target field includes: Obtaining a scoring rule for the target field, wherein the scoring rule includes scoring criteria corresponding to different scores; generating a scoring prompt text for the target field based on the scoring rule, wherein the scoring prompt text includes the scoring rule and an output format; Inputting the scoring prompt text and the original sample data into a pre-trained scoring model, so that the scoring model scores each original sample in the original sample data based on the scoring prompt text; Obtain a score of each original sample in the target domain output by the scoring model in the output format.

4. The method according to claim 2, characterized in that The dividing the original sample data into original positive samples and original negative samples based on the score of each original sample includes: Get the preset scoring threshold; Based on the scoring threshold and the score of each original sample, the original sample data is divided into original positive samples and original negative samples, wherein the original samples with scores higher than the scoring threshold are original positive samples, and the original samples with scores not higher than the scoring threshold are original negative samples.

5. The method according to claim 2, characterized in that The step of determining the positive samples from the original positive samples and the negative samples from the original negative samples according to a preset positive-negative sample ratio to obtain the training data set includes: Get the preset sample quantity value; Determining the number of positive samples and the number of negative samples to be obtained based on the sample quantity value and the positive-negative sample ratio; Determining the positive sample from the original positive samples based on the number of positive samples, and determining the negative sample from the original negative samples based on the number of negative samples; The training data set is constructed based on the positive samples and the negative samples.

6. The method according to claim 5, characterized in that The determining the positive sample from the original positive samples based on the number of positive samples, and determining the negative sample from the original negative samples based on the number of negative samples, includes: Dividing the original positive samples into a plurality of positive sample subsets based on the score of each original positive sample in the original positive samples, and dividing the original negative samples into a plurality of negative sample subsets based on the score of each original negative sample in the original negative samples, wherein samples with the same score are in the same subset; Based on the number of positive samples, evenly obtain the positive samples from the multiple positive sample subsets; Based on the number of negative samples, the negative samples are uniformly obtained from the multiple negative sample subsets.

7. The method according to claim 1, characterized in that The step of dividing the training data set into a training subset and a validation subset comprises: Dividing the training data set into a positive sample set and a negative sample set; Dividing the positive sample set into a positive sample training subset and a positive sample verification subset according to a preset division ratio; Dividing the negative sample set into a negative sample training subset and a negative sample validation subset according to the division ratio; The training subset is constructed based on the positive sample training subset and the negative sample training subset, and the validation subset is constructed based on the positive sample validation subset and the negative sample validation subset.

8. The method according to any one of claims 1 to 7, characterized in that The initial model is trained based on the training subset and the validation subset to obtain a classification model, comprising: Under the current hyperparameters of the initial model, iteratively training the initial model based on the training subset to optimize the model parameters of the initial model to obtain a current model with a training error less than a first preset value; The generalization error of the current model is verified based on the validation subset. When the generalization error is greater than a second preset value, the model hyperparameters of the current model are adjusted, and the adjusted model is used as the initial model for iterative training until the generalization error is no greater than the second preset value. The training is completed to obtain the classification model.

9. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the classification model training method suitable for massive data classification described in any one of claims 1-8 above.

10. A computer-readable storage medium, characterized in that The storage medium stores a computer program, which is used to implement the classification model training method suitable for massive data classification as described in any one of claims 1 to 8.

Images