Method and system for summarizing rules by using large-class sampling data and small-class data

Abstract

The invention provides a method and a system for summarizing rules by utilizing large-class sampling data and small-class data, which comprises the following steps of: processing a basic text: performing word segmentation, text embedding and dimension reduction operations on unstructured text complete sets to obtain a low-dimensional feature vector of each text complete set; text clustering: clustering the text complete set according to the low-dimensional feature vector; extracting rules, sampling M sub-data sets of which the sample size is about n from the text complete set according to a classification result, and assigning one of the sub-data sets as a training set and the rest as test sets; annotation and algorithm iteration are carried out, and all training set data and test set data are annotated; performing algorithm inspection, and sampling an inspection set from the text complete set; according to the method, an unsupervised algorithm is used for preprocessing, and a large amount of early-stage data annotation work is avoided by using a method of performing rule summarization and annotation by using large-class sampling data and small-class data; the text range needing to be concerned by rule summarization is narrowed; and the amount of data needing to be manually processed is reduced.

Description

[0001] This application is a divisional application of the following patent application:

[0002] Application number: 202310224764.8

[0003] Application date: 2023-03-07

[0004] Application title: Clustering-based extraction method and system for unstructured bidding requirement text Technical Field

[0005] The present invention relates to the technical field of knowledge extraction, and in particular, to a method and system for summarizing rules using large-category sampling data and small-category data. Background Art

[0006] At present, many knowledge extraction methods are mainly divided into two categories. One is the extraction method based on deep learning that relies on a large amount of labeled data; the other is the extraction method that manually organizes rules, both of which require a lot of manual work. Bidding requirements are generally written by the purchaser in the form of text. The terms used by each company may vary greatly, but the focus of each company is relatively similar. Clustering can be used to discover these similarities and improve manual efficiency.

[0007] A Bert-based knowledge extraction method and system for unstructured power data is disclosed in the patent document with publication number CN114996408A. In this method, for knowledge with obvious trigger words, relatively fixed sentence patterns and positions, a regular matching method is used to extract knowledge entities. For the problem of fuzzy entity boundaries and entity nesting in unstructured power data, a Bert-based knowledge extraction method is used to extract knowledge entities, and then the results of entity extraction are used to use a Bert-based pre-trained language model to extract relations and attributes, and finally the knowledge extraction of unstructured power data is realized. The sentence patterns and structures of unstructured power data are fully considered, which is suitable for knowledge extraction of unstructured power data. It belongs to a deep learning knowledge extraction method, and the deep learning method can automatically learn features, use low-dimensional, dense real-valued vectors to represent data, and avoid heavy reliance on manual and expert knowledge. Although the rule + deep learning extraction method is used, there is no systematic pattern for the arrangement of rules, which is inefficient and prone to omissions or errors; at the same time, a large amount of data needs to be annotated for Bert, which consumes a lot of labor costs. At the same time, the interpretability of Bert extraction results is not as good as that of rule models, and the risk of errors is greater in business scenarios.

[0008] Therefore, it is necessary to propose a new technical solution to improve the above technical problems. Summary of the invention

[0009] In view of the defects in the prior art, the purpose of the present invention is to provide a method and system for summarizing rules using large-category sampling data and small-category data.

[0010] According to a method for summarizing rules using large-category sampling data and small-category data provided by the present invention, the method comprises the following steps:

[0011] Step S1: Obtain the historical inquiry data of the platform, and the purchaser publishes the bidding requirement fields; perform word segmentation, text embedding, and dimensionality reduction operations on the unstructured text set to obtain the low-dimensional feature vector of each text set;

[0012] Step S2: clustering the entire text set according to the low-dimensional feature vector; adjusting clustering algorithm parameters to optimize the output classification results;

[0013] Step S3: According to the classification results, M sub-datasets with a sample size of n are sampled from the entire text set, one of which is designated as a training set and the rest as test sets; a clustering algorithm plus manual sorting method is used to sort out text extraction rules for the training set, and a code is written to form a text extraction model;

[0014] Step S4: label all training set data and test set data; process the training set using the text extraction model; repeatedly adjust the training set and test set according to the processing results, and iterate the text extraction rules;

[0015] Until the accuracy is greater than or equal to the expected threshold;

[0016] Step S5: Sample a test set from the entire text set, run the text extraction model and evaluate its results using the confusion matrix, paying attention to the precision and recall rates; if both the precision and recall rates meet the expected thresholds, the text extraction model passes, otherwise continue to iterate the error part.

[0017] Preferably, step S1 comprises the following steps:

[0018] Step S1.1: Based on the industrial product domain vocabulary, use the jieba word segmentation tool to segment the text in the training sample; after segmentation, remove meaningless stop words;

[0019] Step S1.2: Use the Word2vec word embedding model to train the word segmentation sequence after word segmentation to obtain the word embedding model in the supply chain field; select the model result dimension, add and normalize all word vectors in the text, and obtain the low-dimensional feature vector of the text.

[0020] Preferably, step S2 comprises the following steps:

[0021] Step S2.1: If it is the first time to cluster text, set the initial number of clusters or initial clustering conditions, gradually increase the number of clusters or tighten the clustering conditions; the number of texts in each result set gradually decreases; the computer displays the result set, and when similar results are divided into different result sets, or the number of invalid results increases, stop and call back the parameters;

[0022] Step S2.2: If the text is clustered again, there is already a text extraction rule; calculate the score S according to the following clustering effect evaluation score calculation formula:

[0023]

[0024] Where C represents the number of categories, C2 represents the number of multiple rules that match a single category; R represents the number of existing rules, R2 represents the number of multiple categories that match a single rule; N s Represents the total sample size of small categories;

[0025] Try multiple sets of parameters; select one of the results and proceed to step S3.

[0026] Preferably, step S3 comprises the following steps:

[0027] Step S3.1: Distinguish major categories from minor categories according to the threshold set in step S2; randomly sample M data sets from each major category, and the number of each data set is n*u / U, where U is the number of samples in the entire set; u is the number of samples in the major category; n is a preset integer used to control the size of the test set; the training set is 1, and the test set is M-1;

[0028] Step S3.2: Manually organize text extraction rules for the training set data extracted from each major category and the data from each minor category;

[0029] Step S3.3: If each classification cannot summarize the text extraction rules, after summarizing the preset number of rules, trigger step S2.2 to continue execution and re-cluster;

[0030] Step S3.4: Merge all the data of the major categories into M groups of n data sets, and distinguish between training sets and test sets; for each minor category, select two data, one to be added to the training set and the other to be added to any test set;

[0031] Step S3.5: Write a code model for all the sorted text extraction rules.

[0032] Preferably, step S4 comprises the following steps:

[0033] Step S4.1: Label all training set data and test set data;

[0034] Step S4.2: Run the text extraction model of step S3 on the training set, and calculate the accuracy based on the annotated data; if the accuracy is less than the threshold, then trigger step S3 to continue execution, and further organize the text extraction rules according to the error content; if the accuracy is greater than or equal to the threshold, then process all test sets; if the accuracy of the test set is less than the threshold, then add the test set to the training set, and re-extract a test set, triggering step S3 to continue execution;

[0035] Until the accuracy on all test sets is greater than or equal to the threshold.

[0036] The present invention also provides a system for summarizing rules using large-category sampling data and small-category data, the system comprising the following modules:

[0037] Module M1: Obtain the historical inquiry data of the platform and the bidding requirement fields released by the purchaser; perform word segmentation, text embedding, and dimensionality reduction operations on the unstructured text set to obtain the low-dimensional feature vector of each text set;

[0038] Module M2: Cluster the entire text set based on low-dimensional feature vectors; adjust clustering algorithm parameters to optimize the output classification results;

[0039] Module M3: Based on the classification results, M sub-datasets with a sample size of n are sampled from the entire text set, one of which is designated as the training set and the rest as the test set; a clustering algorithm plus a manually organized system is used to organize text extraction rules for the training set, and code is written to form a text extraction model;

[0040] Module M4: Label all training set data and test set data; use the text extraction model to process the training set; according to the processing results, repeatedly adjust the training set and test set, and iterate the text extraction rules;

[0041] Until the accuracy is greater than or equal to the expected threshold;

[0042] Module M5: Sample a test set from the entire text set, run the text extraction model and evaluate its results using the confusion matrix, paying attention to the precision and recall rates; if both the precision and recall rates meet the expected thresholds, the text extraction model passes, otherwise continue to iterate the error part.

[0043] Preferably, the module M1 includes the following modules:

[0044] Module M1.1: Based on the industrial product domain vocabulary, use the Jieba word segmentation tool to segment the text in the training sample; after segmentation, remove meaningless stop words;

[0045] Module M1.2: Use the Word2vec word embedding model to train the word segmentation sequence after word segmentation to obtain the word embedding model in the supply chain field; select the model result dimension, add and normalize all word vectors in the text, and obtain the low-dimensional feature vector of the text.

[0046] Preferably, the module M2 includes the following modules:

[0047] Module M2.1: If it is the first time to cluster text, set the initial number of clusters or initial clustering conditions, gradually increase the number of clusters or tighten the clustering conditions; the number of texts in each result set gradually decreases; the computer displays the result set, and when similar results are divided into different result sets, or the number of invalid results increases, stop and call back the parameters;

[0048] Module M2.2: If the text is clustered again, there are already text extraction rules at this time; calculate the score S according to the following clustering effect evaluation score calculation formula:

[0049]

[0050] Where C represents the number of categories, C2 represents the number of multiple rules that match a single category; R represents the number of existing rules, R2 represents the number of multiple categories that match a single rule; N s Represents the total sample size of small categories;

[0051] Try multiple sets of parameters; select one of the results and enter module M3.

[0052] Preferably, the module M3 includes the following modules:

[0053] Module M3.1: Distinguish major and minor categories according to the threshold set in module M2; randomly sample M data sets from each major category, and the number of each data set is n*u / U, where U is the number of samples in the whole set; u is the number of samples in the major category; n is a preset integer used to control the size of the test set; the training set is 1, and the test set is M-1;

[0054] Module M3.2: Manually organize text extraction rules for the training set data extracted from each major category and the data from each minor category;

[0055] Module M3.3: If each category cannot summarize the text extraction rules, after summarizing the preset number of rules, the module M2.2 is triggered to continue execution and re-cluster;

[0056] Module M3.4: Merge all the data of major categories into M groups of n data sets, and distinguish between training sets and test sets; for each minor category, select two data, one to be added to the training set and the other to any test set;

[0057] Module M3.5: Write code models for all the sorted text extraction rules.

[0058] Preferably, the module M4 includes the following modules:

[0059] Module M4.1: Label all training set data and test set data;

[0060] Module M4.2: Call the text extraction model of module M3 on the training set and calculate the accuracy based on the annotated data; if the accuracy is less than the threshold, module M3 is triggered to continue execution and the text extraction rules are further sorted out according to the error content; if the accuracy is greater than or equal to the threshold, all test sets are processed; if the accuracy of the test set is less than the threshold, the test set is added to the training set and a new test set is extracted, triggering module M3 to continue execution;

[0061] Until the accuracy on all test sets is greater than or equal to the threshold.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] 1. The present invention avoids a lot of preliminary data labeling work by using an unsupervised algorithm (clustering) as preprocessing and a method of using large-category sampling data and small-category data for rule summarization and labeling; at the same time, it narrows the scope of text that needs to be paid attention to in rule summary; and greatly reduces the amount of data that needs to be manually processed;

[0064] 2. The present invention ensures the similarity of texts in the same category by adopting a clustering algorithm; at the same time, small categories are obtained as much as possible during iteration, so that the text features of each category are clearer at a glance, reducing the difficulty of manually summarizing text rules; through scoring and repeated iterations of clusters, the final result gradually tends to one rule for each category, avoiding the omission of rules;

[0065] 3. Since the final extraction rules are all summarized manually, the model has strong interpretability; therefore, it is often possible to fine-tune the rules and obtain a higher precision rate by giving up a certain recall rate; the above two points are of great commercial value. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0067] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0068] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0069] Embodiment 1:

[0070] According to a method for summarizing rules using large-category sampling data and small-category data provided by the present invention, the method comprises the following steps:

[0071] Step S1: Obtain the historical inquiry data of the platform, and the purchaser publishes the bidding requirement fields; perform word segmentation, text embedding, and dimensionality reduction operations on the unstructured text set to obtain the low-dimensional feature vector of each text set;

[0072] Step S1.1: Based on the industrial product domain vocabulary, use the jieba word segmentation tool to segment the text in the training sample; after segmentation, remove meaningless stop words;

[0073] Step S1.2: Use the Word2vec word embedding model to train the word segmentation sequence after word segmentation to obtain the word embedding model in the supply chain field; select the model result dimension, add and normalize all word vectors in the text, and obtain the low-dimensional feature vector of the text.

[0074] Step S2: clustering the entire text set according to the low-dimensional feature vector; adjusting clustering algorithm parameters to optimize the output classification results;

[0075] Step S2.1: If it is the first time to cluster text, set the initial number of clusters or initial clustering conditions, gradually increase the number of clusters or tighten the clustering conditions; the number of texts in each result set gradually decreases; the computer displays the result set, and when similar results are divided into different result sets, or the number of invalid results increases, stop and call back the parameters;

[0076] Step S2.2: If the text is clustered again, there is already a text extraction rule; calculate the score S according to the following clustering effect evaluation score calculation formula:

[0077]

[0078] Where C represents the number of categories, C2 represents the number of multiple rules that match a single category; R represents the number of existing rules, R2 represents the number of multiple categories that match a single rule; N s represents the total number of small class samples; try multiple sets of parameters; select one of the results and proceed to step S3.

[0079] Step S3: According to the classification results, M sub-datasets with a sample size of n are sampled from the entire text set, one of which is designated as a training set and the rest as test sets; a clustering algorithm plus manual sorting method is used to sort out text extraction rules for the training set, and a code is written to form a text extraction model;

[0080] Step S3.1: Distinguish major categories from minor categories according to the threshold set in step S2; randomly sample M data sets from each major category, and the number of each data set is n*u / U, where U is the number of samples in the entire set; u is the number of samples in the major category; n is a preset integer used to control the size of the test set; the training set is 1, and the test set is M-1;

[0081] Step S3.2: Manually organize text extraction rules for the training set data extracted from each major category and the data from each minor category;

[0082] Step S3.3: If each classification cannot summarize the text extraction rules, after summarizing the preset number of rules, trigger step S2.2 to continue execution and re-cluster;

[0083] Step S3.4: Merge all the data of the major categories into M groups of n data sets, and distinguish between training sets and test sets; for each minor category, select two data, one to be added to the training set and the other to be added to any test set;

[0084] Step S3.5: Write a code model for all the sorted text extraction rules.

[0085] Step S4: label all training set data and test set data; process the training set using the text extraction model; repeatedly adjust the training set and test set according to the processing results, and iterate the text extraction rules;

[0086] Until the accuracy is greater than or equal to the expected threshold;

[0087] Step S4.1: Label all training set data and test set data;

[0088] Step S4.2: Run the text extraction model of step S3 on the training set, and calculate the accuracy based on the annotated data; if the accuracy is less than the threshold, then trigger step S3 to continue execution, and further organize the text extraction rules according to the error content; if the accuracy is greater than or equal to the threshold, then process all test sets; if the accuracy of the test set is less than the threshold, then add the test set to the training set, and re-extract a test set, triggering step S3 to continue execution;

[0089] Until the accuracy on all test sets is greater than or equal to the threshold.

[0090] Step S5: Sample a test set from the entire text set, run the text extraction model and evaluate its results using the confusion matrix, paying attention to the precision and recall rates; if both the precision and recall rates meet the expected thresholds, the text extraction model passes, otherwise continue to iterate the error part.

[0091] The present invention also provides a system for summarizing rules using large-category sampling data and small-category data. The system for summarizing rules using large-category sampling data and small-category data can be implemented by executing the process steps of the method for summarizing rules using large-category sampling data and small-category data, that is, those skilled in the art can understand the method for summarizing rules using large-category sampling data and small-category data as a preferred implementation of the system for summarizing rules using large-category sampling data and small-category data.

[0092] Embodiment 2:

[0093] The present invention also provides a system for summarizing rules using large-category sampling data and small-category data, the system comprising the following modules:

[0094] Module M1: Obtain the historical inquiry data of the platform and the bidding requirement fields released by the purchaser; perform word segmentation, text embedding, and dimensionality reduction operations on the unstructured text set to obtain the low-dimensional feature vector of each text set;

[0095] Module M1.1: Based on the industrial product domain vocabulary, use the Jieba word segmentation tool to segment the text in the training sample; after segmentation, remove meaningless stop words;

[0096] Module M1.2: Use the Word2vec word embedding model to train the word segmentation sequence after word segmentation to obtain the word embedding model in the supply chain field; select the model result dimension, add and normalize all word vectors in the text, and obtain the low-dimensional feature vector of the text.

[0097] Module M2: Cluster the entire text set based on low-dimensional feature vectors; adjust clustering algorithm parameters to optimize the output classification results;

[0098] Module M2.1: If it is the first time to cluster text, set the initial number of clusters or initial clustering conditions, gradually increase the number of clusters or tighten the clustering conditions; the number of texts in each result set gradually decreases; the computer displays the result set, and when similar results are divided into different result sets, or the number of invalid results increases, stop and call back the parameters;

[0099] Module M2.2: If the text is clustered again, there are already text extraction rules at this time; calculate the score S according to the following clustering effect evaluation score calculation formula:

[0100]

[0101] Where C represents the number of categories, C2 represents the number of multiple rules that match a single category; R represents the number of existing rules, R2 represents the number of multiple categories that match a single rule; N s Represents the total number of small class samples; try multiple sets of parameters; select one of the results to enter module M3.

[0102] Module M3: Based on the classification results, M sub-datasets with a sample size of n are sampled from the entire text set, one of which is designated as the training set and the rest as the test set; a clustering algorithm plus a manually organized system is used to organize text extraction rules for the training set, and code is written to form a text extraction model;

[0103] Module M3.1: Distinguish major and minor categories according to the threshold set in module M2; randomly sample M data sets from each major category, and the number of each data set is n*u / U, where U is the number of samples in the whole set; u is the number of samples in the major category; n is a preset integer used to control the size of the test set; the training set is 1, and the test set is M-1;

[0104] Module M3.2: Manually organize text extraction rules for the training set data extracted from each major category and the data from each minor category;

[0105] Module M3.3: If each category cannot summarize the text extraction rules, after summarizing the preset number of rules, the module M2.2 is triggered to continue execution and re-cluster;

[0106] Module M3.4: Merge all the data of major categories into M groups of n data sets, and distinguish between training sets and test sets; for each minor category, select two data, one to be added to the training set and the other to any test set;

[0107] Module M3.5: Write code models for all the sorted text extraction rules.

[0108] Module M4: Label all training set data and test set data; use the text extraction model to process the training set; according to the processing results, repeatedly adjust the training set and test set, and iterate the text extraction rules;

[0109] Until the accuracy is greater than or equal to the expected threshold;

[0110] Module M4.1: Label all training set data and test set data;

[0111] Module M4.2: Call the text extraction model of module M3 on the training set and calculate the accuracy based on the annotated data; if the accuracy is less than the threshold, module M3 is triggered to continue execution and the text extraction rules are further sorted out according to the error content; if the accuracy is greater than or equal to the threshold, all test sets are processed; if the accuracy of the test set is less than the threshold, the test set is added to the training set and a new test set is extracted, triggering module M3 to continue execution;

[0112] Until the accuracy on all test sets is greater than or equal to the threshold.

[0113] Module M5: Sample a test set from the entire text set, run the text extraction model and evaluate its results using the confusion matrix, paying attention to the precision and recall rates; if both the precision and recall rates meet the expected thresholds, the text extraction model passes, otherwise continue to iterate the error part.

[0114] Embodiment 3:

[0115] The present invention clusters the bidding requirement text, and innovatively uses the clustering results to guide the summary of manual text extraction rules, and optimizes the clustering parameters through the feedback of the rule summary; only a relatively small amount of data needs to be annotated for rule testing, and the data that fails the test is re-clustered and the rules are summarized; and repeated iterations are performed to obtain accurate unstructured text extraction rules.

[0116] The present invention provides a method for summarizing rules using large-category sampling data and small-category data, comprising the following steps:

[0117] Step S1: Basic text processing; the model training samples come from the "bidding requirements" field published by the purchaser in the historical inquiry data of the company platform. Perform operations such as word segmentation, text embedding, and dimensionality reduction on the entire set of unstructured text to obtain a low-dimensional feature vector for each text as input for subsequent algorithms.

[0118] Step S2: Text clustering: cluster the text according to the low-dimensional feature vector, adjust the algorithm parameters and optimize the output classification results.

[0119] Step S3: Rule extraction: Based on the classification results, M sub-datasets with a sample size of approximately n are sampled from the entire text set, one of which is designated as the training set and the rest as the test set. The text extraction rules are sorted out for the training set using a clustering algorithm plus manual sorting, and the code is written to form a text extraction model.

[0120] Step S4: Annotation and algorithm iteration: Annotate all training set and test set data. Use the text extraction model to process the training set. According to the processing results, repeatedly adjust the training set and test set, and iterate the text extraction rules until the accuracy is greater than or equal to the expected threshold (90%).

[0121] Step S5: Algorithm test: Sample a test set from the entire text set, run the text extraction model and evaluate its results using the confusion matrix, paying attention to the precision and recall. Repeat the above operation multiple times. If both the precision and recall meet the expected threshold, the text extraction model passes, otherwise continue to iterate the error part.

[0122] Detailed steps:

[0123] The step 1 includes the following steps (or situations):

[0124] Step S1.1: First, based on the industrial product word library accumulated by our company over a long period of time, we use the Jieba word segmentation tool to segment the text in the training sample. After word segmentation, we remove meaningless stop words.

[0125] Step S1.2: Use the Word2vec word embedding model to train the word segmentation sequence after word segmentation to obtain the word embedding model in the supply chain field. Select a lower model result dimension, add and normalize all word vectors in the text, and obtain a low-dimensional feature vector of the text.

[0126] The step 2 includes the following steps (or situations):

[0127] Step S2.1: If it is the first time to cluster text, set a smaller number of clusters (such as k-means) or looser clustering conditions (such as dbscan) at the beginning, and then gradually increase the number of clusters or tighten the clustering conditions. In the practice of this method, dbscan clustering is selected. In this process, the number of texts in each result set will gradually decrease. The computer displays the result set. When obviously similar results are divided into different result sets, or the number of invalid results increases rapidly, stop and call back the parameters in small amounts. This process does not need to be too sophisticated. Use the result set at this time to proceed to the next step.

[0128] Step S2.2: If the subsequent steps return to step 2 for re-clustering, a certain amount of rules already exist. Calculate the score S according to the following formula:

[0129]

[0130] Formula 2-1: Clustering effect evaluation score calculation formula

[0131] The lower the score, the better. C represents the number of categories, C2 represents the number of multiple rules matched by a single category; R represents the number of existing rules, R2 represents the number of multiple categories matched by a single rule; NS represents the total number of small class samples. At this point, the purpose of clustering is to classify texts with the same matching rules into one category in order to optimize existing rules and find new rules; at the same time, the number of samples in each class should be as small as possible so that the rules can be quickly summarized. In the practice of this method, it is considered that less than 20 samples are small categories, and this threshold can be adjusted according to actual conditions. According to the method described in 2.1, try multiple sets of parameters. In the results, select one of them according to the principle of step 2.1 to proceed to the next step.

[0132] The step 3 comprises the following steps:

[0133] Step S3.1: Distinguish major and minor categories according to the threshold set in step 2. Randomly sample M sets of n / U samples from each major category, where U is the total number of samples. There is 1 training set and M-1 test sets.

[0134] Step S3.2: Manually organize text extraction rules for the training set data extracted from each major category and the data from each minor category.

[0135] Step S3.3: If it is difficult to summarize text extraction rules for each of the above categories, such as there are multiple rules for the same category, or the same rule appears in multiple categories, then after summarizing a certain number of rules (such as one for each category), return to step 2.2 and perform clustering again.

[0136] Step S3.4: Merge all the data of the major categories into M groups of n data sets, and distinguish between training sets and test sets. For each minor category, select two data, one to be added to the training set and the other to any test set.

[0137] Step S3.5: Write a code model for all the sorted text extraction rules.

[0138] The step 4 comprises the following steps:

[0139] Step S4.1: Label all training set and test set data.

[0140] Step S4.2: Run the text extraction model of step 3 on the training set and calculate the accuracy based on the annotated data. If the accuracy is less than the threshold (set to 90% in practice), return to step 3 and further organize the text extraction rules based on the error content.

[0141] Step S4.3: If the accuracy is greater than or equal to the threshold, all test sets are processed. If the accuracy of the test set is less than the threshold, the test set is added to the training set, and a new test set is extracted, and steps 3 and 4 are repeated until the accuracy is greater than or equal to the threshold on all test sets.

[0142] Those skilled in the art may understand this embodiment as a more specific description of Embodiment 1 and Embodiment 2.

[0143] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0144] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A method for summarizing rules using large-category sampling data and small-category data, characterized in that: The method comprises the following steps: Step S1: Obtain the historical inquiry data of the platform and the bidding requirement fields issued by the purchaser, and perform preprocessing to obtain the full text set; Step S2: Sample M sub-datasets with a sample size of n from the entire text set, one of which is a training set and the rest are test sets, and cluster the texts; Step S3: label all training set data and test set data; process the training set using the text extraction model; repeatedly adjust the training set and test set according to the processing results, and iterate the text extraction rules; Step S2.1: If it is the first time to cluster text, set the initial number of clusters or initial clustering conditions, gradually increase the number of clusters or tighten the clustering conditions; the number of texts in each result set gradually decreases; the computer displays the result set, and when similar results are divided into different result sets, or the number of invalid results increases, stop and call back the parameters; Step S2.2: If the text is clustered again, there is already a text extraction rule; calculate the score S according to the following clustering effect evaluation score calculation formula: Where C represents the number of categories, C2 represents the number of multiple rules that match a single category; R represents the number of existing rules, R2 represents the number of multiple categories that match a single rule; N s Represents the total sample size of small categories; Try multiple sets of parameters; select one of the results and proceed to step S3.

2. The method for summarizing rules using large-category sampling data and small-category data according to claim 1, characterized in that: The step S1 performs word segmentation, text embedding and dimensionality reduction operations on the entire set of unstructured text to obtain a low-dimensional feature vector for each text; uses the jieba word segmentation tool to segment the text and remove stop words; uses the Word2vec word embedding model to train the sequence after word segmentation to obtain a supply chain field word embedding model; sums and normalizes all word vectors in the text to obtain a low-dimensional feature vector for the text.

3. The method for summarizing rules using large-category sampling data and small-category data according to claim 1, characterized in that: The step S2 manually organizes the training set data, summarizes the text extraction rules, and writes code to form a text extraction model; if the classification cannot summarize the rules, clustering is performed by adjusting the clustering parameters until each classification can summarize at least one rule; for large-category data, multiple sets of data sets are randomly sampled; for small-category data, at least two data of each small category are used for training and testing.

4. The method for summarizing rules using large-category sampling data and small-category data according to claim 1, characterized in that: The step S3 uses the text extraction model to process the training set and calculates the accuracy based on the labeled data; the training set and the test set are repeatedly adjusted according to the processing results, and the text extraction rules are iteratively optimized until the accuracy on all test sets reaches or exceeds the expected threshold.

5. A system for summarizing rules using large-category sampling data and small-category data, characterized in that: The system includes the following modules: Module M1: Obtain the historical inquiry data of the platform and the bidding requirement fields issued by the purchaser, and perform preprocessing to obtain the full text set; Module M2: Sample M sub-datasets of n sample sizes from the entire text set, one of which is a training set and the rest are test sets, and cluster the texts; Module M3: Label all training set data and test set data; use the text extraction model to process the training set; according to the processing results, repeatedly adjust the training set and test set, and iterate the text extraction rules; Module M2.1: If it is the first time to cluster text, set the initial number of clusters or initial clustering conditions, gradually increase the number of clusters or tighten the clustering conditions; the number of texts in each result set gradually decreases; the computer displays the result set, and when similar results are divided into different result sets, or the number of invalid results increases, stop and call back the parameters; Module M2.2: If the text is clustered again, there are already text extraction rules at this time; calculate the score S according to the following clustering effect evaluation score calculation formula: Where C represents the number of categories, C2 represents the number of multiple rules that match a single category; R represents the number of existing rules, R2 represents the number of multiple categories that match a single rule; N s Represents the total sample size of small categories; Try multiple sets of parameters; select one of the results and enter module M3.

6. The system for summarizing rules using large-category sampling data and small-category data according to claim 5, characterized in that: The module M1 performs word segmentation, text embedding and dimensionality reduction operations on the entire set of unstructured text to obtain a low-dimensional feature vector for each text; uses the jieba word segmentation tool to segment the text and remove stop words; uses the Word2vec word embedding model to train the sequence after word segmentation to obtain a word embedding model in the supply chain field; sums and normalizes all word vectors in the text to obtain a low-dimensional feature vector for the text.

7. The system for summarizing rules using large-category sampling data and small-category data according to claim 5, characterized in that: The module M2 manually organizes the training set data, summarizes the text extraction rules, and writes code to form a text extraction model; if the classification cannot summarize the rules, clustering is performed by adjusting the clustering parameters until each classification can summarize at least one rule; for large-category data, multiple sets of data sets are randomly sampled; for small-category data, at least two data of each small category are used for training and testing.

8. The system for summarizing rules using large-category sampling data and small-category data according to claim 5, characterized in that: The module M3 uses the text extraction model to process the training set and calculates the accuracy based on the labeled data; repeatedly adjusts the training set and the test set based on the processing results, and iteratively optimizes the text extraction rules until the accuracy on all test sets reaches or exceeds the expected threshold.

Images