Material master data intelligent classification method

Abstract

The application discloses a kind of material master data intelligent classification methods, comprising: pre-acquiring historical material description data of labeled material type, constitute training set and test set, text vector is generated for classification model training to material description, according to test performance determines benchmark confidence threshold;Method further comprises: according to the material description text of real-time received to be classified, generate to be predicted vector, based on classification model, the material type of predicted and corresponding prediction confidence are obtained;Based on the performance index parameter of multiple classification prediction in historical time sliding window, benchmark confidence threshold is adjusted, to compare with the prediction confidence of real-time prediction, determine prediction result.The application adjusts confidence threshold based on the performance index in historical time sliding window dynamically, so that classification system can adapt to business data state change, intelligently allocate artificial review resources under the premise of guaranteeing accuracy, realize the balance of efficiency and precision.

Description

Technical Field

[0001] This invention belongs to the field of data classification technology, specifically relating to an intelligent classification method for material master data. Background Technology

[0002] In enterprise information management systems, material master data, as the core foundational data describing the basic attributes of materials (such as name, specifications, model, and classification), serves as the hub connecting business processes such as procurement, production, warehousing, and finance. Its quality directly impacts supply chain efficiency, cost accounting accuracy, and the reliability of business decisions. Material classification, in particular, aims to ensure consistency in the understanding and definition of the same material across different business systems through a standardized classification system, thereby supporting data mining, intelligent analysis, and precise decision-making.

[0003] As enterprises deepen their digital transformation, the scale of material data is growing exponentially, and its descriptive text is becoming increasingly diverse and non-standardized. Against this backdrop, material classification technology itself has undergone a series of iterations.

[0004] Early solutions primarily relied on keyword-based rule matching, where domain experts predefined a large number of mapping rules between keywords and categories. While intuitive, this approach was costly to maintain and struggled to handle linguistic complexities such as synonyms, abbreviations, and colloquial expressions, resulting in poor generalization capabilities.

[0005] Subsequently, semi-automated classification based on expert experience emerged, which improved efficiency to some extent. However, its classification quality heavily relied on the knowledge and experience of specific business personnel, resulting in problems such as strong subjectivity, difficulty in scaling, and lag in response to emerging material types.

[0006] Although existing technologies have attempted to incorporate machine learning methods, using natural language processing (NLP) techniques to convert material text into vectors for automatic classification, this has improved the level of automation in the processing to some extent.

[0007] However, the distribution of material data is not static. In actual business operations, there are dynamic challenges such as noise fluctuations caused by incomplete historical data cleaning, semantic space drift caused by new material descriptions, and inconsistent data entry standards among different business departments. This can lead to a change in the distribution of prediction confidence of a model that performs well on the test set when faced with real-time changing business data streams.

[0008] If the preset parameters are continued to be used, it will inevitably lead to either increased unnecessary manual review costs or poor data quality that pollutes the main database, ultimately making it difficult for the overall efficiency and accuracy of the automatic classification system to meet the requirements of enterprise data governance.

[0009] Therefore, there is an urgent need for a material master data classification method to fundamentally improve the robustness and practicality of intelligent classification systems in real industrial scenarios. Summary of the Invention

[0010] This invention provides an intelligent classification method for material master data. By introducing a dynamic confidence threshold adjustment mechanism based on historical performance feedback, it can solve the problem that existing static machine learning classification methods cannot adapt to dynamic changes in business data, resulting in a difficulty in balancing classification efficiency and accuracy.

[0011] The technical solution adopted in this invention is as follows: A method for intelligent classification of material master data, characterized by comprising: Pre-acquire historical material description data with labeled material types to form training and testing sets. Generate text vectors from the material descriptions for classification model training. Determine the baseline confidence threshold based on test performance. The method further includes: Based on the real-time received material description text to be classified, a prediction vector is generated. Based on the classification model, the predicted material type and the corresponding prediction confidence are obtained. Based on the performance metrics of multiple classification predictions within a historical time sliding window, the baseline confidence threshold is adjusted to compare with the prediction confidence of real-time predictions to determine the prediction result.

[0012] Before generating text vectors from the material description, the process also includes text cleaning of the material description, specifically: Use regular expressions to remove punctuation marks and special characters; Filter words from the stop word list; And remove numeric strings that do not carry classification information.

[0013] Generate text vectors for material descriptions, specifically: Use a Chinese word segmentation tool to segment the cleaned text; The Word2Vec model is used to map segmented words into word vectors; Text vectors are generated based on dynamic clustering grouping.

[0014] Text vectors are generated based on dynamic clustering and grouping, specifically as follows: Based on the number and distribution of word vectors after word segmentation, the number of clusters is dynamically determined to cluster the word vectors and form multiple cluster groups; The average vector within each cluster is obtained by taking the arithmetic mean of the word vectors within each cluster group. Based on the cosine similarity between the average vector within the group and the centroids of all word vectors, the average vectors within the group are sorted and concatenated in descending order of similarity, and then projected through a fully connected neural network layer to generate the text vector.

[0015] Classification model training includes: Multiple different types of machine learning algorithms are trained in parallel using the training set; The hyperparameters of each algorithm are optimized using grid search and cross-validation; Based on the evaluation metrics on the test set, a classification model is selected from multiple optimized algorithms; The machine learning algorithm mentioned above includes at least one of random forest, decision tree and XGBoost.

[0016] The baseline confidence threshold is determined based on test performance, specifically as follows: Obtain the confidence scores of each sample output by the model when making predictions on the test set; Based on the preset initial target pass rate, the corresponding quantiles are determined on the distribution curve of the confidence scores of each sample. The confidence value of this quantile is set as the baseline confidence threshold.

[0017] Based on the performance metrics of multiple classification predictions within a historical time sliding window, the baseline confidence threshold is adjusted as follows: Obtain the actual accuracy of manually verified prediction results within a historical time sliding window; Calculate the average confidence score of multiple classification predictions within a historical time sliding window; Obtain the average text cleaning intensity of material descriptions from multiple classification predictions within a historical time sliding window; Based on the actual accuracy, average confidence level, average text cleaning intensity, and preset target automatic pass rate, the current confidence level threshold is dynamically adjusted using a predefined adjustment function.

[0018] The current confidence threshold is dynamically adjusted as follows: When the actual accuracy rate is lower than the pre-stored benchmark accuracy rate, the current confidence threshold is increased. When the average text cleaning intensity is higher than the pre-stored baseline cleaning intensity, the current confidence threshold is lowered. When the target automatic pass rate is increased, the current confidence threshold is lowered.

[0019] After determining the prediction results, the following also includes: If the predicted confidence level is greater than or equal to the current confidence level threshold, the predicted material type will be automatically written into the material master database. If the predicted confidence level is less than the current confidence level threshold, the material description text, the predicted material type, and the predicted confidence level are pushed to the manual review queue.

[0020] The present invention also provides a material master data intelligent classification system, including at least one processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method described herein is implemented.

[0021] Due to the adoption of the above technical solution, the beneficial effects achieved by this invention are as follows: 1. In this invention, after the model goes live, the baseline confidence threshold is dynamically adjusted based on recent prediction performance parameters within a historical time sliding window, enabling the classification decision boundary to adapt to the real-time state of the business data. This means that when data quality is high and the model has a high degree of confidence, the system can automatically lower the threshold to improve automation; when data noise is high and model performance is unstable, the system automatically raises the threshold to ensure accuracy. This self-adjusting capability effectively overcomes the rigidity of fixed threshold strategies, allowing the system to continuously maintain optimal classification performance.

[0022] This method achieves an intelligent resource allocation strategy by dynamically adjusting thresholds. Based on recent overall performance, it assesses the risk of automated decision-making at the current confidence level. When overall performance is excellent, the scope of automated processing is expanded to automatically classify more boundary cases, significantly reducing unnecessary manual intervention and lowering operating costs. Conversely, when performance fluctuates, more uncertain classifications are delegated to experts, ensuring the accuracy of the input data and preventing contamination by erroneous data. Thus, while ensuring classification accuracy, it effectively reduces the cost of manual review, achieving an intelligent balance between efficiency and accuracy. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart illustrating the intelligent classification method for material master data according to one embodiment of the present invention. Detailed Implementation

[0024] To more clearly illustrate the overall concept of the present invention, a detailed description will be provided below with reference to the accompanying drawings and examples.

[0025] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0026] like Figure 1 As shown, a method for intelligent classification of material master data includes: S100: Pre-acquire historical material description data of labeled material types to form training and testing sets, generate text vectors for material descriptions for classification model training, and determine the baseline confidence threshold based on test performance.

[0027] The steps described above form the foundation and starting point for building the entire intelligent classification system. Specifically, First, historical material master data records are collected from multiple heterogeneous business systems within the enterprise (such as ERP, SCM, MES, etc.). The material description text in these records (e.g., "304 stainless steel internal hex bolt M6*20") constitutes the initial corpus. To ensure the effectiveness of subsequent model training, data with standard material types (e.g., "fasteners - bolts") that have been reviewed and labeled by domain experts must be obtained.

[0028] Next, the classification model training phase begins. The labeled and vectorized complete dataset is randomly divided into training and test sets according to a preset ratio (e.g., 8:2). Understandably, the training set is used to train the machine learning classifier. After the model training is complete, a comprehensive evaluation is performed on the test set to determine or optimize the classification model used for subsequent predictions.

[0029] Based on the established classification model, the baseline confidence threshold is determined. All samples in the test set are input into the model for prediction, and the model outputs the probability value (i.e., confidence score) for each sample corresponding to its predicted class. Subsequently, the baseline confidence threshold is determined based on the test results. This threshold serves as the initial standard for judging the reliability of the prediction results and whether manual review is necessary during the initial online deployment of the model, providing a performance benchmark and starting point for dynamic adjustments in the subsequent online prediction phase.

[0030] Through the above steps, not only was a high-performance classification model constructed, but a scientific and reasonable initial decision boundary was also established based on offline test data for adaptive optimization in real business environments.

[0031] The method further includes: S200: Based on the real-time received material description text to be classified, generate a vector to be predicted, and based on the classification model, obtain the predicted material type and the corresponding prediction confidence.

[0032] The purpose of this step is to apply the capabilities of the model built during the training phase to actual business data flows, enabling real-time, automated type identification and reliability assessment of new or unclassified materials.

[0033] First, the system receives real-time descriptions of materials to be classified from various business systems via a data interface. It should be noted that, to ensure consistency between online processing and offline training, and to avoid performance degradation due to differences in data processing, the preprocessing procedure for the text to be classified must be strictly consistent with the preprocessing procedure for the training data.

[0034] Next, the vectorization process begins. The word segmentation model and word vector model, which were persisted during the training phase, are loaded. The cleaned text is segmented using this model, ensuring that the segmentation criteria are the same as the training corpus. Then, for each segmented word, its corresponding word vector is obtained from the pre-trained word vector model. Finally, the text vector generation method used in the training phase is strictly followed to generate text vectors. This series of operations ensures that the online feature extraction and the offline model training reside in the exact same feature space, laying the foundation for accurate prediction.

[0035] After obtaining the vector to be predicted, it is input into a pre-trained and optimized classification model (such as a random forest model) that has been loaded into memory. Upon receiving the input, the model performs forward inference and outputs a probability distribution vector, where each dimension of the vector represents the probability that the input sample belongs to a certain predefined material type.

[0036] The system iterates through the probability distribution vector, identifies the category with the highest probability value, and uses it as the predicted material type (e.g., the label corresponding to the highest probability is "fasteners-bolts"). Simultaneously, the system records this highest probability value as the prediction confidence level for this prediction. This confidence level directly reflects the classification model's confidence in the current prediction result; a higher value indicates greater confidence in the model's judgment.

[0037] This step automates the process from raw text input to structured prediction results and reliability assessment. This process not only efficiently outputs classification conclusions, but more importantly, it provides indispensable data for subsequent confidence-based intelligent decision-making, achieving intelligent and adaptive optimization.

[0038] S300: Based on the performance index parameters of multiple classification predictions within a historical time sliding window, the baseline confidence threshold is adjusted to compare with the prediction confidence of real-time prediction to determine the prediction result.

[0039] The purpose of this step is to break through the limitations of traditional fixed confidence thresholds and enable the decision boundary to dynamically respond to the system's recent real performance and data status through a closed-loop feedback mechanism, thereby continuously optimizing the efficiency and accuracy of classification in a changing business environment.

[0040] It's important to note that a continuously running monitoring and data collection module is required. This module maintains a historical sliding window (e.g., saving the 10,000 most recently processed prediction records, or all prediction data from the last 24 hours). Within this window, three types of key data are collected and correlated: first, the prediction confidence score for each record; second, the final correct classification result after manual verification (used to calculate the actual accuracy); and third, the text cleaning intensity metric for processing that record (e.g., the proportion of removed characters to the original text length, serving as a proxy indicator of data noise level).

[0041] Based on the data accumulated within the sliding window, a dynamic threshold adjustment calculation is performed periodically (e.g., after processing 500 new data points, or every hour). Specifically, a multi-parameter adjustment function is constructed. This function considers at least the following inputs: Actual accuracy, within the sliding window, is the proportion of records whose model predictions match the final human judgment among all records that have undergone manual review. This directly reflects the model's recent real performance. Average confidence score, the arithmetic mean of the confidence scores of all predicted records within the sliding window, reflects the model's overall confidence in its recent predictions; Average text cleaning intensity is the average text cleaning intensity of all records within the sliding window, which indirectly reflects the degree of normalization and noise level of recently processed data. The preset target automatic pass rate is the expected automated processing ratio set at the business level.

[0042] The adjustment function outputs a new confidence threshold based on the above parameters and according to a predetermined strategy logic. After calculating the new dynamic threshold, it is updated to the current confidence threshold for subsequent real-time prediction decisions.

[0043] For each newly arrived material to be classified, after obtaining the prediction result and confidence level, the real-time predicted confidence level is immediately compared with this dynamically adjusted current confidence level threshold. Branch logic is executed based on the comparison result.

[0044] Through the above steps, the system can optimize its decision-making criteria in real time based on actual performance. This not only improves the robustness and practicality of the system in long-term operation, but also achieves a dynamic balance between classification efficiency and accuracy, significantly reducing the system's reliance on manual parameter tuning and overall operation and maintenance costs.

[0045] As a preferred embodiment of the present invention, before generating a text vector from the material description, text cleaning of the material description is further included, specifically: Using regular expressions to remove punctuation marks and special characters; Filtering words in the stop word list; And removing numerical strings that do not carry classification information.

[0046] Before generating a text vector from the material description, a text cleaning step is included. The core purpose of this step is to systematically eliminate noise and non-semantic information in the original material description text, thereby improving the purity and semantic discrimination of subsequent text vector representations, providing high-quality feature inputs for machine learning models, and ultimately ensuring the accuracy and robustness of the classification task.

[0047] First, use regular expressions to remove punctuation marks and special characters. The system predefines a comprehensive regular expression pattern, which is designed to match and remove various types of noise characters. This includes but is not limited to: All Chinese and English punctuation marks, such as full stops, commas, quotation marks, brackets, etc.; Various special characters, such as #, &, *, $, %, etc., which may be garbled or irrelevant symbols generated during system transmission or input; Non-printable characters and redundant whitespace characters, such as consecutive spaces, tab characters, line break characters, etc.

[0048] Through this step, an original description such as "[High-quality] 304 stainless steel bolt, M6*20;" can be normalized to "High-quality 304 stainless steel bolt M6*20", effectively removing formatting symbols that interfere with semantic understanding.

[0049] Secondly, filter words in the stop word list. The system maintains a Chinese stop word list optimized for the material master data field. This word list not only includes common meaningless function words such as "of", "already", "in", "and", "with", "and", etc., but also includes common words that usually do not carry classification information in this context according to the characteristics of the material description, such as some verbs ("for", "suitable") or overly broad nouns ("product", "goods", "fitting").

[0050] During cleaning, the tokenized word sequence is compared with this stop word list, and all matching words are removed. For example, processing "bolt of 304 stainless steel for equipment" to "equipment 304 stainless steel bolt", thereby focusing on core entities and attribute words.

[0051] Finally, remove numerical strings that do not carry classification information. This step is a specific key strategy for material data cleaning. The system makes a judgment through a rule engine: Recognize all strings of numbers, letters, and mixed numbers in a text. Based on predefined material description patterns, semantically meaningful classification information numbers are distinguished from meaningless identifier numbers. Classification information numbers are typically related to specifications, models, material codes, etc., and often adjacent to specific units or keywords, such as "6" and "20" in "M6*20" or "304" in "304 stainless steel." These numbers will be retained. Numerical strings that do not carry classification information mainly refer to pure serial numbers, batch numbers, internal product codes, etc., such as "123456" and "BATCH001". These numerical strings do not contribute to general material classification in the absence of specific context and may introduce noise, so they are systematically removed.

[0052] By implementing the above text cleaning strategy, it is ensured that the text input to the vectorization module is a highly refined and semantically rich sequence of words. This not only significantly reduces the dimensionality and noise of the feature space and improves the efficiency of model training and prediction, but more importantly, by eliminating irrelevant variations, it enables the model to more accurately learn and capture the key semantic features that determine material classification, thus laying a data foundation for the high-performance operation of the entire intelligent classification system.

[0053] In a preferred embodiment of the present invention, the text vector for the material description is generated as follows: Use a Chinese word segmentation tool to segment the cleaned text; The Word2Vec model is used to map segmented words into word vectors; Text vectors are generated based on dynamic clustering grouping.

[0054] The core objective of this implementation is to address the semantic information loss that may result from simple arithmetic average pooling. By semantically grouping and reorganizing the words in the material description, an enhanced text vector representation that better preserves the local semantic structure and global semantic composition is generated.

[0055] First, after segmenting the cleaned text using a Chinese word segmentation tool (such as Jieba) and mapping each word to a word vector using a pre-trained Word2Vec model, the system obtains a set of word vectors.

[0056] Subsequently, a dynamic clustering grouping operation is performed, specifically as follows: Based on the number and distribution of word vectors after word segmentation, the number of clusters is dynamically determined to cluster the word vectors and form multiple cluster groups; The average vector within each cluster is obtained by taking the arithmetic mean of the word vectors within each cluster group. Based on the cosine similarity between the average vector within the group and the centroids of all word vectors, the average vectors within the group are sorted and concatenated in descending order of similarity, and then projected through a fully connected neural network layer to generate the text vector.

[0057] Specifically, instead of using a fixed number of clusters, the value of k is dynamically calculated based on the number of words (n) after segmentation of the current material description. The strategy adopted is: k = max(2, min(ceil(n / 2), 5)). This ensures that the number of clusters is at least 2 (to distinguish different semantic segments) and at most 5 (to prevent overfitting and control computational complexity), and the number of groups is appropriately increased as the length of the descriptive text increases, reflecting the adaptive ability to text complexity.

[0058] Then, cluster analysis is performed, using the K-means++ algorithm to cluster all word vectors of the current description, forming k cluster groups. The initialization strategy of K-means++ helps to avoid the local optima problem that the ordinary K-means algorithm may get stuck in, thus obtaining more reasonable semantic grouping results.

[0059] For example, the word vectors of the description "304 stainless steel hexagon socket head cap screw" may be clustered into two groups: one group contains "304" and "stainless steel" (characterizing the material), and the other group contains "hexagon socket head cap screw" and "bolt" (characterizing the shape and category).

[0060] To generate intra-group average vectors, for each group obtained after clustering, the arithmetic mean of all word vectors within that group is calculated, resulting in k intra-group average vectors. Each intra-group average vector represents the core information of that semantic group, and its calculation process suppresses intra-group noise to a certain extent.

[0061] Furthermore, inter-group vector sorting and concatenation are performed to construct an ordered and information-rich global representation. All intra-group average vectors are sorted according to predetermined rules. Specifically, the cosine similarity between each intra-group average vector and the global centroid vector of the entire word vector set is calculated, and the intra-group average vectors are sorted in descending order of similarity. Therefore, vectors closer to the global semantic center may contain more general and core category information; placing them at the beginning of the vector representation helps the model prioritize capturing key features. After sorting, these k intra-group average vectors are concatenated sequentially into a longer joint vector.

[0062] It should be noted that because the number of clusters k is dynamically changing, the dimension of the concatenated joint vector (k * d, where d is the word vector dimension) is not fixed. To meet the requirement of the downstream classification model for fixed-dimensional input, this variable-length joint vector is fed into a pre-trained fully connected neural network layer (i.e., a linear projection layer). This layer projects the variable-length input into a fixed-dimensional output space (e.g., 200 dimensions), thereby generating the final standardized, fixed-dimensional semantically enhanced global text vector.

[0063] Through the series of operations described above, including dynamic clustering, group averaging, sorting and concatenation, and dimensional projection, the text vectors generated by this invention not only integrate global information but also preserve and enhance the local semantic structure within the text through clustering. This method can more precisely express the semantic composition of material descriptions, and is particularly suitable for short texts containing multiple attributes (such as material, specifications, and shape), providing higher-quality feature inputs to improve the discriminative ability of classification models and ensuring high-precision classification.

[0064] As a preferred embodiment of the present invention, the classification model training includes: Multiple different types of machine learning algorithms are trained in parallel using the training set; The hyperparameters of each algorithm are optimized using grid search and cross-validation; Based on the evaluation metrics on the test set, a classification model is selected from multiple optimized algorithms; The machine learning algorithm mentioned above includes at least one of random forest, decision tree and XGBoost.

[0065] The core objective of this implementation method is to overcome the performance limitations and insufficient generalization ability that may exist in a single model. Through a complete process of parallel training, fine-tuning, and scientific evaluation, it ensures that the final deployed model achieves optimal performance in terms of accuracy, stability, and generalization ability.

[0066] Perform parallel training of multiple algorithms. Utilize the text vector features of the training set and the corresponding material type labels to simultaneously launch training tasks for multiple different types of machine learning classification algorithms.

[0067] These algorithms include at least Random Forest, Decision Tree, and XGBoost. These algorithms were chosen based on their complementary characteristics: Random Forest improves the model's generalization ability and robustness by integrating multiple decision trees and employing a bagging strategy, effectively preventing overfitting; Decision Tree models have an intuitive structure, facilitating the understanding of feature importance and requiring fewer assumptions about data distribution; XGBoost uses a gradient boosting framework, iteratively training a series of weak learners and focusing on previously misclassified samples, efficiently capturing complex nonlinear relationships and feature interactions. This parallel and diverse training strategy aims to provide a rich pool of candidate models with different strengths for subsequent optimization.

[0068] Based on parallel training, the hyperparameters of each candidate algorithm are automated and verified.

[0069] Specifically, a strategy combining grid search and k-fold cross-validation is adopted. The system predefines a hyperparameter space to be explored for each algorithm. For example, for Random Forest, the parameter space may include the number of decision trees (e.g., values ​​[100, 200, 300]) and the maximum depth of the trees (e.g., values ​​[10, 20, None]); for XGBoost, it may include the learning rate learning_rat (e.g., values ​​[0.01, 0.1, 0.2]) and the maximum depth of the trees (e.g., values ​​[3, 6, 9]).

[0070] The grid search traverses all combinations of the parameter space. For each hyperparameter combination, the system uses k-fold cross-validation (usually k=5 or 10) to evaluate on the training set. That is, the training set is divided into k parts each time, and k-1 parts are used for training in turn, while the remaining part is used for validation. This is repeated k times, and the average performance metric is taken.

[0071] The optimization objective in this process is typically set to maximize the F1 score, as it balances precision and recall. Through this strategy, the system can find the optimal hyperparameter configuration for each candidate algorithm within its parameter space, effectively avoiding overfitting on the training set and ensuring the robustness of the optimized model.

[0072] Finally, the model selection and determination phase begins. Each candidate model, after hyperparameter optimization, undergoes final performance evaluation on an independent test set that was not used for training or tuning. The system employs a comprehensive evaluation metric system, which includes at least accuracy, precision, recall, and F1 score. The system calculates the specific values ​​for each candidate model on these metrics and performs cross-sectional comparative analysis.

[0073] The selection strategy does not rely solely on a single metric but rather makes a comprehensive judgment: priority is given to models that perform best on core business metrics (such as F1 score) and are balanced across other metrics; if multiple models have similar performance, their computational efficiency, complexity, and interpretability can be further examined. Ultimately, the system selects the model with the best overall performance as the classification model to be used online and serializes and saves it.

[0074] The above implementation methods ensure that the constructed classification model is the optimal solution after thorough comparison and verification, rather than relying on the default performance of a single model or empirical selection. This systematic training and optimization process greatly improves the reliability, robustness, and technological advancement of the final classification solution, providing a core driving force for high-precision intelligent classification of material master data.

[0075] In a preferred embodiment of the present invention, the baseline confidence threshold is determined based on the test performance, specifically as follows: Obtain the confidence scores of each sample output by the model when making predictions on the test set; Based on the preset initial target pass rate, the corresponding quantiles are determined on the distribution curve of the confidence scores of each sample. The confidence value of this quantile is set as the baseline confidence threshold.

[0076] The core objective of this implementation method is to provide a scientific, quantifiable, and business-aligned initial judgment standard for classification decisions in the early stages of model deployment, rather than relying on arbitrary settings based on empiricism, thereby establishing a reliable performance benchmark for subsequent dynamic adjustments.

[0077] After the optimal classification model has been trained and validated, the system uses the model to make a comprehensive prediction on the complete test set (i.e., a subset of data with real labels that has not been used during the entire training phase).

[0078] In this process, the system not only records the predicted class of each test sample, but more importantly, it collects the probability value of the model output for each sample corresponding to its predicted class, i.e., the confidence score of each sample. This step ensures that the collected confidence score data can truly reflect the model's performance on data with known performance levels, providing a reliable data foundation for threshold determination.

[0079] Next, the confidence values ​​of all test samples are sorted in ascending order, and their empirical distribution function curves are plotted. Based on this, an initial target automatic pass rate is preset according to business needs. This parameter is a quantitative indicator with clear business significance, representing what percentage of prediction results are expected to be accepted by the system directly without manual review in the early stages of model deployment. For example, the business side may set an initial target automatic pass rate of 95% based on a balance between cost and efficiency.

[0080] The system then finds the quantile corresponding to (1 - initial target auto-pass rate) on this confidence distribution curve. Specifically, if the target auto-pass rate is 95%, the system searches for the 95% quantile of this distribution curve. The specific confidence value corresponding to this quantile means that 95% of the samples in the test set have a prediction confidence level lower than or equal to this value. From a business perspective, setting this value as a threshold allows the model to be expected to achieve an auto-classification rate of approximately 95% at the test set performance level.

[0081] Finally, the system officially sets the calculated 95% quantile confidence value as the system's baseline confidence threshold.

[0082] By employing the aforementioned quantification method based on the test set confidence distribution and the pass rate of business objectives, the benchmark threshold established in this implementation combines technical rationality with business orientation. It ensures that, during initial deployment, the scale of the model's automatic classification matches the reliability demonstrated during the testing phase, providing a stable and reliable starting benchmark for the system.

[0083] In a preferred embodiment of the present invention, the baseline confidence threshold is adjusted based on the performance index parameters of multiple classification predictions within a historical time sliding window, specifically as follows: Obtain the actual accuracy of manually verified prediction results within a historical time sliding window; Calculate the average confidence score of multiple classification predictions within a historical time sliding window; Obtain the average text cleaning intensity of material descriptions from multiple classification predictions within a historical time sliding window; Based on the actual accuracy, average confidence level, average text cleaning intensity, and preset target automatic pass rate, the current confidence level threshold is dynamically adjusted using a predefined adjustment function.

[0084] The purpose of this implementation is to establish a dynamic feedback control system that can sense the system's operating status, assess changes in data quality, and automatically optimize decision criteria accordingly, thereby solving the key problem that fixed thresholds cannot adapt to dynamic changes in business data.

[0085] First, the system establishes a continuously running monitoring and data collection mechanism. This mechanism maintains a historical time sliding window (e.g., saving the most recent 24 hours or the most recent 10,000 prediction records). Within this window, the system continuously collects and correlates four types of key data: It's important to note that a continuously running monitoring and data collection module is required. This module maintains a historical time-sliding window (e.g., saving the 10,000 most recently processed prediction records, or all prediction data from the last 24 hours). Within this window, key data is collected and correlated: actual accuracy, average confidence level, average text cleaning intensity, and the set target automatic pass rate.

[0086] Specifically, dynamically adjusting the current confidence threshold involves: When the actual accuracy rate is lower than the pre-stored benchmark accuracy rate, the current confidence threshold is increased. When the average text cleaning intensity is higher than the pre-stored baseline cleaning intensity, the current confidence threshold is lowered. When the target automatic pass rate is increased, the current confidence threshold is lowered.

[0087] When the actual accuracy rate is consistently lower than the pre-stored baseline accuracy rate (from test set performance), it indicates that the reliability of the model's recent output has decreased. The system should adopt a conservative strategy, namely, raising the confidence threshold, at the cost of sacrificing some automation rate, to ensure the accuracy of the data entering the database.

[0088] When the average text cleaning intensity is significantly higher than the baseline level, it indicates that the recent data is quite noisy. When processing such data, the absolute confidence value of the model's output may be systematically low. If the original threshold is maintained, a large number of relatively reliable predictions will be sent for manual review. Therefore, the system should adopt an adaptation strategy, that is, lower the confidence threshold to match the characteristics of the current data.

[0089] When business needs change and the target automatic pass rate is proactively increased, the system must prioritize efficiency requirements. The strategy is to lower the confidence threshold and expand the scope of automatic processing. Simultaneously, the system will closely monitor the impact of this strategy on the actual accuracy rate.

[0090] Finally, the system updates the new threshold calculated by the adjustment function to the current confidence threshold and immediately applies it to subsequent real-time prediction decisions.

[0091] The above implementation method achieves dynamic self-optimization. By sensing the system state through multiple parameters and adjusting itself based on explicit business logic, the classification system can maintain optimal performance in changing business environments, effectively balancing classification accuracy, automated processing efficiency, and manual review costs, and greatly improving the system's practicality and robustness in long-term operation.

[0092] In a preferred embodiment of the present invention, after determining the prediction result, the method further includes: If the predicted confidence level is greater than or equal to the current confidence level threshold, the predicted material type will be automatically written into the material master database. If the predicted confidence level is less than the current confidence level threshold, the material description text, the predicted material type, and the predicted confidence level are pushed to the manual review queue.

[0093] The core objective of this implementation method is to construct a human-machine collaborative, resource-adaptive classification decision-making and execution mechanism. During automated classification, it aims to improve processing efficiency and reduce labor costs without sacrificing the accuracy of master data.

[0094] The system compares the real-time prediction confidence level with the dynamically adjusted current confidence threshold in real time. This is a binary decision logic. Based on the comparison result, the system automatically enters two distinct processing branches, thereby achieving separation of classification decisions.

[0095] A high-confidence branch is defined as a branch where the predicted confidence level is greater than or equal to the current confidence threshold. The system then determines that the classification result is reliable and proceeds with a fully automated processing flow.

[0096] The system first converts the numerical or code-based classification labels output by the model into standard material type names readable by the business system by querying a pre-set label-material type mapping table (e.g., converting the label "BOLT_001" into "Fasteners-Bolts").

[0097] Subsequently, the system generates standardized database operation instructions, which automatically write the converted standard material type, the original material description text, and other necessary metadata into the corresponding fields of the enterprise's material master database within a single database transaction.

[0098] This approach ensures that high-quality classification results can be seamlessly, accurately, and efficiently integrated into the company's core data assets without any manual intervention, greatly improving data entry efficiency.

[0099] The low-confidence branch, i.e., the predicted confidence level is lower than the current confidence threshold, indicates that the system determines the classification result has high uncertainty and automatically triggers a manual review mechanism.

[0100] The system will create a structured review task object, which will contain at least the following key information: the material description text to be reviewed, the material type predicted by the model, and the prediction confidence level as an important reference.

[0101] This system does not simply record data; instead, it inserts such review tasks into a centrally managed queue of manual review tasks based on priority rules (such as confidence level from low to high, or task creation time). This queue can be accessed concurrently by multiple business personnel to achieve load balancing.

[0102] Business personnel receive tasks from the queue through a dedicated review interface and make a final judgment on the material descriptions based on their professional knowledge. The personnel's correction results (confirming model predictions or specifying new correct classifications) will be recorded by the system.

[0103] It should be noted that all final results of manual review (whether confirmation or correction) are recorded and labeled as high-quality annotated samples. These samples are periodically or in real-time fed back to the system's dynamic threshold adjustment module to calculate the actual accuracy of recent predictions, thus providing crucial data for dynamic threshold optimization. Simultaneously, these samples can also serve as incremental data for future model retraining, continuously improving the system's intelligence.

[0104] This implementation ensures that the system operates at full capacity when there is confidence, thereby increasing throughput; and promptly hands over to human experts when there is uncertainty, fundamentally avoiding the risk of misclassification contaminating the main database and ensuring the authority and accuracy of the main data.

[0105] The present invention provides a material master data intelligent classification system, including at least one processor and a memory, wherein the memory stores a computer program that, when executed by the processor, implements the method described herein.

[0106] Therefore, it can achieve any effect of intelligent classification of material master data, which will not be elaborated here.

[0107] For any parts not mentioned in this invention, existing technologies can be used or referenced.

[0108] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0109] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A method for intelligent classification of material master data, characterized in that, include: Pre-acquire historical material description data of labeled material types to form training and testing sets. Generate text vectors from the material descriptions for classification model training. Determine the baseline confidence threshold based on test performance. Specifically, obtain the confidence of each sample output by the model when making predictions on the test set. Determine the corresponding quantile on the distribution curve of the confidence of each sample based on the preset initial target pass rate. The confidence value of this quantile is set as the baseline confidence threshold; The method further includes: Based on the real-time received material description text to be classified, a prediction vector is generated. Based on the classification model, the predicted material type and the corresponding prediction confidence are obtained. Based on the performance metrics of multiple classification predictions within a historical time sliding window, the baseline confidence threshold is adjusted and compared with the prediction confidence of real-time predictions to determine the prediction result, specifically as follows: Obtain the actual accuracy of manually reviewed prediction results within a historical time sliding window; calculate the average confidence level of multiple classification predictions within the historical time sliding window; obtain the average text cleaning intensity of multiple classification predictions for material descriptions within the historical time sliding window; based on the actual accuracy, average confidence level, average text cleaning intensity, and preset target automatic pass rate, dynamically adjust the current baseline confidence threshold using a predefined adjustment function, specifically: When the actual accuracy rate is lower than the pre-stored baseline accuracy rate, the current baseline confidence threshold is increased; when the average text cleaning intensity is higher than the pre-stored baseline cleaning intensity, the current baseline confidence threshold is decreased; when the target automatic pass rate is increased, the current baseline confidence threshold is decreased.

2. The method according to claim 1, characterized in that, Before generating text vectors from the material description, the process also includes text cleaning of the material description, specifically: Use regular expressions to remove punctuation marks and special characters; Filter words from the stop word list; And remove numeric strings that do not carry classification information.

3. The method according to claim 1, characterized in that, Generate text vectors for material descriptions, specifically: Use a Chinese word segmentation tool to segment the cleaned text; The Word2Vec model is used to map segmented words into word vectors; Text vectors are generated based on dynamic clustering grouping.

4. The method according to claim 3, characterized in that, Text vectors are generated based on dynamic clustering and grouping, specifically as follows: Based on the number and distribution of word vectors after word segmentation, the number of clusters is dynamically determined to cluster the word vectors and form multiple cluster groups; The average vector within each cluster is obtained by taking the arithmetic mean of the word vectors within each cluster group. Based on the cosine similarity between the average vector within the group and the centroids of all word vectors, the average vectors within the group are sorted and concatenated in descending order of similarity, and then projected through a fully connected neural network layer to generate the text vector.

5. The method according to claim 1, characterized in that, Classification model training includes: Multiple different types of machine learning algorithms are trained in parallel using the training set; The hyperparameters of each algorithm are optimized using grid search and cross-validation; Based on the evaluation metrics on the test set, a classification model is selected from multiple optimized algorithms; The machine learning algorithm mentioned above includes at least one of random forest, decision tree and XGBoost.

6. The method according to claim 1, characterized in that, After determining the prediction results, the following also includes: If the predicted confidence level is greater than or equal to the baseline confidence level threshold, the predicted material type will be automatically written into the material master database. If the predicted confidence level is less than the baseline confidence level threshold, the material description text, the predicted material type, and the predicted confidence level are pushed to the manual review queue.

7. A material master data intelligent classification system, characterized in that, It includes at least one processor and a memory, the memory storing a computer program that, when executed by the processor, implements the method as described in any one of claims 1 to 6.

Images