Processing method and system for rejecting and hanging work order data

Abstract

The invention discloses a processing method and system for rejecting and hanging work order data, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining the rejecting and hanging work order data and a real label corresponding to the rejecting and hanging work order data; inputting the rejected work order data into a trained basic text model to obtain a reasoning result, the reasoning result comprising a prediction label and a confidence coefficient; screening out the rejected work order data of which the confidence coefficient is smaller than a preset value and the predicted tag is inconsistent with the real tag from the reasoning result as low-quality data; determining a quality problem type of the low-quality data based on a quality problem determination rule; performing iterative optimization on the basic text model by adopting a corresponding optimization strategy based on the quality problem type to obtain a multi-label classification model; and inputting the to-be-improved work order rejecting and hanging data into the multi-label classification model to obtain an optimal reasoning result, thereby facilitating solving the problem that the reasoning result of the work order rejecting and hanging data cannot be accurately obtained in the prior art.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for processing rejected work order data. Background Technology

[0002] In the public service sector, core operations such as public service feedback and facility maintenance scheduling rely heavily on the efficient processing of public service data. Among these, the work order for removing anomalies and attaching related tasks (such as the work order for checking anomalies in facility maintenance and the work order for attaching materials to be supplemented in business processes) is a key link in ensuring the smooth operation of public services. Its classification accuracy directly determines the efficiency of work order diversion, service response speed and statistical analysis accuracy. Currently, the industry mainly uses two technical solutions for classifying and analyzing rejected work orders: one is a manual single-label classification scheme, where business personnel label work orders with a single category label based on experience (such as "facility damage - road potholes" or "material missing - ID card copy"), and then train a text classification model based on the labeled data to achieve automated classification; the other is a basic model-assisted single-label labeling scheme, which uses low-accuracy basic models such as simple Naive Bayes or basic BERT models to initially predict the work order category, and the human staff only checks the prediction results as "yes" or "no". Finally, the work order category is determined in the form of a single label, which is used for model iteration or business flow. However, the "data-model" closed loop constructed by these two types of solutions has significant technical defects, making it difficult to meet the public service's demand for "high accuracy and high adaptability" in work order classification: First, single-label annotation does not match the multiple feature attributes of work orders. In reality, more than 70% of work orders simultaneously possess multiple category features (e.g., "road potholes causing pedestrians to fall require the replacement of warning signs" simultaneously meets the criteria of "facility damage" and "safety protection"). Single-label annotation forcibly discards key information, causing the model to be unable to learn complete features, resulting in a multi-feature work order classification accuracy of less than 80%. Second, long-tail categories... First, the model fails to cover missing data. Long-tail work orders, which account for less than 5% of the total data (such as work orders for facility frost damage due to special weather), have a recognition rate of only 30% due to omissions in manual annotation and insufficient model sample size. A large number of work orders are misclassified as "other categories". Second, the model lacks timeliness adaptability. Adjustments to public service policies and seasonal business changes (such as work orders for adding heating facilities in winter) can cause significant changes in work order categories within a short period of time. However, the existing model adopts the "full-scale one-time training + fixed parameter deployment" mode, which requires full manual retraining to adapt, which is both time-consuming and interrupts business continuity.

[0003] Therefore, there is an urgent need for a method to process rejected work order data. Summary of the Invention

[0004] In view of this, the present invention proposes a method and system for processing rejected work order data, which can accurately obtain the classification results of rejected work orders.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for processing rejected work order data includes: Obtain the rejection work order data and the corresponding real tags of the rejection work order data; The rejected work order data is input into a trained basic text model to obtain inference results, which include predicted labels and confidence levels. The work order data with a confidence level lower than the preset value and a predicted label that is inconsistent with the actual label are selected from the inference results and identified as low-quality data. The quality problem type of the low-quality data is determined based on the quality problem determination rules. Based on the aforementioned quality problem types, the basic text model is iteratively optimized using corresponding optimization strategies to obtain a multi-label classification model. Input the work order data to be improved into the multi-label classification model to obtain the optimal inference result.

[0006] Based on the above technical solution, the present invention can be further improved as follows: Optionally, before the step of determining the quality problem type of the low-quality data based on the quality problem determination rules, the following steps are included: The low-quality data is analyzed, and those with common problems are grouped into the same category of samples.

[0007] Optionally, determining the type of quality problem for the low-quality data based on the quality problem determination rules includes: If the number of core semantic elements of multiple category labels in the same type of sample exceeds the first preset threshold, the quality problem type of the sample is determined to be a mismatch between multi-label features and single-label annotation; If the same type of sample is not fully recognized by the basic text model, and the proportion of the sample size to the total low-quality data is less than the second preset threshold, then the quality problem type of the sample is determined to be a long-tail category. If, when comparing similar rejection work orders from different time periods, the core semantic difference rate of similar rejection work orders is found to be greater than a third preset threshold, then the quality problem type of the sample is determined to be data timeliness.

[0008] Optionally, the iterative optimization of the basic text model based on the corresponding optimization strategy according to the quality problem type includes: If the quality problem type is a mismatch between multi-label features and single-label annotation, the rejection work order data is preliminarily processed based on the basic text model to obtain inference results, which include predicted labels. The predicted labels are then labeled with auxiliary annotations based on the real labels to construct a multi-label training dataset; The base text model is retrained using a multi-label loss function and the multi-label training dataset to obtain a multi-label classification model.

[0009] Optionally, the iterative optimization of the basic text model based on the corresponding optimization strategy according to the quality problem type further includes: Extract the core keywords for each long-tail category, and build a dynamically updated long-tail category keyword library based on the core keywords; When the rejected work order data enters the classification process, a keyword matching algorithm is used to compare the rejected work order data with the long-tail category keyword library. If the rejected work order data includes the core keywords of a certain long-tail category and the matching degree is greater than or equal to the fourth preset threshold, the long-tail category is marked as a candidate label for the rejected work order data. Input the rejection work order data labeled with candidate tags into the multi-label classification model to obtain the inference result, which includes the predicted label and confidence level; If the predicted label does not include the candidate label, a manual review process is triggered to update the correct label obtained from the manual review to the long-tail category keyword library.

[0010] Optionally, the iterative optimization of the basic text model based on the corresponding optimization strategy according to the quality problem type further includes: Determine whether the number of new samples accumulated through manual review has reached a set threshold. If it has, use the new samples to incrementally train the multi-label classification model.

[0011] Optionally, the iterative optimization of the basic text model based on the corresponding optimization strategy according to the quality problem type further includes: New samples are assigned high weights to complete the iterative optimization of the multi-label classification model.

[0012] A system for processing rejected work order data includes: The data acquisition module is used to acquire rejection work order data and the real tags corresponding to the rejection work order data; The initial inference module is used to input the rejected work order data into the trained basic text model to obtain inference results, which include predicted labels and confidence levels. The low-quality data filtering module is used to filter out the rejection work order data with a confidence level less than a preset value and a predicted label that is inconsistent with the actual label from the inference results as low-quality data. The quality problem determination module is used to determine the type of quality problem of the low-quality data based on the quality problem determination rules. The model iterative optimization module is used to iteratively optimize the basic text model based on the quality problem type and the corresponding optimization strategy to obtain a multi-label classification model; The final inference module is used to input the work order data to be improved into the multi-label classification model to obtain the optimal inference result.

[0013] An electronic device includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the method described herein.

[0014] A non-transitory computer-readable storage medium having a computer program stored thereon, the computer program implementing the steps of the method when executed by a processor.

[0015] The present invention has the following advantages: The method for processing work order data in this invention accurately filters low-quality data, determines the quality problem type of the low-quality data based on quality problem judgment rules, and optimizes the basic text model based on the quality problem type to obtain a multi-label classification model, thereby obtaining more accurate classification results. This solves the problems of mismatch between single labels and multiple features and insufficient coverage of long-tail categories, adapts to changes in data timeliness, and can respond to policy or business changes without full retraining. Attached Figure Description

[0016] For illustrative and not limiting purposes, the present invention will now be described in conjunction with embodiments and accompanying drawings, wherein: Figure 1 This is a flowchart illustrating the method for processing rejected work order data in an embodiment of the present invention. Figure 2 This is a schematic diagram of the main components of the processing system for canceled work order data in an embodiment of the present invention; Figure 3 This is a schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "first," "second," etc., in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be used interchangeably where appropriate for the embodiments of the invention described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] It should be noted that, where there is no conflict, the embodiments and features of the present invention can be combined with each other. The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0020] Figure 1 This is a flowchart illustrating the method for processing rejected work order data in an embodiment of the present invention, as shown below. Figure 1 As shown, the method for processing rejected work order data provided in this embodiment of the invention includes the following steps S101 to S106.

[0021] S101, obtain the rejection work order data and the corresponding real tags of the rejection work order data.

[0022] The data for the removed work orders comes from public service business systems (such as municipal facilities maintenance platforms and public service hotline work order systems), and includes fields such as work order ID, submission time, business type, original text description (such as "streetlights on XX section are damaged, posing a safety hazard at night"), and associated facility number. Cancellation work order: In the public service sector, a special record text used to "cancel out abnormal items (such as facility abnormalities or missing materials) and link them to corresponding processing tasks," which includes information such as abnormality description, location of occurrence, and related business; The actual labels are marked by two or more business experts based on the original work order text. The labeling rules refer to the public service classification standards (such as the "Classification Specification for Municipal Facility Maintenance Work Orders"). The label format is "Primary Category - Secondary Category" (such as "Facility Damage - Lighting Equipment"). If there are any discrepancies in the labeling, the final label is determined through expert consultation.

[0023] S102, input the rejected work order data into the trained basic text model to obtain the inference results, which include the predicted label and confidence level.

[0024] The inference results include work order ID, original text, model predicted label, prediction confidence, and true label, and are stored in CSV format (field example: work_order_id, original_text, predicted_label, confidence, true_label).

[0025] The basic text model used is a pre-trained BERT model (such as bert-base-chinese). The training data is historically labeled and completed work order data (sample size ≥ 5000). The training task is text classification. The loss function is the cross-entropy loss function. The number of training iterations is set to 10 rounds, and the learning rate is set to 2e-5. During inference, the original text of the rejection work order is first preprocessed (including Chinese word segmentation, stop word deletion, and text truncation / completion to 512 characters), and then input into the model to obtain the predicted label and confidence score. The confidence score ranges from [0,1], which represents the degree of trust the model has in the predicted label.

[0026] S103, filter out the work order data with confidence levels lower than the preset value and predicted labels inconsistent with the actual labels from the inference results as low-quality data.

[0027] The preset value is set according to business needs, and the value range is 0.5-0.7, preferably 0.6; The filtering logic is "logical AND": it must simultaneously satisfy "confidence level < 0.6" and "predicted label ≠ true label". For example, if the predicted label of a work order is "facility damage - lighting equipment" (confidence level 0.52) and the true label is "safety protection - night warning", it is judged as low-quality data.

[0028] S104, determine the type of quality problem for low-quality data based on the quality problem judgment rules.

[0029] Single-label classification: A classification method that labels / predicts only one category label for a single data point; The low-quality data is analyzed, and those with common problems are grouped into the same category of samples.

[0030] If the number of core semantic elements of multiple category labels in the same type of sample exceeds the first preset threshold, the quality problem type of the sample is determined to be a mismatch between multi-label features and single-label annotation; If the same type of sample is not fully recognized by the basic text model, and the proportion of the sample size to the total low-quality data is less than the second preset threshold, then the quality problem type of the sample is determined to be a long-tail category. If, when comparing similar rejection work orders from different time periods, the core semantic difference rate of similar rejection work orders is found to be greater than a third preset threshold, then the quality problem type of the sample is determined to be data timeliness.

[0031] One example is: if more than 80% of the sampled text contains the core semantics of multiple category labels (such as simultaneously containing the keywords "facility damage" and "safety protection"), then the data is judged to have a problem of "mismatch between multi-label features and single-label annotations". If, after sampling a certain type of work order, it is found that "the model does not recognize this category" and the sample size is less than 5% of the total data, it is determined to be a "long-tail category" problem. If a comparison of similar work orders from different time periods (e.g., January-June vs. June-December) reveals a core semantic difference rate greater than 30%, it is determined to be a "data timeliness" issue.

[0032] S105, based on the type of quality problem, the corresponding optimization strategy is used to iteratively optimize the basic text model to obtain a multi-label classification model.

[0033] If the quality problem type is a mismatch between multi-label features and single-label annotation, the rejection work order data is preliminarily processed based on the basic text model to obtain inference results, which include predicted labels. The predicted labels are then labeled with auxiliary annotations based on the real labels to construct a multi-label training dataset; The base text model is retrained using a multi-label loss function and the multi-label training dataset to obtain a multi-label classification model.

[0034] Multi-label classification: A classification method that labels / predicts multiple category labels that match the characteristics of a single data point; One example is: Assisted manual classification: The basic text model first outputs 1-3 predicted labels. Based on the work order data, the manual staff judges and supplements all relevant category labels (e.g., the basic text model outputs "facility damage - road surface", the manual staff adds "safety protection - warning sign", and the final label is "facility damage - road surface; safety protection - warning sign"). The manually labeled "multi-label work order data" is stored in the format of "work order text + multi-label set" (labels are separated by semicolons). When more than 1,000 work orders are accumulated, a multi-label training dataset is constructed. The task of classifying work order rejections is transformed into a multi-label classification task. The model is retrained using a multi-label loss function (such as BCEWithLogitsLoss) (the basic model retains the original architecture, and the output layer is adjusted to "number of labels × sigmoid activation"). After deployment, the model can directly output "multiple qualified category labels and corresponding confidence scores", achieving accurate classification of multi-feature data.

[0035] Extract the core keywords for each long-tail category, and build a dynamically updated long-tail category keyword library based on the core keywords; Long-tail categories: The phenomenon where the sample size of a certain type of data in a dataset is extremely low (usually <5%), causing the model to be unable to fully learn the features of that type; When the rejected work order data enters the classification process, a keyword matching algorithm is used to compare the rejected work order data with the long-tail category keyword library. If the rejected work order data includes the core keywords of a certain long-tail category and the matching degree is greater than or equal to the fourth preset threshold, the long-tail category is marked as a candidate label for the rejected work order data. Input the rejection work order data labeled with candidate tags into the multi-label classification model to obtain the inference result, which includes the predicted label and confidence level; If the predicted label does not include the candidate label, a manual review process is triggered to update the correct label obtained from the manual review to the long-tail category keyword library.

[0036] One example is: Organize all long-tail categories (such as "frozen damage to facilities in special weather" and "re-attachment of historical work orders"), extract the core keywords / phrases for each long-tail category (such as "frozen damage + special weather" and "re-attachment + historical work orders"), and build a dynamically updated keyword library (supporting manual addition / deletion of keywords). When a work order enters the classification process, it is first compared with the long-tail category keyword library using a "keyword matching algorithm" (such as TF-IDF keyword matching, regular expression matching): if the work order text contains the core keyword of a certain long-tail category (matching degree ≥ 80%), then the long-tail category is marked as a "candidate tag". The work order labeled with "candidate label" is input into the multi-label model. If the label output by the model contains the long-tail category and the confidence level is ≥0.7, then the work order is finally determined to contain the long-tail label. If the model does not output the long-tail category, manual review is triggered. The review results are used to update the keyword library (such as supplementing unrecognized keywords) and the model training data.

[0037] Confidence level: The probability value (usually in the range of 0-1) that the model predicts a certain result and that the result is correct.

[0038] Determine whether the number of new samples accumulated through manual review has reached a set threshold. If it has, use the new samples to incrementally train the multi-label classification model.

[0039] New samples are assigned high weights to complete the iterative optimization of the multi-label classification model.

[0040] One example is: In the model deployment architecture, a "parameter update interface" is reserved, which only allows parameter modification permissions for layers related to "category feature learning" (such as the top attention layer and output layer of Transformer), while freezing the underlying general feature learning layer to reduce parameter update costs. Incremental training trigger condition setting: When the number of multi-label work orders (including newly added categories and policy adjustment related work orders) that have been manually reviewed reaches 500, incremental training will be automatically triggered; During incremental training, the "Elastic Weight Consolidation (EWC) algorithm" is used to apply regularization constraints to the parameters of "learning historical policy knowledge" in the model (such as the feature weights of the original category labels) to avoid the forgetting of historical knowledge due to training with new data; EWC (Elastic Weight Consolidation) algorithm: An algorithm used to suppress "catastrophic forgetting" of models. It ensures that the model retains knowledge of old tasks when learning new tasks by imposing regularization constraints on important parameters of the model. For newly added policy-related work orders (such as work orders for "abnormal connection of heating facilities" after the implementation of new regulations), allocate twice the training weight of ordinary samples (achieved through weighting by loss function) to ensure that the model learns the features of the newly added categories / policy adjustments first; After incremental training is completed (training time is controlled within 30 minutes), the original model parameters are replaced through the parameter update interface to achieve "no-downtime update". The model can adapt to changes in timeliness without full retraining.

[0041] The processing method for discarded work order data in this invention transforms the passively discarded features of single-label annotations into actively covered features of multi-label annotations through a "progressive multi-label transformation." This increases the matching accuracy between labeled data and real data features from less than 70% to over 95%, allowing the model to learn complete text features. The accuracy of multi-feature work order classification is improved by over 25%, fundamentally solving the problem of low model accuracy caused by "disorganized classification." A hybrid system of "keyword detection + model inference" is constructed. First, long-tail categories are quickly located using keywords (solving the problem of "few samples leading to model inability to recognize"), and then model validation ensures accuracy (avoiding keyword mismatch). The long-tail category recognition rate is improved from less than 30% to over 75%, and the keyword library supports dynamic updates to adapt to newly emerging long-tail categories.

[0042] By adopting the "500 data points triggering incremental training + EWC algorithm", the high time consumption of "full retraining" is avoided (reduced from 12 hours / time to 0.5 hours / time), and the model is adapted to both "historical data" and "new data" through weight allocation and forgetting suppression. The model adaptation cycle after policy / business changes is shortened from 1 week to 1 day, which greatly reduces the cost of manual maintenance.

[0043] By using the method of "model inference results + confidence CSV output + sampling analysis", the core problems of data quality (such as multi-label features, long tail, and timeliness) can be quickly located. The efficiency is far higher than that of manual screening. Moreover, the solution is designed to address the identified problems, with no redundant modules and strong feasibility.

[0044] S106. Input the work order data to be improved into the multi-label classification model to obtain the optimal inference result.

[0045] Figure 2 This is a schematic diagram of the main components of the system for processing rejected work order data in an embodiment of the present invention. (See diagram below.) Figure 2 As shown, the processing system 1 for rejecting work order data provided in this embodiment of the invention includes a data acquisition module 10, an initial inference module 20, a low-quality data filtering module 30, a quality problem judgment module 40, a model iteration optimization module 50, and a final inference module 60.

[0046] Data acquisition module 10 is used to acquire rejection work order data and the real tags corresponding to the rejection work order data; The initial inference module 20 is used to filter out the rejection work order data with a confidence level lower than a preset value and a predicted label that is inconsistent with the actual label from the inference results as low-quality data. The low-quality data filtering module 30 is used to determine the type of quality problem of the low-quality data based on the quality problem judgment rules. The quality problem determination module 40 is used to iteratively optimize the basic text model based on the type of quality problem and the corresponding optimization strategy to obtain a multi-label classification model. The model iteration optimization module 50 is used to input the work order data to be improved into the multi-label classification model in order to obtain the optimal inference result; The final inference module 60 is used to input the data of the work orders to be improved into the multi-label classification model to obtain the inference results.

[0047] Figure 3 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as... Figure 3 As shown, the electronic device 70 includes: a processor 701, a memory 702, and a bus 703; The processor 701 and the memory 702 communicate with each other via the bus 703. The processor 701 is used to call program instructions in the memory 702 to execute the methods provided in the above-described method embodiments, and to execute the methods provided in the embodiments of the present invention.

[0048] This embodiment provides a non-transitory computer-readable storage medium that stores computer instructions, which cause a computer to execute the method provided in this embodiment of the invention.

[0049] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various storage media capable of storing program code, such as ROM, RAM, magnetic disk, or optical disk.

[0050] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method of processing a pick ticket data, characterized by, The method comprises the following steps: obtaining defect order data and real labels corresponding to the defect order data; inputting the defect order data into a trained basic text model to obtain inference results, wherein the inference results comprise predicted labels and confidence levels; screening defect order data with confidence levels less than a preset value and inconsistent predicted labels from the real labels as low-quality data; determining the quality problem type of the low-quality data based on quality problem determination rules; iteratively optimizing the basic text model based on the quality problem type using a corresponding optimization strategy to obtain a multi-label classification model; inputting defect order data to be improved into the multi-label classification model to obtain optimal inference results.

2. The method of claim 1, wherein, Before the step of determining the quality problem type of the low-quality data based on quality problem determination rules, the method comprises the following steps: analyzing the low-quality data and dividing low-quality data with common problems into the same class of samples.

3. The method of claim 2, wherein, The step of determining the quality problem type of the low-quality data based on quality problem determination rules comprises the following steps: if the number of core semantics of multiple class labels in the same class of samples exceeds a first preset threshold, it is determined that the quality problem type of the sample is that multi-label features do not match single-label annotations; if the same class of samples is not completely recognized by the basic text model, and the proportion of the sample quantity in the total low-quality data quantity is less than a second preset threshold, it is determined that the quality problem type of the sample is a long-tail class; if the core semantic difference rate of the same class of defect order data in different time periods is greater than a third preset threshold, it is determined that the quality problem type of the sample is data timeliness.

4. The method of claim 1, wherein, The step of iteratively optimizing the basic text model based on the quality problem type using a corresponding optimization strategy comprises the following steps: if the quality problem type is that multi-label features do not match single-label annotations, performing preliminary processing on the defect order data based on the basic text model to obtain inference results, wherein the inference results comprise predicted labels; performing auxiliary annotation on the predicted labels based on the real labels to construct a multi-label training data set; retraining the basic text model using a multi-label loss function and the multi-label training data set to obtain a multi-label classification model.

5. The method of claim 4, wherein, The step of iteratively optimizing the basic text model based on the quality problem type using a corresponding optimization strategy further comprises the following steps: extracting core keywords of each long-tail class and constructing a dynamically updateable long-tail class keyword library based on the core keywords; when defect order data enters the classification process, using a keyword matching algorithm to compare the defect order data with the long-tail class keyword library, if the defect order data includes core keywords of a long-tail class and the matching degree is greater than or equal to a fourth preset threshold, marking the long-tail class as a candidate label of the defect order data; inputting the defect order data marked with the candidate label into the multi-label classification model to obtain inference results, wherein the inference results comprise predicted labels and confidence levels; If the predicted label does not include the candidate label, an artificial review process is triggered, and a correct label obtained through artificial review is updated to the long-tail category keyword library.

6. The method of claim 5, wherein, The iterative optimization of the basic text model based on the quality problem type by using a corresponding optimization strategy further includes: It is determined whether the number of new samples accumulated through artificial review reaches a set threshold, and if so, the multi-label classification model is incrementally trained using the new samples.

7. The method of claim 1, wherein, The iterative optimization of the basic text model based on the quality problem type by using a corresponding optimization strategy further includes: The iterative optimization of the multi-label classification model is completed by assigning new samples with high weights.

8. A system for processing of a pick order data, characterized by It includes: A data acquisition module is configured to acquire rejected hanging order data and real labels corresponding to the rejected hanging order data; An initial inference module is configured to input the rejected hanging order data into a trained basic text model to obtain an inference result, the inference result including a predicted label and a confidence level; A low-quality data screening module is configured to screen rejected hanging order data with a confidence level less than a preset value and a predicted label inconsistent with a real label from the inference result as low-quality data; A quality problem determination module is configured to determine a quality problem type of the low-quality data based on a quality problem determination rule; A model iterative optimization module is configured to iteratively optimize the basic text model based on the quality problem type by using a corresponding optimization strategy to obtain a multi-label classification model; A final inference module is configured to input to-be-improved rejected hanging order data into the multi-label classification model to obtain an optimal inference result.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1-7.

10. A non-transitory computer readable medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1-7.

Images