A low-cognitive-load step-by-step data labeling method and system, and a storage medium

Abstract

The application discloses a low-cognitive-load step-by-step data labeling method and system and a storage medium, belongs to the technical field of data processing, collects unstructured field knowledge text as standardized knowledge, constructs a low-cognitive-load sequence decision tree through a structured Prompt prompt word, carries out step-by-step labeling based on the low-cognitive-load sequence decision tree, then respectively carries out node data acquisition and model quantitative analysis, updates the structured Prompt prompt word based on obtained structured node behavior data, a confusion matrix and quantitative indexes, and iteratively updates the low-cognitive-load sequence decision tree until a terminal value condition of iteration is met, and labels data based on the low-cognitive-load sequence decision tree after iteration. The application can realize low-cognitive-load step-by-step general labeling, achieves the goal of rapid operation of non-professionals, double improvement of efficiency and precision, and significant cost reduction, and has good practicability.

Description

Technical Field

[0001] This invention belongs to the technical field of data processing, specifically relating to a step-by-step data annotation method, system, and storage medium with low cognitive load. Background Technology

[0002] Data annotation is a core foundational step in training artificial intelligence models and digitizing industry operations. Existing data annotation technical solutions specifically include:

[0003] 1. Expertise-dependent annotation, the core process is as follows:

[0004] (1) Recruit annotation personnel with relevant professional backgrounds (e.g., backgrounds in microbiology, medical imaging, industrial design, etc.);

[0005] (2) Conduct systematic professional knowledge training, requiring annotation personnel to master all characteristics of the target object, the logic of category differentiation, and industry judgment standards;

[0006] (3) The annotators annotate the data based on a complete knowledge system, and finally, after cross-validation by multiple people / expert review, an annotated dataset is formed.

[0007] 2. Model-driven automated annotation; for example, the core process of hierarchical classification model-driven annotation is as follows:

[0008] (1) Obtain a large number of labeled samples by having experts in the corresponding field annotate a large number of samples, or by finding public databases / purchasing paid databases;

[0009] (2) A classification model is trained based on a large number of labeled samples to achieve automated classification of new data;

[0010] (3) Manual review and correction of model prediction results, or supplementary labeling for samples with low confidence in model prediction.

[0011] For example, Chinese patent CN113761197A describes a multi-label hierarchical classification method for application documents that utilizes expert knowledge. The steps include: 1) For each category of text data in the application document, extracting text information for each category; encoding each text data and adding a corresponding category code, then sending it to a word-level encoder to obtain a representation vector; 2) Inputting the representation vector corresponding to each text data into a document-level encoder to obtain the application document's representation vector; 3) Using the application document's representation vector and initialized prediction information, predicting the application document's label at the nth level of the subject system; 4) Based on the predicted label of the application document at the nth level of the subject system and the application document's representation vector, predicting the label of the application document at the (n+1)th level of the subject system; 5) If the current level is the last level of the subject system, completing the prediction and obtaining the finest-grained label; otherwise, returning to step 4) to perform the next level prediction. This technical solution constructs a text hierarchical classification model based on a Transformer encoder, and achieves multi-label hierarchical prediction of application document text by introducing category code Tokens and initializing expert knowledge parameters. This technical solution is essentially a model-driven automatic classification method.

[0012] For example, the existing Chinese patent CN117591676A, "A Method for Identifying Enterprises on the Supply Chain of Coarsetofine," includes the following steps: S1, acquiring patent data and performing data preprocessing; S2, generating pseudo-labels for patent data; S3, constructing an on-chain patent coarse identification network; S4, training the on-chain patent coarse identification network; S5, obtaining the full set of invention patent identification results from the on-chain patent coarse identification network; S6, constructing an on-chain patent fine identification network; S7, training the on-chain patent fine identification network; S8, acquiring the patent data to be identified; S9, obtaining the on-chain patent identification results; and S10, obtaining the on-chain enterprise identification results. This technical solution adopts a two-stage model structure of "coarse identification-fine identification," achieving supply chain enterprise identification through pseudo-label pre-training and a Transformer network. Although this technical solution has hierarchical division logic, its implementation relies entirely on trained model reasoning and automatic identification, resulting in weak versatility.

[0013] 3. Human-computer collaborative annotation; a human-computer collaborative mode of model pre-annotation + manual correction, the core process of which is as follows:

[0014] (1) Find existing pre-trained models;

[0015] (2) Use the pre-trained model to perform preliminary labeling of the data and output labels with confidence scores;

[0016] (3) The labelers only manually review and correct samples with confidence levels below the threshold;

[0017] (4) The labeled data is fed back to the model to improve model iteration and labeling efficiency.

[0018] For example, Chinese patent CN108985293A describes a deep learning-based automated image annotation method and system, which includes: collecting data to be annotated and storing it in a distributed file system; creating annotation tasks for the data to be annotated; determining whether an available automated annotation model exists; if not, allocating a portion of the annotation data for manual annotation and training a preliminary automated annotation model based on the manual annotation results; if an automated annotation model exists, performing automated annotation on the newly created annotation task; after automated annotation is completed, the annotators manually review the automated annotation results and store the annotation results in a database for subsequent model training and optimization.

[0019] In summary, existing data annotation technologies still rely on professional annotators or existing knowledge bases (such as public / paid databases, model parameters, etc.), which means that people with weak or no foundational knowledge cannot participate in core annotation decisions, resulting in poor versatility. Specifically, the above-mentioned technical solutions mainly suffer from the following shortcomings:

[0020] (1) High dependence on professional knowledge: Existing annotation methods usually require annotators to have systematic professional knowledge, which is difficult for ordinary people to get started directly, resulting in high training costs and long learning cycles;

[0021] (2) Complex single-step global decision-making: Traditional annotation processes lack step-by-step strategies, requiring annotators to process all categories or features at once, resulting in excessive cognitive load. This leads to high learning costs, low efficiency, and high error rates for annotators.

[0022] (3) Lack of self-optimization mechanism: Traditional processes are mostly static operation steps, which cannot dynamically adjust decision nodes and judgment order according to actual annotation behavior, thus limiting the improvement of efficiency and accuracy. Summary of the Invention

[0023] The purpose of this invention is to provide a step-by-step data annotation method, system, and storage medium with low cognitive load, aiming to solve the problems of high professional dependence, low efficiency, high cost, and high personnel threshold in existing annotation methods.

[0024] This invention is mainly achieved through the following technical solutions:

[0025] A step-by-step data annotation method with low cognitive load includes the following steps:

[0026] Step S1: Collect unstructured domain knowledge texts to obtain basic domain information as standardized knowledge;

[0027] Step S2: Construct a low-cognitive-load sequence decision tree based on standardized knowledge using structured Prompt prompts;

[0028] Step S3: Perform step-by-step annotation based on the low cognitive load sequence decision tree, and enable the annotators to form prior knowledge of the category features; then, proceed to steps S4 and S5 respectively;

[0029] Step S4: Bind the judgment behavior to the decision tree node, and count the annotation time and subjective feedback from personnel to form structured node behavior data;

[0030] Step S5: Quantitatively analyze the labeled data in Step S3 based on the YOLO model, and obtain the confusion matrix and quantification index;

[0031] Step S6: Path structure optimization; Based on the structured node behavior data in step S4 and the confusion matrix and quantification indicators in step S5, update the structured Prompt prompts and proceed to step S2 to perform targeted reconstruction and iteration of the decision tree for the low cognitive load sequence until the final value condition of the iteration is met.

[0032] Step S7: Label the data based on the iterative low cognitive load sequence decision tree.

[0033] To better realize the present invention, in step S2, in the low cognitive load sequence decision tree, the first-level nodes adopt macroscopic and intuitive features to divide into major categories and quickly compress the candidate space; the second-level and lower nodes adopt local fine-grained features, and each node retains 1 to 3 discriminative features that are not technical terms and can be directly observed, so as to achieve the gradual narrowing of the discriminative range.

[0034] To better realize the present invention, step S3 further includes the following steps:

[0035] Step S31: In the coarse labeling stage, frequently exposed samples are used to establish an intuitive impression of category distribution and similarity;

[0036] Step S32: In the fine labeling stage, the discrimination range is gradually narrowed through the low cognitive load sequence decision tree to make local distinctions of easily confused features, and a single judgment is made for local features, focusing attention on easily confused categories.

[0037] To better realize the present invention, in step S4, each discrimination behavior of the annotator in step S3 is bound to a node of the low cognitive load sequence decision tree. Data on the annotation time of each node and the degree of category confusion in the subjective feedback of the personnel are collected and statistically analyzed to form structured node behavior data, so that the low cognitive load sequence decision tree can continuously evolve into a dynamic structure with the annotation behavior.

[0038] To better implement the present invention, further, in step S5, a YOLO model is trained based on labeled data to obtain an object detection model; the labeled data in step S3 is input into the object detection model, and a confusion matrix and quantization index are generated to identify easily confused categories, high error rate nodes, and high time-consuming nodes; the quantization index includes accuracy and recall.

[0039] To better realize the present invention, step S6 further includes the following steps:

[0040] Step S61: In the structured Prompt prompts, construct the core basis based on the structured node behavior data using a confusion matrix and quantitative indicators, and search for relevant professional content to perform targeted optimization;

[0041] Step S62: In the structured Prompt prompts, construct optimization principles: retain the core framework, add intuitive subdivision features, adjust the discrimination order, and split complex nodes; then, proceed to step S2 to generate the optimized low cognitive load sequence decision tree.

[0042] To better realize the present invention, further, in step S62, the construction optimization principle is as follows: retain the original discrimination framework of first-level nodes and second-level nodes, and only update the second-level nodes corresponding to easily confused categories; add third-level nodes to supplement intuitive and non-technical subdivision discrimination features, and each node has only one judgment target; the construction optimization goal is to reduce the misjudgment rate of easily confused categories without increasing the annotation time, and meet the needs of subjective feedback from personnel.

[0043] This invention is mainly achieved through the following technical solutions:

[0044] A low-cognitive-load step-by-step data annotation system, used to implement the aforementioned low-cognitive-load step-by-step data annotation method, includes:

[0045] The knowledge input module is used to collect unstructured domain knowledge text to obtain basic domain information as standardized knowledge.

[0046] The decision tree generation module is used to construct low cognitive load sequence decision trees based on standardized knowledge using structured Prompt prompts.

[0047] The step-by-step annotation module is used for step-by-step annotation based on low cognitive load sequence decision trees;

[0048] The node data acquisition module is used to bind the judgment behavior to the decision tree node, and to count the time spent on annotation and subjective feedback from personnel to form structured node behavior data.

[0049] The model quantization analysis module is used to quantify and analyze the labeled data of the step-by-step annotation based on the YOLO model, and obtain the confusion matrix and quantization index.

[0050] The path structure optimization module is used to update the structured Prompt prompts based on structured node behavior data, confusion matrix, and quantification indicators.

[0051] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the aforementioned low-cognitive-load step-by-step data annotation method.

[0052] The beneficial effects of this invention are as follows:

[0053] (1) This invention constructs a complete closed loop encompassing knowledge input, generation of low-cognitive-load decision trees, step-by-step annotation, node data acquisition, model quantification analysis, and path structure optimization. It enables low-cognitive-load, step-by-step, universal annotation, achieving the goals of rapid onboarding for non-professionals, improved efficiency and accuracy, and significantly reduced costs. It is adaptable to high-professional-threshold annotation scenarios across multiple fields. Specifically, this invention generates low-cognitive-load sequence decision trees driven by structured prompt words, breaking down complex annotation tasks that rely on systematic professional knowledge into standardized, single-step, low-complexity sequential discrimination processes. This achieves structured, path-based, and reproducible annotation. Annotators do not need to possess domain knowledge; they can complete high-precision annotation simply by making selections, demonstrating good practicality.

[0054] (2) The present invention uses a step-by-step discrimination mechanism for data annotation: In the coarse annotation stage, high-frequency sample exposure enables the annotators to passively form prior knowledge of category features, which significantly reduces the cognitive load and learning cost in the fine annotation stage; In the fine annotation stage, the discrimination range is further narrowed through decision trees, and only local distinctions are needed for easily confused features, which greatly reduces the cognitive load, learning cost and decision difficulty of a single judgment.

[0055] (3) This invention introduces a path structure self-optimization closed-loop mechanism: real-time collection of data such as time consumption and category confusion of each decision node, automatic self-optimization of path branches for high-error / high-time-consumption nodes, so that the decision tree changes from a static rule to a dynamic structure that continuously evolves with the annotation behavior, continuously reducing the overall error rate and average decision cost. The path of this invention dynamically adjusts the decision nodes, feature order and path depth according to the annotation behavior, improving the overall annotation efficiency and accuracy.

[0056] (4) This invention forms a cross-domain universal path annotation framework that can be adapted to various high-professional-threshold scenarios such as microbial identification, medical imaging, remote sensing interpretation, and text classification. It truly realizes decision-making to replace knowledge dependence, enabling non-professionals to stably output high-precision annotation results that can be used in engineering. It achieves the optimal balance between efficiency, accuracy, and human cost, and has significant technical and commercial value.

[0057] (5) This invention solves the problems of high professional dependence, fixed and unoptimizable paths, and high human intervention in existing annotation technologies. This invention can achieve automatic iteration of annotation paths without the intervention of professional personnel, solving the pain point of high professional dependence, significantly improving annotation efficiency, and effectively reducing misjudgment of easily confused categories. The test results show that the accuracy rate of annotation by non-professional personnel is 82%, which is close to the level of professional personnel and can meet the requirements of engineering applications; at the same time, the annotation efficiency is improved by 26%, and the personnel cost is reduced by 32%, achieving a balanced optimization between efficiency, accuracy and cost. Attached Figure Description

[0058] Figure 1 The flowchart shows the step-by-step data annotation method with low cognitive load of the present invention.

[0059] Figure 2 This is a schematic diagram of the low cognitive load decision tree generated in Example 1;

[0060] Figure 3 This is a schematic diagram of the updated low cognitive load decision tree in Example 1. Detailed Implementation

[0061] Example 1:

[0062] A low-cognitive-load, step-by-step data annotation method is proposed, aiming to achieve universal, low-cognitive-load step-by-step annotation, enabling non-professionals to quickly learn, improving both efficiency and accuracy, and significantly reducing costs. It is adaptable to annotation scenarios with high professional barriers in multiple fields. For example... Figure 1 As shown, it includes the following steps:

[0063] Step S1: Knowledge Input; Collect unstructured domain knowledge texts to obtain basic domain information as standardized knowledge;

[0064] It can receive unstructured domain knowledge texts, eliminating the need for professionals to perform knowledge modeling and rule sorting. It only requires obtaining basic domain information from publicly available materials to provide standardized input for decision tree generation, significantly reducing the threshold and cost of knowledge organization.

[0065] For example, in the scenario of microbial annotation:

[0066] Inputting the collected content: "Vorticella and Cyclocarya have stalk-like structures; Cyclocarya has interconnected muscle filaments; Cyclocarya's head is bell-shaped; Mormonella has a bottle-shaped shell; Stratocarya has a stalk and needle-like tentacles; Heliozoa is round; Chaetoceros has a bulb-shaped shell," it can be seen that the above content does not require piling up technical terms. Based on existing AI network search capabilities, more professional knowledge can be searched in the target field.

[0067] Step S2: Construct a decision tree; using structured prompt words, construct a low-cognitive-load sequence decision tree based on standardized knowledge;

[0068] Using a general large model as an aid, and through a pre-set structured dedicated prompt, standardized knowledge is automatically transformed into a low-cognitive-load sequential decision tree, possessing unique design rules:

[0069] The first-level nodes use macroscopic and intuitive features to quickly compress the candidate space and complete the coarse labeling;

[0070] Nodes at level two and below employ local fine-grained features to progressively narrow the discrimination range;

[0071] Each node retains only 1 to 3 discriminative features that are free of technical jargon and can be directly observed, thus completely avoiding complex logical judgments;

[0072] Strictly adhering to the principle of "from coarse to fine, single step and single goal", the complex annotation task is broken down into a series of low-complexity, low-cognitive-load sequential discrimination steps.

[0073] The fundamental difference between this invention and existing decision trees is that it eliminates the need for manual rule compilation, domain expert modeling, and specialized model training. It automatically generates low-cognitive-load decision trees usable across multiple domains using only a general-purpose large model and a standardized prompt. Generation is completed in 10–20 seconds, with a structural compliance rate of 99%, and can directly guide non-professionals in annotation. Preferably, the general-purpose large model can be existing large language models such as Doubao, Qianwen, ChatGPT, and Gmini.

[0074] For example, the prompt word is:

[0075] {You are a professional low-cognitive-load decision tree generation expert. Generate decision trees strictly according to the following rules, without adding any extra explanations, only outputting the tree structure:}

[0076] 1. First-level nodes: Using the most intuitive and directly observable macroscopic features, the nodes are divided into major categories to quickly compress the candidate space;

[0077] 2. Second-level and lower nodes: Local fine-grained features are used to gradually subdivide the nodes. Each node has only 1 to 3 discriminative features and no technical terms are used.

[0078] 3. Each final node is labeled with a specific category and a "discriminative feature" (intuitive and observable, requiring no technical understanding);

[0079] 4. Output format: Use ├─ and └─ to represent the hierarchy, which is clear and easy to understand.

[0080] The following is standardization knowledge: {processed_knowledge}. You will need to search for relevant professional content to supplement this.

[0081] Here, {processed_knowledge} refers to the basic domain information collected in the previous step. Taking the microbial annotation scenario as an example, the generated low-cognitive-load decision tree is as follows: Figure 2 As shown.

[0082] As can be seen, based on the characteristic that general large models can understand semantics, and guided by prompts, low-cognitive-load sequence decision trees can be generated, allowing non-experts to immediately start annotation. Research shows that generating decision trees takes only 10-20 seconds per iteration and requires no additional cost; multiple cross-domain tests show that the decision tree structure has a compliance rate of 99% and is universally adaptable to various fields.

[0083] Preferably, the hierarchy (indentation / symbols) can also be identified through Python code, and the output AI decision tree can be directly output as a standard JSON tree structure for subsequent use in online annotation platforms / other development.

[0084] Step S3: Distributed annotation; Step-by-step annotation is performed based on a low cognitive load sequence decision tree, enabling annotators to form prior knowledge of category features; then, steps S4 and S5 are performed respectively.

[0085] Step-by-step annotation is performed based on low cognitive load sequence decision trees. Through multiple rounds of traversal and repeated target observation, annotators passively form prior knowledge of category features without active learning.

[0086] (1) Coarse labeling stage: High-frequency exposed samples to establish an intuitive impression of category distribution and similarity;

[0087] (2) Precision labeling stage: Only a single judgment needs to be made on local features, and attention is focused on easily confused categories. There is no need to master complete domain knowledge.

[0088] (3) Each step only performs a simple two-choice / multiple-choice operation, minimizing cognitive load and enabling novices to get started after a short training period. The annotation process is easy, stable and replicable.

[0089] The aforementioned prior knowledge is intended to scientifically and objectively prove the possibility that step-by-step annotation can improve efficiency. This is a subtle influence that cannot be quantified, but science has proven its existence.

[0090] Step S4: Node data collection; bind the judgment behavior to the decision tree node, and statistically analyze the annotation time and subjective feedback from personnel to form structured node behavior data;

[0091] By precisely binding each judgment action of the annotator to a node in the decision tree, collecting and statistically analyzing the annotation time and subjective feedback of each node, structured node behavior data is formed, providing a real and quantitative underlying basis for subsequent problem localization and path optimization, and realizing the traceability and analyzability of the annotation process.

[0092] The time consumed can be collected manually or through a platform. Manual collection relies on manual timing, while platform-based collection can use JSON lists to divide tasks and then calculate the time using the system. For example, the time consumed in collecting annotations for each node:

[0093] First-level nodes (with or without handle-like structures): 2.9 seconds / frame;

[0094] Secondary nodes (with or without myofilaments): 5.2 seconds / frame;

[0095] Secondary nodes (whether or not they have needle-like tentacles): 3.4 seconds / sheet;

[0096] Secondary nodes (with or without a cover): 3.7 seconds / frame.

[0097] For example, the subjective feedback from the data collectors was:

[0098] "Some of the images are just too blurry! The skin filaments are hard to see clearly, and I often hesitate here."

[0099] "Some shells look like bottles, and some look like light bulbs. Are there any other characteristics that can help us make a judgment?"

[0100] Step S5: Model Quantitative Analysis; Based on the YOLO model, perform quantitative analysis on the labeled data in Step S3 to obtain the confusion matrix and quantification index;

[0101] The YOLO model is trained based on labeled data to generate confusion matrices and quantify metrics (accuracy, recall, etc.). This allows for the accurate identification of core issues such as easily confused categories, high error rate nodes, and high time consumption nodes. It transforms subjective human perception into calculable and optimization-driven objective data, avoiding blind optimization.

[0102] The generated confusion matrix reveals which classes are well-identified and which are easily confused. For example, in a microbial annotation scenario, a YOLOv11 model was trained using 1000 labeled samples, as shown in Table 1, generating a confusion matrix (translated into Chinese). Analysis revealed that the annotation accuracy was 82%, with only minor confusion in a few cases, such as between *Sphaerothora* and *Branchiaceae*, and between *Mormonella* and *Ceratophyllum demersum*.

[0103] Table 1 Microbial Labeling Confusion Matrix

[0104]

[0105] Step S6: Path structure optimization; Based on the structured node behavior data in step S4 and the confusion matrix and quantification indicators in step S5, update the structured Prompt prompts and proceed to step S2 to perform targeted reconstruction and iteration of the decision tree for the low cognitive load sequence until the final value condition of the iteration is met.

[0106] Based on structured node behavior data and YOLO model quantitative analysis results, a general large model is driven to perform targeted reconstruction and iteration of decision trees for low cognitive load sequences, achieving path self-optimization without human intervention. The specific steps are as follows:

[0107] (1) Decision tree reconstruction: AI generates an optimized decision tree according to the rules of "retaining the core framework, adding intuitive subdivision features, adjusting the discrimination order, and splitting complex nodes";

[0108] (2) Automatic landing and verification: Replace the old decision tree, maintain the continuity of the annotation process, and do not require the annotators to relearn;

[0109] (3) Closed-loop iteration: The optimized decision tree is labeled, and data is continuously collected → problem is analyzed → optimization is performed again, forming a dynamic path mechanism for continuous evolution.

[0110] This invention enables low cognitive load sequence decision trees to be continuously iterated and optimized with annotation behavior, constructing a dynamic system that can sustainably reduce error rate, shorten annotation time, and improve annotation accuracy.

[0111] For example, the Prompt message in step S2 is updated as follows:

[0112] {You are a professional expert in low-cognitive-load decision tree generation and optimization. Strictly follow these general rules to optimize decision trees without adding any extra explanations, outputting only the optimized complete tree structure to ensure universality, low cognitive load, feasibility, and close alignment with actual annotation scenarios:}

[0113] Core basis: Combining the model confusion matrix {yolo_confusion_matrix}, quantitative metrics {quantitative_metrics}, annotation time {time_consumption}, and personnel feedback {personnel_feedback}, we focused on the core pain points, searched for relevant professional content online, and optimized accordingly;

[0114] Optimization principles: ① Retain the core logic of the original decision tree's first and second-level nodes, without changing the basic discrimination framework, and only modify / insert after the second-level nodes corresponding to easily confused categories; ② New third-level nodes need to be supplemented with one intuitive, directly observable, non-technical, and non-professional sub-discrimination feature, with only one judgment target per node; ③ After optimization, the misjudgment rate of core easily confused categories should be reduced, while not increasing the annotation time (single node time should not exceed 6 seconds), in line with the needs of personnel feedback;

[0115] Detailed requirements: ① The final-level nodes must be clearly labeled with the specific category and corresponding discriminant features (intuitive and observable, no professional principles required); ② The output format still uses ├─ and └─ to represent the hierarchy, completely consistent with the original decision tree format, clear and concise; ③ No irrelevant categories should be added, no original core categories should be omitted, and reasonable fallback nodes should be added to avoid misjudgment of non-target categories; ④ The optimized decision tree must be compatible with the original labeling scenario, so that labelers do not need to relearn and can execute it directly, maintaining low cognitive load characteristics;

[0116] The following is the original decision tree data: {decision_tree}}.

[0117] Taking microbial annotation as an example, simply insert the corresponding data into the {} boxes. The updated low-cognitive-load decision tree will look like this: Figure 3 As shown.

[0118] Step S7: Label the data based on the iterative low cognitive load sequence decision tree.

[0119] Preferably, the present invention can be systematized through Python development, enabling automatic generation of low cognitive load sequence decision trees from input data, automatic determination of whether optimization is needed, and automatic optimization of low cognitive load sequence decision trees, etc.

[0120] Example 2:

[0121] A stepwise data annotation method with low cognitive load is proposed, which annotates data based on the iteratively derived low cognitive load sequence decision tree. In an office environment, 1000 microbial samples were selected as annotation data, with the annotation format uniformly set to YOLO (target category + bounding box coordinates) for subsequent accuracy comparison and model training. Details are as follows:

[0122] 1. Staffing of the experimenters;

[0123] (1) Expert benchmarking group (benchmark group);

[0124] Huang XX: Biology lab technician, Bachelor's degree, with a background in microbiology;

[0125] Zhang XX: Master's student in biology, with relevant experimental experience;

[0126] The labeled results of this group are used as the "standard answer," with a default accuracy of 100%.

[0127] (2) Step-by-step annotation group (experimental group, none of whom had relevant professional background);

[0128] Xiang XX: Undergraduate student at a 985 university (majoring in marine science);

[0129] Huang XX: Graduated from a local junior college (majoring in mechanical engineering);

[0130] Lu XX: Graduated from a 211 university with a bachelor's degree (majoring in accounting);

[0131] Wang XX: Graduated from an ordinary university with a bachelor's degree (majoring in computer science).

[0132] 2. Experimental procedure;

[0133] The expert labeling group directly labels the final scientific name; the step-by-step labeling group labels the data based on the iterative low cognitive load sequence decision tree mentioned above—in the coarse labeling stage, it first determines whether the target has a "stalk-like structure," and in the fine labeling stage, it determines whether the stalk-like samples have a "myofilament structure" based on the low cognitive load sequence decision tree.

[0134] 3. Comparison of annotation efficiency;

[0135] Because step-by-step annotation uses a multi-round traversal mechanism, it is not compared by the speed per hour, but by the total time to complete the same task.

[0136] (1) Expert labeling team: 2 people completed the labeling of 1,000 samples, with a total time of about 9.6 hours;

[0137] (2) Step-by-step annotation group: 4 people worked together to complete 1,000 samples, with a total time of about 7.1 hours (excluding about 0.5 hours of training).

[0138] As can be seen, under conditions of multi-person collaboration, the total completion time is reduced by approximately 26%. At the same time, step-by-step annotation does not require the participation of professional personnel and can be scaled in parallel, which means that there is still room for further optimization of efficiency.

[0139] 4. Comparison of labeling accuracy;

[0140] We used the expert annotation results as the "standard answer," removing content that was 32% unidentified to ensure consistency. Due to objective factors such as data clarity, the expert annotation group had 32% of cases where scientific names were not identified, but the step-by-step annotation group, due to its strict judgment criteria, distinguished all categories. The step-by-step annotation results were then compared for consistency on a sample-by-sample basis.

[0141] (1) The accuracy rate of the expert labeling group: 100% (32% of the content that was not identified was removed).

[0142] (2) Accuracy rate of step-by-step annotation group: 82%;

[0143] As can be seen, the step-by-step annotation group achieved 82% of the annotation accuracy of the expert annotation group without any professional knowledge.

[0144] 5. Personnel cost analysis;

[0145] Taking Shanghai as an example, recruiting biology lab technicians with a bachelor's degree would cost approximately 400 yuan per day, or 50 yuan per hour. Based on the labeling efficiency above, two people working 9.6 hours would produce 2 × 50 × 9.6 = 960 labels per thousand sheets. Recruiting part-time labelers without specific professional requirements would cost approximately 23 yuan per hour. Based on the labeling efficiency above, four people working 7.1 hours would produce 4 × 23 × 7.1 = 653.2 labels per thousand sheets. As can be seen, in this experiment, with a 26% increase in efficiency and an 82% accuracy rate, personnel costs decreased by approximately 32%.

[0146] Of course, "number of people and hourly wage" can also be incorporated into the cost function to design an optimal cost model per unit sample, thereby achieving the optimal personnel allocation plan.

[0147] 6. Experiment feedback;

[0148] (1) Subjective feedback from the statistical expert standardization team;

[0149] Huang XX: "There are too many sample categories, and we often have to compare them repeatedly between multiple similar categories. Sometimes we hesitate for a long time when labeling, and it gets quite tiring after a while."

[0150] Zhang XX: "Because some categories have very subtle differences, I need to stay highly focused at all times to ensure the accuracy of the labeling, otherwise it is easy to make mistakes in judgment, which makes the overall burden quite heavy."

[0151] (2) Subjective feedback from the personnel in the step-by-step annotation group;

[0152] Xiang XX: "At first I wasn't very familiar with it, but just follow the steps. You don't need to know biology. After a few rounds, you'll get the hang of it because you've seen these microorganisms several times before."

[0153] Huang XX: "I just make decisions based on my judgment in each round. It's very easy. I don't have to consider many things at the same time. There is no pressure in doing it. It feels much simpler than going to school."

[0154] Lu XX: "I am a liberal arts student and have no professional knowledge at all, but after looking at a lot of samples, I subconsciously feel which ones look similar, and I can make judgments faster and faster later on."

[0155] Wang XX: "It will be a little slow at first, but after watching it several times, you can distinguish many of them at a glance, and you basically don't need to think about it much."

[0156] Feedback indicates that the expert labeling group relies heavily on knowledge memorization and comprehensive judgment, which can easily lead to cognitive burden. The step-by-step labeling group, through repeated observation of a single target in multiple rounds, naturally develops an intuitive understanding of category distribution and similarity, which gradually simplifies subsequent judgments and makes the labeling process significantly easier.

[0157] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications or equivalent changes made to the above embodiments based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A step-by-step data annotation method with low cognitive load, characterized in that, Includes the following steps: Step S1: Collect unstructured domain knowledge texts to obtain basic domain information as standardized knowledge; Step S2: Construct a low-cognitive-load sequence decision tree based on standardized knowledge using structured Prompt prompts; Step S3: Perform step-by-step annotation based on the low cognitive load sequence decision tree, and enable the annotators to form prior knowledge of the category features; then, proceed to steps S4 and S5 respectively; Step S31: In the coarse labeling stage, frequently exposed samples are used to establish an intuitive impression of category distribution and similarity; Step S32: In the fine labeling stage, the discrimination range is gradually narrowed through the low cognitive load sequence decision tree to make local distinctions of easily confused features, and a single judgment is made for local features, focusing attention on easily confused categories; Step S4: Bind the judgment behavior to the decision tree node, and count the annotation time and subjective feedback from personnel to form structured node behavior data; In step S4, each discrimination action of the annotator in step S3 is bound to a node of the low cognitive load sequence decision tree. Data on the annotation time of each node and the degree of category confusion in the subjective feedback of the personnel are collected and statistically analyzed to form structured node behavior data, so that the low cognitive load sequence decision tree can continuously evolve with the annotation behavior in a dynamic structure. Step S5: Quantitatively analyze the labeled data in Step S3 based on the YOLO model, and obtain the confusion matrix and quantification index; Step S6: Path structure optimization; Based on the structured node behavior data in step S4 and the confusion matrix and quantification indicators in step S5, update the structured Prompt prompts and proceed to step S2 to perform targeted reconstruction and iteration of the decision tree for the low cognitive load sequence until the final value condition of the iteration is met. Step S7: Label the data based on the iterative low cognitive load sequence decision tree.

2. The step-by-step data annotation method with low cognitive load according to claim 1, characterized in that, In step S2, in the low cognitive load sequence decision tree, the first-level nodes use macroscopic and intuitive features to classify major categories and quickly compress the candidate space. Nodes below level two employ local fine-grained features, and each node retains 1 to 3 discriminative features that are straightforward and easy to observe, in order to progressively narrow the discriminative range.

3. The step-by-step data annotation method with low cognitive load according to claim 1, characterized in that, In step S5, a YOLO model is trained based on labeled data to obtain an object detection model; the labeled data from step S3 is input into the object detection model, and a confusion matrix and quantization metrics are generated to identify easily confused categories, high error rate nodes, and high time-consuming nodes; the quantization metrics include accuracy and recall.

4. A step-by-step data annotation method with low cognitive load according to any one of claims 1-3, characterized in that, Step S6 includes the following steps: Step S61: In the structured Prompt prompts, construct the core basis based on the structured node behavior data using a confusion matrix and quantitative indicators, and search for relevant professional content to perform targeted optimization; Step S62: In the structured Prompt prompts, the optimization principles are: retain the core framework, add intuitive subdivision features, adjust the discrimination order, and split complex nodes; then, proceed to step S2 to generate the optimized low cognitive load sequence decision tree.

5. The step-by-step data annotation method with low cognitive load according to claim 4, characterized in that, In step S62, the construction optimization principle is as follows: retain the original first-level and second-level node discrimination framework, and only update the second-level nodes corresponding to easily confused categories; add third-level nodes to supplement intuitive and non-technical subdivision discrimination features, and each node has only one judgment target; the construction optimization goal is to reduce the misjudgment rate of easily confused categories without increasing the annotation time, and meet the needs of subjective feedback from personnel.

6. A low-cognitive-load step-by-step data annotation system, used to implement the low-cognitive-load step-by-step data annotation method according to any one of claims 1-5, characterized in that, include: The knowledge input module is used to collect unstructured domain knowledge text to obtain basic domain information as standardized knowledge. The decision tree generation module is used to construct low cognitive load sequence decision trees based on standardized knowledge using structured Prompt prompts. The step-by-step annotation module is used for step-by-step annotation based on low cognitive load sequence decision trees; The node data acquisition module is used to bind the judgment behavior to the decision tree node, and to count the time spent on annotation and subjective feedback from personnel to form structured node behavior data. The model quantization analysis module is used to quantify and analyze the labeled data of the step-by-step annotation based on the YOLO model, and obtain the confusion matrix and quantization index. The path structure optimization module is used to update the structured Prompt prompts based on structured node behavior data, confusion matrix, and quantification indicators.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements a step-by-step data annotation method with low cognitive load as described in any one of claims 1-5.

Images