OpenCSG large model intelligent data annotation system

Abstract

According to the OpenCSG large model intelligent data annotation system provided by the invention, by virtue of an automatic annotation technology driven by a large model, the requirement of manual annotation is greatly reduced, and the data annotation efficiency is remarkably improved. The system can generate a high-precision labeling result by using the intelligent labeling module and the automatic quality control module, and ensures the consistency and accuracy of the labeling result through an automatic auditing process. No matter sentiment analysis, entity recognition or complex text classification, the system can efficiently adapt to various types of labeling tasks, and different data processing requirements are met. Through modular design, the system supports flexible expansion, and can meet various data labeling requirements from small-scale test projects to large-scale commercial projects. According to the overall architecture, all the function modules are closely coordinated in the system, so that efficient operation and high-quality output are always kept in a large-scale data annotation project.

Description

Technical Field

[0001] The invention belongs to the field of data processing, and in particular relates to an OpenCSG large model intelligent data labeling system. Background Art

[0002] With the rapid development of artificial intelligence and machine learning technologies, the application of large models has gradually become a core tool for data processing and analysis. However, although large models can efficiently process and analyze data, they perform poorly in large-scale data labeling, becoming one of the bottlenecks restricting the promotion and application of technology.

[0003] Traditional data labeling is heavily dependent on labelers, which not only consumes a lot of manpower and time, but is also prone to labeling errors due to human factors, thus affecting data quality. The decline in data quality will not only have a negative impact on the training effect of the model, but also directly affect the performance of the model in practical applications. Especially in large-scale data scenarios, the high cost and low efficiency of manual labeling are more prominent, making it difficult to achieve effective resource allocation and management.

[0004] Although existing automated labeling methods have alleviated the burden of manual labeling to a certain extent, they are less effective when faced with complex and diverse labeling tasks. They find it difficult to ensure the accuracy and consistency of labeling and are unable to adapt to the complex contextual requirements of different types of data.

[0005] Therefore, the prior art has significant defects in the following aspects:

[0006] High cost of manual labeling: The traditional manual labeling model requires a lot of manpower investment, especially when dealing with large-scale data, it is difficult to achieve cost-effectiveness.

[0007] Low labeling efficiency: The manual labeling process is cumbersome and highly repetitive, resulting in low efficiency and extending the overall data processing cycle.

[0008] Unstable labeling quality: Due to the intervention of human factors, the labeling quality is inevitably subjective, resulting in a lack of consistency in data labeling results, which in turn affects the training effect of the model.

[0009] Existing automated tools have poor flexibility: Although some existing automated labeling tools can provide help to a certain extent, their flexibility and adaptability are obviously insufficient when faced with complex and diverse labeling tasks, and they are difficult to meet the data labeling needs in different scenarios.

[0010] In summary, the field of data labeling urgently needs an efficient, accurate and flexible labeling solution to break through the bottleneck of existing technologies. Summary of the invention

[0011] The present invention provides an OpenCSG large model intelligent data labeling system to solve the problem that the prior art cannot label large-scale data efficiently, accurately and flexibly.

[0012] In order to solve the above technical problems, the embodiments of the present invention disclose the following technical solutions:

[0013] One aspect of the present invention provides an OpenCSG large model intelligent data annotation system, the system is driven by the OpenCSG large model, and includes:

[0014] The automatic annotation module, including a semantic analysis unit, a multi-level annotation unit, and a dynamic adaptation unit, is configured to be built based on the OpenCSG large model to perform semantic analysis and automatic annotation on the text data of the annotation task;

[0015] An intelligent annotation module in communication with the automatic annotation module, including an annotation suggestion unit, an intelligent error correction unit and an interactive annotation unit, configured to provide annotation suggestions, detect the annotation results and generate an annotation correction plan;

[0016] A task management and allocation module in communication with the intelligent labeling module, including a task allocation unit, a real-time progress tracking unit and a task coordination unit, configured to allocate labeling tasks to one or more labelers and integrate the labeling results;

[0017] The automated quality control module, which is in communication with the task management allocation module, includes a quality review unit, an automatic correction unit, and a version control and result optimization unit, and is configured to review the annotation results and automatically correct detected annotation errors.

[0018] Optionally, the semantic analysis unit is constructed based on the OpenCSG large model, and is configured to extract semantic information, sentiment tendency and entity relationship in the text data, and annotate the text data according to the extracted content.

[0019] Optionally, the multi-level annotation unit is constructed based on the OpenCSG large model, and is configured to perform semantic analysis on the text data and then perform annotation in a layer-by-layer parsing manner.

[0020] Optionally, the dynamic adaptation unit is configured to automatically adjust the parameters and annotation strategy of the OpenCSG large model in a preset manner according to data characteristics and annotation task requirements.

[0021] Optionally, the annotation suggestion unit is configured to output annotation suggestions for the current text data based on the historical annotation data and context information of the current text data.

[0022] Optionally, the intelligent error correction unit is communicatively connected to the semantic analysis unit and the multi-level annotation unit respectively, and is configured to receive annotation information output by the semantic analysis unit and / or the multi-level annotation unit, and detect input information of the annotation personnel based on the annotation information, and generate prompt information when errors are detected in the input information, wherein the prompt information includes an annotation correction plan.

[0023] Optionally, the task assignment unit is configured to automatically assign the labeling task to the most matching labeling personnel based on a preset machine learning algorithm and information about the labeling task and the labeling personnel.

[0024] Optionally, the task coordination unit is configured to decompose a labeling task into multiple subtasks and assign the subtasks to multiple labelers, and integrate the labeling results of all labelers.

[0025] Optionally, the quality review unit is configured to perform a primary review on the annotation results, detect annotation errors, and, after completing the primary review, perform a secondary review on the annotation results according to quality standard information of the annotation task to ensure that the annotation results meet the quality standards.

[0026] Optionally, the automatic correction unit is communicatively connected to the quality review unit, and is configured to automatically correct the labeling errors detected during the primary review and the secondary review, and replace the labeling result with the correction result after receiving a confirmation signal from the labeler.

[0027] The OpenCSG large model intelligent data annotation system provided by the embodiment of the present invention greatly reduces the need for manual annotation and significantly improves the efficiency of data annotation by relying on the large model-driven automated annotation technology. The system uses the intelligent annotation module and the automated quality control module to generate high-precision annotation results and ensure the consistency and accuracy of the annotation results through an automated review process. Whether it is sentiment analysis, entity recognition, or complex text classification, the system can efficiently adapt to various types of annotation tasks to meet different data processing requirements.

[0028] Through modular design, the system supports flexible expansion and can meet various data annotation requirements from small-scale experimental projects to large-scale commercial projects. The overall architecture enables each functional module to work closely together in the system to ensure efficient operation and high-quality output in large-scale data annotation projects.

[0029] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the disclosure, nor is it intended to limit the scope of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0031] Figure 1 A schematic diagram of the structure of an OpenCSG large model intelligent data annotation system provided by an embodiment of the present invention;

[0032] Figure 2 A method provided by an embodiment of the present invention Figure 1 Schematic diagram of the structure of the subunits of the automatic annotation module;

[0033] Figure 3 A method provided by an embodiment of the present invention Figure 1 The schematic diagram of the structure of the subunit of the intelligent marking module;

[0034] Figure 4 A method provided by an embodiment of the present invention Figure 1 A schematic diagram of the structure of the task management allocation module subunit;

[0035] Figure 5 A method provided by an embodiment of the present invention Figure 1 Schematic diagram of the structure of the subunits of the automated quality control module. DETAILED DESCRIPTION

[0036] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0037] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0038] Figure 1 Schematic diagram of the structure of an OpenCSG large model intelligent data annotation system disclosed in an embodiment of the present invention. The intelligent data annotation system is driven by the OpenCSG large model. Figure 1As shown, it includes an automatic labeling module 1, an intelligent labeling module 2, a task management and allocation module 3 and an automated quality control module 4.

[0039] 1. Automatic annotation module

[0040] The automatic annotation module 1 is built based on the OpenCSG large model, which can perform semantic analysis and automatic annotation on the text data of the annotation task. The OpenCSG large model is a platform for intelligent data annotation, which combines large-scale pre-trained models and automated data processing technology to improve the efficiency and quality of data annotation. In an embodiment of the present invention, the CSG-WuKong-Enterprise V2 model is adopted. This model is an enterprise-level large-scale AI model, which is mainly used to process complex natural language processing (NLP) tasks and large-scale data annotation.

[0041] In one embodiment disclosed in the present invention, Figure 2 As shown, the automatic annotation module 1 includes a semantic analysis unit 11 , a multi-level annotation unit 12 and a dynamic adaptation unit 13 .

[0042] (1) Semantic Analysis Unit

[0043] The semantic analysis unit 11 can understand and parse the input text data, extract the semantic information, sentiment tendency and entity relationship in the text data. The semantic analysis unit 11 can assign appropriate labels to the text data according to the extracted content, such as sentiment classification, entity recognition, topic classification, etc., by combining the training model with the labeling task.

[0044] When performing the labeling task, the semantic analysis unit 11 uses the trained CSG-WuKong-EnterpriseV2 model to improve the automation and accuracy of labeling. This process combines machine learning or deep learning models (such as pre-trained natural language processing models) to understand and process the semantic information in the text, and automatically label the data based on this semantic information.

[0045] The semantic analysis unit 11 can understand and parse the text, identify different entities in the text (such as names of people, places, time, etc.), judge the emotional orientation of the text (such as positive, negative, and neutral emotions), and extract relevant topic information, providing a key foundation for subsequent automated processing and data annotation.

[0046] 1. The semantic analysis unit 11 first performs word segmentation, part-of-speech tagging and syntactic analysis on the input text, and then understands the various components in the text. Using the trained deep learning model, the system can identify the main semantic elements in the sentence. For example, for the sentence "Today's weather is very good, suitable for going out for a walk", the system can understand that the semantics is a positive emotional expression, and can identify "weather" as a subject entity.

[0047] 2. The semantic analysis unit 11 can classify the sentiment orientation of the text. Through the OpenCSG large model, it can determine whether the text shows a positive, negative or neutral sentiment tendency. For example, for consumer reviews, the system can mark whether the evaluation is positive (such as "the product is very good") or negative (such as "the service is too bad").

[0048] 3. The semantic analysis unit 11 is not limited to the surface meaning of words and phrases, but can also identify the relationship between different entities in the text. Through entity recognition and relationship extraction technology, the system can mark the key entities and their relationships in the text. For example, in "Xiao Ming joined Company A in 2022", the system can recognize that "Xiao Ming" is a person's name and "A" is a company name, and define the relationship between the two as "employment".

[0049] According to the extracted semantic information, the semantic analysis unit 11 assigns appropriate labels to the text. For example, the system can label news articles with topic labels (such as "technology", "sports", etc.), assign sentiment classification labels (such as "positive", "negative") to social media posts, or label character relationships and key topics for conversation records, which can greatly improve the retrievability and analysis efficiency of text data.

[0050] (2) Multi-level annotation unit

[0051] The multi-level annotation unit 12 is constructed based on the OpenCSG large model, and can annotate the text data by layer-by-layer analysis after semantic analysis.

[0052] The multi-level annotation unit 12 responds to different types of annotation requirements through a multi-level annotation strategy. For simple text data that is structured, semantically clear, or has strong regularity, the system can directly apply the output results generated by the OpenCSG large model for annotation. The large model has learned rich semantics and contextual relationships through massive training data, so when faced with simple annotation tasks, the model can quickly identify key semantics and automatically assign appropriate labels. This method greatly improves annotation efficiency and reduces human intervention.

[0053] When faced with complex text data (such as text with rich content and frequent polysemous words), a single-level annotation strategy may not be enough to ensure the accuracy of the results. To solve this problem, the multi-level annotation unit 12 gradually deepens into different semantic levels of the text through a layer-by-layer parsing strategy:

[0054] The first layer: macro semantic analysis. At this stage, the system first performs overall macro semantic analysis through a large model. Starting from the global semantics of the text, it captures core information, such as the theme, main emotional tendency, or the overall intention of the text. Macro semantic analysis can provide global guidance for subsequent refined annotations and help the system maintain consistency when processing details.

[0055] The second layer: detail-level annotation. After completing the macro-analysis, the system begins to go deeper into the detailed level to analyze the specific semantics of each sentence, phrase, and even word. This process may include identifying entities in the text (such as names of people, places, and organizations), determining the contextual meaning of polysemous words, and analyzing the grammatical structure in complex sentences. Through this layer-by-layer analysis method, the system can ensure that the annotation of each detail conforms to the logic of the global semantics while ensuring the accuracy of the results.

[0056] The multi-level annotation strategy not only improves the annotation quality of complex text data through progressive analysis, but also greatly reduces the inconsistency caused by manual participation. The annotation results of each level are mutually verified and optimized, and the final output annotation data has high consistency and accuracy in semantic understanding and detail processing.

[0057] The following is an explanation of the detailed analysis:

[0058] 1. Word-level tagging

[0059] The model first needs to segment the text, identify each word, and perform corresponding semantic annotations. For example, when dealing with entity recognition tasks, the model will identify proper nouns in the text (such as names of people, places, and organizations) and assign appropriate labels to them. This level of annotation also includes the distinction between polysemous words. The model will determine the meaning of a word in a specific scenario based on the context. For example, the word "bank" can mean a financial institution, and the model needs to select the correct meaning based on the context.

[0060] 2. Phrase and sentence level annotation

[0061] On the basis of word-level annotation, the model further analyzes the relationship between words to identify the semantic structure of phrases or entire sentences. For example, the model may identify noun phrases, verb phrases, etc., and annotate them according to the grammatical function and context of the phrases. This stage also includes grammatical parsing, identifying sentence components such as subjects, objects, modifiers, and judging their relationship with each other. For example, when processing a complex sentence, the model needs to annotate the different parts of the sentence and identify the logical relationship between the subordinate clause and the main clause.

[0062] 3. Context-dependent annotation

[0063] One of the important features of detail-level annotation is the need to process contextual information. The model analyzes the position of the target word, phrase, or sentence in the entire text to determine its semantic function. For example, in sentiment analysis tasks, the model needs to determine the sentiment of a phrase based on the context. For example, "Although the movie is long, it is very exciting." The model will recognize the transition word "although" and understand that the overall sentiment of the sentence is positive.

[0064] 4. Semantic Role Labeling

[0065] Semantic role labeling is one of the important steps in detail-level labeling, which means that the model identifies the roles of various elements in a sentence (such as actions, participants, time, place, etc.). For example, in the sentence "Xiao Ming plays football in the park", the model will label "Xiao Ming" as the performer of the action, "football" as the object of the action, and "in the park" as the place where the action takes place.

[0066] 5. Ambiguity resolution

[0067] In complex texts, there are often polysemous words or syntactic ambiguities. Detailed level annotation requires the model to resolve these ambiguities through context.

[0068] 6. Entity and Relationship Annotation

[0069] In detail annotation, the model also needs to identify entities in the text (such as names of people, places, time, etc.) and annotate the relationship between them. For example, in the sentence "Company A is headquartered in Beijing", the model needs to annotate that "A" is a company name and "Beijing" is a place name, and there is a "located in" relationship between the two.

[0070] 7. Output tags

[0071] After all the detailed analysis is completed, the model will integrate the annotation information into structured data output. For example, in sentiment analysis tasks, each sentence or paragraph may be assigned a sentiment label (such as "positive", "negative" or "neutral"); in entity recognition tasks, entities will be labeled as "person name", "place name", "institution name", etc.

[0072] (3) Dynamic Adaptive Unit

[0073] The dynamic adaptation unit 13 can automatically adjust the parameters and annotation strategies of the model according to the characteristics of the data and the requirements of the annotation task. For example, when processing long texts, the model can automatically segment the text and annotate it separately, and then integrate the annotation results of each part; for words with multiple contextual meanings, the model can select the most appropriate annotation based on the context.

[0074] 1. Adjust model parameters according to text length

[0075] When processing longer texts, the model first detects that the text length exceeds the processing capacity. Then, it automatically divides the text into multiple logical paragraphs or sentence parts and annotates them one by one. This processing method ensures that the model will not make analysis errors due to information overload when processing long texts. After the annotation is completed, the system will integrate the annotation results of each paragraph to form a complete annotation output while ensuring the overall semantic consistency.

[0076] For long texts, the model needs to increase its internal memory capacity to retain the previous context information so that it can still understand the context after segmentation. For example, in long paragraphs involving sentiment analysis or topic transitions, the model will expand the context window to ensure sentiment coherence across sentences or paragraphs.

[0077] 2. Contextual judgment and dynamic selection of polysemous words

[0078] Words with multiple meanings or multiple contexts may have different annotations in different contexts. The model infers the exact meaning of the word in this specific context by analyzing the surrounding context, such as the subject, verb, related phrases, etc. For example, for the word "Java", the model will annotate it as a programming language in the "programming" context, but may annotate it as a geographical location in the "travel" context.

[0079] To ensure accuracy, the model adjusts word weights based on context, giving priority to keywords in specific contexts. For example, the word "window" in architectural text tends to mean "house structure", while in scientific text it may mean "scope".

[0080] 3. Task-driven annotation strategy optimization

[0081] Different labeling tasks may have different requirements for details and overall accuracy. Sentiment analysis tasks tend to be roughly divided into "positive", "negative" or "neutral", while entity recognition tasks require higher precision to label sub-labels such as people and places. The dynamic adaptation unit 13 will select a more appropriate model level and strategy based on the task type, such as using shallow labeling in sentiment analysis and enabling deep labeling in complex semantic analysis.

[0082] When the task requirements change, the model will automatically increase or decrease the depth of analysis. For example, in advanced sentiment analysis tasks, the model can pay attention to the tone and emotional fluctuations of the sentence during the annotation process, automatically enhancing the sensitivity of sentiment annotation; in simple keyword extraction tasks, the model will reduce unnecessary complex analysis and optimize speed.

[0083] 4. Feature recognition and adaptive parameter fine-tuning

[0084] When processing text with a specific frequency distribution or grammatical structure, the model will automatically fine-tune its specific parameters. For example, for documents with a lot of technical terms, the model might adjust its vocabulary expansion to assign more weight to technical terms.

[0085] When faced with tasks with diverse data sources (such as social media, press releases, academic papers, etc.), the model will dynamically adapt to the tone and structure of different sources. The system can automatically select the appropriate corpus pre-trained model to better match the language characteristics of the input data.

[0086] 2. Intelligent Labeling Module

[0087] The intelligent annotation module 2 is connected to the automatic annotation module 1 in communication, and can provide annotation suggestions based on the OpenCSG large model, as well as detect the annotation results and generate annotation correction solutions, such as Figure 3 As shown, it includes a labeling suggestion unit 21, an intelligent error correction unit 22 and an interactive labeling unit 23.

[0088] (1) Labeling suggestion unit

[0089] The annotation suggestion unit 21 can output annotation suggestions for the current text data based on the historical annotation data and the context information of the current text data. For example, when the system detects that the text contains certain keywords, it will automatically suggest related tags and prompt possible context associations to help the annotator make decisions quickly.

[0090] 1. Matching and analysis of historical annotation data

[0091] Keyword and pattern matching: The tag suggestion unit 21 first searches for similar keywords, phrases, and contextual patterns in the historical tag data. For example, if the system detects that the current text contains similar keywords or sentence structures, it will refer to the tags that were annotated for similar texts in the past.

[0092] Data weight adjustment: The system sets weights for the quality and frequency of annotations in historical annotation data. Frequently used or highly accurate historical annotation data are given higher weights to ensure that the recommended labels are not only accurate but also meet the contextual needs.

[0093] Synonym and polysemy recognition: This unit can recognize synonyms and polysemy related to the current keyword and select the most commonly used annotation tags based on historical data. In this way, even if the current text and historical annotations have slight differences in specific vocabulary, the system can generate accurate annotation suggestions.

[0094] 2. Extraction and integration of contextual information

[0095] Contextual relevance analysis: The annotation suggestion unit 21 extracts key contextual information from the text, such as sentiment, sentence style, and target topic. The system conducts a comprehensive analysis of the contextual information to ensure that when providing annotation suggestions, it not only considers the words themselves, but also understands the overall context of the text. For example, when the word "bank" is identified, the system will look around it to see if it contains financial terms such as "deposit" and "interest" to confirm the financial meaning of the word in the current context.

[0096] Syntactic dependency parsing: The system uses syntactic parsing to understand the relationship between the components of a sentence, such as the subject-verb-object structure. This allows the system to identify entities or objects related to keywords in the text, making annotation suggestions more targeted. For example, when the text involves "purchasing goods", the system will identify the specific object of the purchase behavior and generate detailed annotation tags.

[0097] 3. Generation and prompting of annotation suggestions

[0098] Tag recommendation and confidence score: Based on historical data and context information, the tag suggestion unit 21 will give recommended tags and assign a confidence score to each tag. The confidence score is based on the consistency of historical tags and the matching degree of the current context, helping the annotator to judge the accuracy of the suggestion more intuitively.

[0099] Tag relationship prompts: While providing tagging suggestions, the system will also give relevant contextual prompts to help the annotator quickly understand the recommended tags. For example, if the system recognizes the tag "commercial activities", it may automatically prompt that the context contains related words such as "transaction" and "contract", further helping the annotator to judge the accuracy of the tag.

[0100] 4. Real-time feedback and adaptive optimization

[0101] Annotation feedback and continuous optimization: The annotation suggestion unit 21 will record the annotator's choices during the annotation process, identify the annotation preferences and adjustment tendencies. Then, the system continuously optimizes and adjusts the logic of tag recommendation by analyzing the annotation feedback. For example, if an annotator selects a specific tag multiple times, the system will increase the recommendation probability of the tag in similar texts.

[0102] (2) Intelligent Error Correction Unit

[0103] The intelligent error correction unit 22 is communicatively connected with the semantic analysis unit 11 and the multi-level annotation unit 12, and is used to receive the annotation information output by the semantic analysis unit 11 and / or the multi-level annotation unit 12, and detect the input information of the annotation personnel based on the annotation information, and generate prompt information when errors are detected in the input information, and the prompt information includes the annotation correction plan.

[0104] During the labeling process, the system will monitor the labeler's input in real time and compare it with the prediction results of the large model. If it detects that there may be errors or inconsistencies in the labeling, the system will automatically issue a reminder and propose corresponding correction plans. This method can effectively reduce human errors and improve the consistency of labeling.

[0105] 1. Real-time detection of annotation errors

[0106] Multiple verification mechanism: The intelligent error correction unit 22 will integrate the semantic analysis and multi-level annotation results to establish a multiple verification mechanism. For example, when it is detected that the label selected by the annotator is seriously inconsistent with the semantic analysis result, the system will mark the potential error.

[0107] Anomaly detection algorithm: The intelligent error correction unit 22 can use anomaly detection algorithms to identify abnormal annotations in the annotation data that do not conform to common patterns. For example, in a sentiment classification task, if a text is annotated with a "positive" sentiment but contains obvious negative words, the system will detect this mismatch and issue a prompt.

[0108] 2. Generation of annotation correction scheme

[0109] Correction suggestions based on model predictions: The system compares the input of the annotator with the best label predicted by the model. If a mismatch is found, the system will suggest using the label recommended by the model. For example, in sentiment classification, if the system predicts the label to be "negative" and the annotator labels it as "positive", the intelligent error correction unit 22 will suggest correcting it to "negative".

[0110] Reference to historical annotation data: The intelligent error correction unit 22 will also compare historical annotations of similar data to find the label distribution of similar texts. If the annotator's choice deviates significantly from the common historical annotations, the system will recommend using more common labels to increase the accuracy and consistency of the annotations.

[0111] 3. Contextual Reminders

[0112] Providing contextual correction evidence: When generating a correction plan, the intelligent error correction unit 22 will add contextual prompts. For example, when the system detects inconsistencies in specific domain terms or emotional expressions, the intelligent error correction unit 22 will display relevant contextual evidence (such as keywords and related words) to support the correction suggestion and help the annotator judge the rationality of the correction.

[0113] Multi-level annotation decomposition: For more complex texts, the intelligent error correction unit 22 can decompose the suggestions into specific annotation steps. For example, in subject classification, if the input contains multiple subject words and the annotation is ambiguous, the system can provide step-by-step classification suggestions to help the annotator make detailed judgments at multiple levels.

[0114] 4. Feedback and Adaptation of Error Correction Suggestions

[0115] Dynamic learning: The system records the annotator’s decision after each correction, so as to gradually optimize the accuracy of the recommendations. For example, if an annotator rejects a particular tag multiple times, the system will reduce the frequency of recommending that tag in similar texts, thereby improving the adaptability of the annotation.

[0116] Continuously optimize model parameters: By recording the data of each error correction, the intelligent error correction unit 22 continuously adjusts the parameter settings of the model, so that the system can better adapt to the needs of the labelers and data changes, and improve the overall labeling accuracy and consistency of the system.

[0117] (3) Interactive annotation unit

[0118] The interactive annotation unit 23 provides an intuitive interactive interface for annotation personnel, and the annotation personnel can quickly complete the annotation task through simple operations such as dragging, clicking, etc. In addition, the system interface also supports the import and export of multiple data formats, which is convenient for annotation personnel to exchange and share data between different platforms.

[0119] 1. Simple drag and click operation

[0120] Drag-and-drop annotation function: The interface allows annotators to quickly annotate by dragging text or image objects to the corresponding label area. For example, in an image classification task, annotators can drag and drop images to predefined categories to complete label assignment. For text annotation, annotators can select specific text fragments by dragging and directly associate them with labels.

[0121] Click-to-select and batch operations: The system provides a convenient click function that allows annotators to quickly mark or unmark content in a large amount of data. In addition, the batch selection function can achieve one-click annotation, which is very suitable for repetitive annotation tasks.

[0122] 2. Import and export of multiple data formats

[0123] Support for multi-format import: The system is compatible with a variety of data formats, including text (such as CSV, TXT), images (such as JPEG, PNG) and video formats (such as MP4), enabling annotators to process a variety of data sources. The interface automatically identifies the data format and loads it accordingly so that annotators can quickly start the task.

[0124] Data export and cross-platform sharing: After the annotation is completed, the system supports exporting the annotation results to common formats such as CSV, JSON, and XML, which makes it easier for users to apply the data to different analysis platforms and machine learning models. At the same time, annotators can also quickly exchange data between multiple platforms for further analysis or model training.

[0125] 3. Task Management Assignment Module

[0126] The task management and assignment module 3 is in communication with the intelligent annotation module 2, and can assign the annotation task to one or more annotation personnel and integrate the annotation results, such as Figure 4 As shown, it includes a task allocation unit 31, a real-time progress tracking unit 32 and a task coordination unit 33.

[0127] (1) Task allocation unit

[0128] The task assignment unit 31 automatically assigns the labeling task to the most suitable labeling personnel based on the preset machine learning algorithm and the information of the labeling task and the labeling personnel.

[0129] The system can intelligently assign labeling tasks based on the amount of data, the complexity of the labeling task, and the skill level of the labeler. The task assignment unit 31 prioritizes the labeling tasks according to preset rules or machine learning algorithms, and automatically assigns the tasks to the most suitable labelers to ensure efficient completion of the tasks.

[0130] 1. Task and personnel information collection

[0131] Labeling task information: The system first collects and analyzes key information about the labeling task, including data volume, task complexity, labeling type, and completion deadline. This information helps assess the difficulty of the task and determine the priority.

[0132] Personnel information database: The system establishes a database of labeled personnel to record their skill level, work efficiency, task completion and other relevant indicators. The system will also continuously update this information based on historical task data and feedback to maintain data accuracy and real-time.

[0133] 2. Task Prioritization

[0134] The system prioritizes tasks based on task complexity, data volume, task importance, and urgency. Higher priority tasks are assigned first to ensure critical tasks are completed on time. The sorting algorithm takes into account factors such as task deadlines and the impact of annotated tasks on the final project.

[0135] 3. Machine learning algorithms match people to tasks

[0136] Matching model: Through machine learning algorithms, such as KNN (K-nearest neighbor algorithm) or similarity-based clustering algorithms, the task allocation unit 31 matches the labeling task with the most qualified labeler, and comprehensively considers factors such as the labeler's skill level, task history, current workload, expertise and preferences.

[0137] Matching task complexity and skill level: For complex tasks, the system gives priority to assigning them to labelers with higher skill levels or more experience, while simple tasks can be assigned to relatively novice personnel.

[0138] 4. Dynamic adjustment and optimized allocation

[0139] The system will continue to track the progress and quality of the annotation during the task. If a task is progressing slowly or has quality problems, the task allocation unit 31 can reallocate the task based on real-time data. The algorithm will learn and record these situations to improve the accuracy of future task matching.

[0140] 5. Real-time feedback and self-learning mechanism

[0141] During the entire assignment process, the task assignment unit 31 collects feedback and annotation result data of each task and inputs it into the machine learning model. By analyzing the time of task completion, the performance of the annotator and the quality of task completion, the system can self-adjust and improve the accuracy and adaptability of the assignment strategy.

[0142] For example:

[0143] Task A is a complex sentiment analysis task. The task assignment unit 31 recognizes that it contains a lot of details and complex sentiment tendencies, and preferentially assigns the task to an annotator who is experienced in sentiment analysis.

[0144] Task B is a simple keyword labeling task, which is suitable for labelers with lower skills. The task allocation unit 31 will give priority to assigning it to novice labelers.

[0145] This task allocation process helps optimize resource allocation, improve efficiency and ensure high-quality task completion.

[0146] (2) Real-time progress tracking unit

[0147] The task allocation unit 31 can monitor the completion of each annotation task and generate real-time reports. The annotation administrator can check the task progress at any time and adjust the task allocation strategy to ensure that the entire annotation project is delivered on time.

[0148] 1. Task progress tracking and status updates

[0149] Each labeling task is assigned a unique ID and status tag (e.g., to be started, in progress, completed) when assigned. The real-time progress tracking unit 32 will regularly record the progress information of the labeling task and update the task status. The system dynamically updates and pushes the current progress status, including the completion percentage, remaining time, estimated completion time, etc., based on the task completion status of the labeler, to ensure that the latest status of all tasks is clear at a glance.

[0150] 2. Real-time report generation

[0151] The system can automatically generate real-time reports and summarize and subdivide them as needed to help administrators grasp the details of tasks. The report content usually includes key information such as task completion rate, progress of each labeler, task difficulty analysis, quality feedback, etc. Real-time reports can be exported to PDF, Excel and other formats, and can be shared among teams and used for progress reporting.

[0152] 3. Annotation task exception reminder

[0153] The real-time progress tracking unit 32 is provided with an abnormality monitoring function. When a task is stagnant for a long time or progresses abnormally slowly, the system will automatically send a reminder. For example, if a task does not reach the set progress within the expected time, the system will generate a notification to remind the administrator of possible task obstacles so that timely measures can be taken.

[0154] 4. Visual progress chart and interface

[0155] To help administrators quickly grasp the overall progress, the system provides intuitive charts and progress views, such as Gantt charts, bar charts, and progress bars. These charts visually mark different dimensions of the project (task completion rate, time progress, personnel distribution, etc.), allowing administrators to more intuitively observe the task status and make adjustments quickly.

[0156] 5. Dynamic adjustment of task allocation strategy

[0157] Based on the data provided by the real-time progress tracking unit 32, the administrator can dynamically adjust the task allocation strategy. For example, when the progress of some labelers is significantly behind, the system will suggest reallocating some tasks to labelers with faster progress, or provide resource support to ensure that the project is completed on time. By adjusting task allocation, the system improves labeling efficiency and reduces the risk of delays.

[0158] 6. Archiving and analysis of historical progress data

[0159] The real-time progress tracking unit 32 archives all task progress data to support analysis and improvement of future projects. By analyzing past annotated progress data, administrators can identify bottlenecks, best practices and efficient staffing methods, which will help optimize the efficiency of future projects.

[0160] (3) Task coordination unit

[0161] The task coordination unit 33 is configured to decompose a labeling task into multiple subtasks and assign them to multiple labelers, and to integrate the labeling results of all labelers.

[0162] The task coordination unit 33 supports multi-person collaborative labeling, and labelers can collaborate to complete complex labeling tasks. The task coordination unit 33 automatically coordinates task allocation and progress tracking to avoid task conflicts, and integrates the labeling results of each part after the task is completed.

[0163] 1. Decomposition and allocation of labeling tasks

[0164] When a large annotation task is received, the task coordination unit 33 analyzes the specific content, scale and difficulty of the task and decomposes it into multiple manageable subtasks. This process takes into account the structure, paragraphs and themes of the text, splits the parts with independent content into subtasks, and assigns them to the most suitable annotators based on their skill level, workload, historical performance and other factors to improve work efficiency and ensure annotation quality.

[0165] 2. Seamless connection between subtasks

[0166] During the assignment process, the system ensures that the subtasks maintain logical coherence. The task coordination unit 33 creates task dependencies, for example, some subtasks can only be performed after other tasks are completed, to ensure consistent annotation results. For example, if a project involves multiple texts with closely related contexts, the task coordination unit 33 will assign these texts to the same annotator or team to reduce the understanding bias between different people.

[0167] 3. Real-time progress tracking and conflict management

[0168] The task coordination unit 33 works in conjunction with the real-time progress tracking unit 32 to coordinate the progress of tasks by monitoring the task progress of each annotator. During the task allocation process, the task coordination unit 33 will automatically adjust the task allocation or priority based on data such as work progress and personnel schedule to avoid task conflicts and duplication of work. In addition, the task coordination unit 33 will identify and resolve potential resource conflicts. For example, when multiple people try to edit the same text paragraph, the system will automatically assign independent editing permissions or suggest task rescheduling.

[0169] 4. Integration of annotation results

[0170] After each task is completed, the task coordination unit 33 will automatically collect and integrate the annotation results of all annotators. The task coordination unit 33 will perform a preliminary logical check on the integrated results to ensure that the results of different annotators are consistent in content and format. If there are any inconsistencies, the task coordination unit 33 will generate a report for the administrator to review. In addition, the task coordination unit 33 has an automated conflict resolution function that can mark conflicting labels and opinions so that the administrator can quickly check and unify the annotation results.

[0171] 5. Multiplayer collaboration mode and task synchronization

[0172] Annotation projects usually involve teamwork. The task coordination unit 33 supports multiple people to participate in the same annotation task in real time, and provides message reminders, task synchronization and annotation functions to facilitate communication and feedback between annotators. For example, an annotator can mark the annotation of a paragraph in the system interface and provide annotation suggestions for other people. The task coordination unit 33 can update the project status in real time according to the collaboration situation, and supports cross-platform sharing of tasks to ensure efficient and smooth teamwork.

[0173] 6.Automated aggregation and version management

[0174] To avoid task conflicts and ensure the accuracy of results, the system also manages the version of each subtask. The task coordination unit 33 will store all integrated annotation results in versions, support backtracking and restoring previous versions, and ensure that they can be restored at any time when problems occur. Automated version control also allows administrators to switch and compare between versions and select the best results to retain and apply.

[0175] 4. Automated quality control module

[0176] The automated quality control module 4 is in communication with the task management and allocation module 3, and can review the labeling results and automatically correct the detected labeling errors, such as Figure 5 As shown, it includes a quality review unit 41, an automatic correction unit 42 and a version control and result optimization unit 43.

[0177] (1) Quality Audit Unit

[0178] The quality review unit 41 is configured to perform a primary review on the annotation results, detect annotation errors, and, after completing the primary review, perform a secondary review on the annotation results according to the quality standard information of the annotation task, so that the annotation results meet the quality standards.

[0179] The quality audit unit 41 is designed with a multi-level audit mechanism. First, the labeling results are automatically reviewed to detect common labeling errors or inconsistencies. Secondly, according to the preset quality standards, the labeling results are reviewed in detail, including the accuracy and consistency of the labels, and whether the labeling results meet expectations.

[0180] 1. Primary review (automatically detect common annotation errors)

[0181] Automatic tagging error detection: In the preliminary review stage, the semantic analysis function of the large model is used to detect common errors and inconsistencies in tagging, such as mismatched labels, missing labels, and repeated labels. According to the preset rules or the established error detection model, these problems are automatically identified and a preliminary review report is generated to mark the problematic parts. For example, in sentiment analysis tagging, sentences with inconsistent sentiment classification are automatically identified and marked as items for review.

[0182] Consistency check: The primary audit also includes checking the consistency of the annotation results to ensure that the style, format, and labels of the annotation results meet the project standards and avoid differences in the presentation of the annotation results. Compare the formats of all annotation results to check whether the annotators follow the unified standards.

[0183] 2. Secondary audit (detailed audit based on quality standards)

[0184] In-depth content review: After completing the primary review, a more detailed secondary review of the annotation results is conducted based on the quality standards of the annotation task. At this stage, the annotation results are checked to see if they are accurate and meet expectations according to the specific requirements of each annotation task. For example, in the topic classification annotation, the annotation results are compared in depth with the standard labels to ensure that all categories are accurate and not missed.

[0185] Manual review and intelligent error correction: For some complex annotation results, manual review is prompted to participate in the review. The annotator or quality auditor can further review and confirm the annotation errors detected by the system. The intelligent error correction function will generate a correction plan based on the review opinions to assist the annotator to quickly adjust the annotation content and reduce unnecessary rework.

[0186] Annotation quality score: Generates a quality score for annotation results, scoring the accuracy and consistency of each annotation item according to the quality standard. The final score of the annotation result can be used as an important indicator for annotation quality assessment to help administrators screen qualified annotation content.

[0187] 3. Result feedback and continuous optimization

[0188] Multiple iterations and feedback mechanisms: The quality review unit 41 designed a feedback loop to feedback problems to the annotators after the review is completed and make improvements in subsequent tasks. All review processes and problem labels are recorded, and the model's understanding of annotations and error detection capabilities are optimized through a continuous learning mechanism.

[0189] Version control and standard updates: The quality audit unit 41 performs version management on all annotation results, allowing auditors to go back to earlier annotation versions when necessary. Quality standards are updated according to the latest project requirements to ensure that the system always meets the specific requirements of different annotation tasks.

[0190] (2) Automatic correction unit

[0191] The automatic correction unit 42 is in communication connection with the quality review unit 41, and can automatically correct the marking errors detected during the primary review and the secondary review, and replace the marking result with the correction result after receiving the confirmation signal from the marking personnel.

[0192] During the review process, the automatic correction unit 42 automatically corrects common labeling errors, such as mismatched labels, missing labels, duplicate labels, etc., and submits the corrected results to the labeler for confirmation. This intelligent error correction mechanism can significantly reduce human errors and improve labeling quality.

[0193] 1. Error detection and preliminary diagnosis

[0194] Error type identification: The automatic correction unit 42 first identifies the error type, such as label mismatch, missing label or repeated label, etc. According to the error report detected by the quality audit unit 41, the automatic correction unit 42 uses the OpenCSG large model and the predefined error pattern library to quickly screen out the errors in the labeling.

[0195] Context matching check: In the case of mismatched labels, the system will check whether the labels match the text content. Through the semantic analysis unit 11, the automatic correction unit 42 can verify the context of the annotation to ensure that the semantics of the label and the text are consistent and avoid irrelevant or irrelevant labels.

[0196] 2. Intelligent recommendation of correction solutions

[0197] Model automatic labeling suggestions: The automatic correction unit 42 automatically generates correction suggestions based on the prediction results of the large model, such as regenerating reasonable label recommendations in the case of mismatched labels. If the system detects missing labels, the automatic correction unit 42 will prompt and fill in the missing labels, while ensuring that the correction content meets the context requirements.

[0198] Correction using historical data: The automatic correction unit 42 also refers to the annotated historical data of similar texts to generate tag suggestions that better meet project requirements. By analyzing the annotated historical data, the automatic correction unit 42 can improve the accuracy and consistency of the correction results.

[0199] 3. Intelligent automatic correction

[0200] Label replacement and completion: For confirmed label mismatches or missing labels, the system automatically replaces inappropriate labels or adds missing labels. Over-labeled or duplicated labels are automatically removed to ensure data simplicity and consistency.

[0201] Semantic depth adjustment: For texts with multiple semantic levels, the automatic correction unit 42 can also adjust the details of the label according to the requirements of the annotation task. For example, in the sentiment analysis task, the automatic correction unit 42 will adjust the strength of the label according to the context (such as changing "positive" to "very positive").

[0202] 4. Confirmation by the labeler and final application

[0203] Interactive confirmation interface: After the automatic correction, the automatic correction unit 42 sends the correction suggestion to the annotator for confirmation. The annotator can view the correction suggestion through the intuitive interface, accept or modify the correction content, and avoid system misjudgment.

[0204] Final replacement and version control: Once the annotator confirms the correction result, the system will automatically replace the correction content with the official annotation result and save it to the latest version of the task. The automatic correction unit 42 also retains the annotation version of each stage for easy traceability and review, ensuring the transparency and compliance of the annotation process.

[0205] (3) Version control and result optimization unit

[0206] The version control and result optimization unit 43 supports version control of annotation results. Annotators can view historical versions and roll back or re-annotate as needed. The unit also has a built-in result optimization function, which generates the best annotation results by analyzing multiple annotation versions to ensure data accuracy and consistency.

[0207] 1. Version control mechanism

[0208] Multi-version storage: After each annotation is completed or revised, a version copy of the annotation is automatically generated and stored. In this way, annotators can view the annotation history at different time points at any time and track the changes and adjustment processes of each version. The multi-version storage provided by the system ensures that the modification traces of each annotation result are fully recorded.

[0209] Rollback and recovery of historical versions: Labelers can choose to roll back to previous versions as needed. Especially when the labeling is wrong, deviated, or does not meet expectations, the rollback function can quickly restore to the appropriate version state to avoid duplication of work. This version management not only saves time, but also reduces the impact of human errors on the overall data.

[0210] 2. Result optimization function

[0211] Automated optimization strategy: The version control and result optimization unit 43 has a built-in intelligent optimization algorithm that can automatically analyze the annotation differences between different versions and extract and merge the best annotation content. For example, when multiple annotators have inconsistent annotations on the same text, the system will compare and integrate the labels of different versions to generate an optimal annotation result that meets the standards.

[0212] Multi-level data fusion: The system uses a multi-level analysis method to combine the best labels, semantic information, and contextual clues from each version to generate the most accurate and consistent final annotations. For example, in sentiment classification, the system will detect the similarity of labels from each version and take the label with the highest frequency and the most consistent context as the final result.

[0213] Adaptive feedback optimization: The system continuously records user feedback during the optimization process and adaptively adjusts the optimization strategy. For example, if annotators select a specific type of correction multiple times, the system will incorporate this feedback into the optimization model to enhance the automatic recognition and optimization capabilities of similar annotations.

[0214] 3. Visualization and auxiliary analysis tools

[0215] Version difference comparison: The system provides an intuitive visual interface that displays the annotation content of different versions side by side, allowing annotators to easily view the differences between each version. Visualization tools can efficiently display the subtle differences between multiple versions, helping annotators quickly locate problems or confirm the optimal results.

[0216] Historical track and change log: Each version of the annotation modification operation is recorded in detail in the form of a log, including the modification content, time, and executor information, making the entire annotation process transparent and standardized, and convenient for managers and annotators to check and review at any time.

[0217] 4. Ensure data accuracy and consistency

[0218] Consistency check: The system will perform consistency checks on the final annotation results to ensure that there are no redundancies, errors, omissions, or annotation conflicts, and to meet the standards for high-precision data output. Through multi-version fusion and optimization mechanisms, the system maintains a high degree of consistency and semantic coherence in the final annotation results.

[0219] Compliance audit: For specific tasks with quality standards, the system will add compliance audits to the version control and optimization process to ensure that the final annotation results meet the preset requirements. This not only improves the application value of the data, but also ensures efficient performance in model training and actual applications.

[0220] The OpenCSG large model intelligent data annotation system disclosed in the present invention also has a module collaborative working mechanism, which can realize the following application scenarios:

[0221] (1) Data transfer between modules: Efficient collaboration is achieved between modules through data transfer. For example, the initial annotation results generated by the automatic annotation module 1 are automatically transferred to the intelligent annotation module 2 for further optimization by the annotation staff; the optimized results are then transferred to the task management and allocation module 3 for review and management, and finally reviewed and corrected by the automated quality control module 4.

[0222] The data flow between modules achieves efficient collaboration through structured information transmission and module-level communication. Under this mechanism, data from input to the final output of the annotation process is modularized into a smooth and automated multi-stage processing flow. The specific process is as follows:

[0223] The automatic labeling module 1 first receives the original data and performs preliminary processing to generate basic labeling results, and passes them to the intelligent labeling module 2.

[0224] The intelligent annotation module 2 receives the initial results from the automatic annotation module 1 and provides them to the annotator as suggestions. The annotator can further optimize the annotation content based on the recommendations. After optimization, the system automatically transmits the data to the task management and allocation module 3.

[0225] The task management and allocation module 3 is responsible for tracking the status of the annotation task and ensuring that the annotation is carried out smoothly within the scope of the project. Whenever the annotation is completed, the task management and allocation module 3 sends the annotation content to the automated quality control module 4.

[0226] In the automated quality control module 4, the system conducts primary and in-depth audits to ensure that the annotation results meet the standards. Any errors that may exist in the annotations will be automatically corrected or annotated, and the final annotation results will be saved.

[0227] This data flow mode not only reduces repetitive manual operations, but also ensures that each annotation task undergoes multi-level review and optimization, improving the quality and consistency of annotation.

[0228] (2) Real-time feedback and self-learning: The system has a real-time feedback mechanism. Each module will continuously collect operational feedback from annotators during the working process, and gradually optimize the annotation strategy and model parameters through the self-learning mechanism. This closed-loop feedback mechanism enables the system to continuously evolve and improve annotation efficiency and quality.

[0229] The system has a real-time feedback mechanism. During the working process, each module will automatically record and analyze the labeler's operating habits, acceptance or rejection of suggestions and other information to optimize the labeling strategy and model parameters.

[0230] Feedback collection: Detailed data of each annotation operation, including the annotator’s modification, selection, and confirmation, will be recorded in real time. This data provides an important operational data source for system self-learning.

[0231] Self-learning and optimization: The system will conduct data mining on operation records, analyze the labelers' preferences and adjustment tendencies for different types of suggestions, and gradually optimize the decision logic of the model. For example, in sentiment analysis, the system can continuously adjust the parameters of the sentiment classification model based on feedback and select the most matching classification criteria.

[0232] Closed-loop feedback mechanism: By integrating feedback into model training and strategy optimization, the system implements a closed-loop feedback process, which can absorb new operating modes and dynamically adjust the operating strategies of each module in each iteration, allowing the system to evolve gradually.

[0233] This mechanism effectively improves the intelligence level of annotation, and the annotation efficiency and quality are gradually improved based on continuous self-learning.

[0234] (3) Expansion and integration: The system is designed with an open API interface to support integration with third-party data processing tools and annotation tools. Users can expand the system's functions as needed or integrate the system into existing data processing processes to further improve work efficiency.

[0235] To meet the needs of complex data processing scenarios, the system has designed an open API interface to support seamless integration with third-party tools, including:

[0236] API interface: Through open API, users can quickly connect the system with existing data processing tools, annotation tools or machine learning models to achieve data exchange and sharing. This integration allows users to smoothly call annotation services or processing results on different platforms.

[0237] Cross-platform collaboration: Supports integration of the system into existing data flows. Whether in local systems, the cloud, or other data processing environments, the system can be integrated with different data sources and processing flows through standardized interfaces, further improving work efficiency and flexibility.

[0238] Functional extension: The open interface also allows users to customize functions or extend modules. For example, in specific domain annotation, users can introduce domain-specific vocabulary libraries or domain-specific models, and integrate multiple modules through the system to obtain more accurate annotation results.

[0239] In the embodiment disclosed in the present invention, the annotated data is firstly processed by the large model driven automatic annotation module 1 to generate the initial annotation result, and then the intelligent annotation module 2 assists the annotator to complete the final annotation. The task management and allocation module 3 ensures that the annotation task is completed on time, and the annotation result is reviewed by the automated quality control module 4, and finally high-quality annotation data is generated.

[0240] The large model-driven automatic annotation module 1 is based on the advanced OpenCSG large model technology to directly extract information from the data and generate preliminary annotations. The core of the automatic annotation module 1 is to use the model's learning ability to automatically identify key entities, sentiment tendencies or other annotation requirements in the data. This module can not only significantly reduce the manual participation in annotation, but also gradually optimize its own performance through continuous training and adaptive learning, and improve the accuracy and consistency of initial annotations.

[0241] The intelligent labeling module 2 analyzes the contextual association between historical labeling data and current data, and intelligently generates labeling suggestions that are most relevant to the current task, thereby effectively reducing the manual operation time of labelers and ensuring the accuracy of the suggestions and the consistency of the labeling.

[0242] The task management and allocation module 3 is responsible for managing the entire labeling process, including task allocation, progress tracking, and resource management. The system automatically analyzes the complexity of the task, the amount of data, and the skill level of the labeler, and matches the task to the appropriate labeler. This module also provides real-time progress monitoring to ensure that the labeling task can be completed on time, thereby ensuring the efficient execution of the entire project. At the same time, the task management and allocation module 3 will record the completion status of each task to support the allocation and optimization of subsequent tasks.

[0243] At the last step of the labeling process, the labeling results will be automatically transmitted to the automated quality control module 4 for systematic quality review and correction. The automated quality control module 4 identifies and corrects common labeling errors, such as inconsistent labels, missing labels, and duplicate labels, through steps such as automatic preliminary review, detailed inspection, and correction suggestions. In addition, the automated quality control module 4 also conducts a secondary review of the data according to specific quality standards to ensure the accuracy and consistency of the labeled data. The reviewed data will be marked as the final result and saved to the system database for subsequent data analysis or model training.

[0244] When data flows through each module, the results of each step will be communicated and updated collaboratively between modules, so that the output of each module can automatically obtain optimized input information when entering the next step. This modular and automated process not only improves the labeling efficiency, but also ensures the high accuracy and consistency of data labeling, so that the system can ultimately output high-quality labeled data to meet the needs of different application scenarios.

[0245] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An OpenCSG large model intelligent data annotation system, characterized in that: The system is driven by the OpenCSG large model, including: The automatic annotation module, including a semantic analysis unit, a multi-level annotation unit, and a dynamic adaptation unit, is configured to be built based on the OpenCSG large model to perform semantic analysis and automatic annotation on the text data of the annotation task; An intelligent annotation module in communication with the automatic annotation module, including an annotation suggestion unit, an intelligent error correction unit and an interactive annotation unit, configured to provide annotation suggestions, detect the annotation results and generate an annotation correction plan; A task management and allocation module in communication with the intelligent labeling module, including a task allocation unit, a real-time progress tracking unit and a task coordination unit, configured to allocate labeling tasks to one or more labelers and integrate the labeling results; The automated quality control module, which is in communication with the task management allocation module, includes a quality review unit, an automatic correction unit, and a version control and result optimization unit, and is configured to review the annotation results and automatically correct detected annotation errors.

2. The intelligent data labeling system according to claim 1, characterized in that: The semantic analysis unit is constructed based on the OpenCSG large model, and is configured to extract semantic information, sentiment tendency and entity relationship in text data, and annotate the text data according to the extracted content.

3. The intelligent data labeling system according to claim 1, characterized in that: The multi-level annotation unit is constructed based on the OpenCSG large model and is configured to perform semantic analysis on the text data and then perform annotation in a layer-by-layer parsing manner.

4. The intelligent data labeling system according to claim 1, characterized in that: The dynamic adaptation unit is configured to automatically adjust the parameters and annotation strategy of the OpenCSG large model in a preset manner according to data characteristics and annotation task requirements.

5. The intelligent data labeling system according to claim 1, characterized in that: The annotation suggestion unit is configured to output annotation suggestions for current text data based on historical annotation data and context information of current text data.

6. The intelligent data labeling system according to claim 1, characterized in that: The intelligent error correction unit is respectively connected to the semantic analysis unit and the multi-level annotation unit for communication, and is configured to receive annotation information output by the semantic analysis unit and / or the multi-level annotation unit, and detect input information of the annotation personnel based on the annotation information, and generate prompt information when an error is detected in the input information, wherein the prompt information includes an annotation correction plan.

7. The intelligent data labeling system according to claim 1, characterized in that: The task assignment unit is configured to automatically assign the labeling task to the most matching labeling personnel based on a preset machine learning algorithm and information about the labeling task and the labeling personnel.

8. The intelligent data labeling system according to claim 1, characterized in that: The task coordination unit is configured to decompose a labeling task into multiple subtasks and assign the subtasks to multiple labelers, and to integrate the labeling results of all labelers.

9. The intelligent data labeling system according to claim 1, characterized in that: The quality review unit is configured to perform a primary review on the annotation results, detect annotation errors, and, after completing the primary review, perform a secondary review on the annotation results according to the quality standard information of the annotation task, so that the annotation results meet the quality standards.

10. The intelligent data labeling system according to claim 1, characterized in that: The automatic correction unit is in communication with the quality review unit, and is configured to automatically correct the labeling errors detected during the primary review and the secondary review, and to replace the labeling result with the correction result after receiving a confirmation signal from the labeler.

Images