Intelligent data labeling method, system, electronic device and storage medium

Abstract

The present invention discloses an intelligent data annotation method, system, electronic device, and storage medium. The method includes: matching and loading a pre-trained deep learning model based on fabric type and surface characteristics, including texture, color, and defect type, and preset model setting conditions, wherein the fabric type and surface characteristics include recognition accuracy requirements and computing resource constraints; inputting the image to be annotated into the deep learning model, and generating preliminary annotation results based on a preset confidence threshold; receiving user operation information on the preliminary annotation results, adjusting the preliminary annotation results based on the operation information to obtain an annotation result, and adjusting the deep learning model based on the operation information, wherein the operation information includes deleting, modifying categories, and adjusting bounding boxes; and generating an annotation file based on the annotation results. The technical solutions provided by the embodiments of the present application effectively improve the accuracy and efficiency of data annotation.

Description

Technical Field

[0001] The present application relates to the technical field of data labeling, and specifically to an intelligent data labeling method, system, electronic device, and storage medium. Background Art

[0002] Data annotation technology plays a crucial role in the field of artificial intelligence. Its primary task is to classify, label, or annotate data such as images, text, or audio for subsequent machine learning model training. With the rapid development of deep learning technology, high-quality data annotation has become a key factor in improving model performance. Data annotation not only directly affects model accuracy but also largely determines its generalization ability. Therefore, it has garnered widespread attention and application in both industry and academia.

[0003] At present, in order to achieve efficient data labeling, the industry usually adopts manual labeling or rule-based automatic labeling. Manual labeling is the most traditional method, in which professionals manually label the data one by one. Although the labeling accuracy is high, it is inefficient and costly. The rule-based automatic labeling method automatically completes labeling through pre-set rules. It is suitable for specific scenarios, but it has poor flexibility and is difficult to adapt to complex and diverse data types. Existing data labeling methods often find it difficult to meet high-precision requirements when processing pictures of fabrics with complex textures, diverse colors, and defects. At the same time, they are limited by computing resources, resulting in reduced labeling accuracy or low labeling efficiency.

[0004] Therefore, there is an urgent need for an efficient data annotation method that can adapt to various scenarios. Summary of the Invention

[0005] This application provides an intelligent data labeling method, system, electronic device and storage medium. By combining user interaction and model adaptive adjustment, it effectively improves the accuracy and efficiency of data labeling, while reducing the cost of manual intervention. It is suitable for data labeling tasks in complex scenarios.

[0006] In a first aspect of the present application, an intelligent data annotation method is provided, which is applied to a data annotation platform. The method includes:

[0007] Matching and loading a pre-trained deep learning model based on fabric type and surface characteristics, including texture, color, and defect type, and preset model setting conditions, including recognition accuracy requirements and computing resource limitations;

[0008] Input the image to be annotated into the deep learning model and generate preliminary annotation results based on a preset confidence threshold;

[0009] Receive user operation information on the preliminary annotation result, adjust the preliminary annotation result according to the operation information to obtain an annotation result, and adjust the deep learning model according to the operation information, wherein the operation information includes deleting, modifying categories, and adjusting bounding boxes;

[0010] Generate a marking file according to the marking result.

[0011] Optionally, adjusting the preliminary annotation result according to the operation information to obtain the annotation result includes:

[0012] When the operation information is deletion, changing the border color of the original annotation box in the preliminary annotation result to a preset first color;

[0013] When the operation information is a category, the original category in the preliminary annotation result is replaced with the new category, and the border color of the original annotation box in the preliminary annotation result is changed to a preset second color;

[0014] When the operation information is editing, the boundary box of the marked area is redrawn according to the trajectory generated by the user dragging the mouse, and the border color of the original marked box in the preliminary marking result is changed to a preset third color.

[0015] Optionally, adjusting the deep learning model according to the operation information includes:

[0016] When it is detected that the user performs a deletion operation on the original annotation box, extracting the image feature vector corresponding to the original annotation box, and calculating the cosine similarity between the image feature vector and the weight vector of each category in the classifier false detection weight matrix in the deep learning model to determine the most similar category;

[0017] The weight value of the most similar category in the classifier false detection weight matrix is ​​reduced according to a preset ratio.

[0018] Optionally, adjusting the deep learning model according to the operation information includes:

[0019] When it is detected that the user modifies the category label of the original annotation box, a mapping relationship between the original category and the new category is obtained, and a feature map of the area corresponding to the original annotation box is extracted;

[0020] Inputting the feature map into an incremental learning module of a classifier, generating a first prototype vector of a new category through sliding window mean pooling, and calculating a difference vector between the first prototype vector and the second prototype vector of the original category;

[0021] The weight corresponding to the new category is determined based on the difference vector, the update rate parameter, and the weight corresponding to the original category.

[0022] Optionally, calculating the difference vector between the first prototype vector and the second prototype vector of the original category includes:

[0023] Calculating the cosine similarity and the Euclidean distance between the first prototype vector and the second prototype vector, and performing weighted fusion on the cosine similarity and the Euclidean distance to generate a comprehensive difference;

[0024] Determine a difference threshold based on the number of samples of the original category and the annotation confidence, and judge whether the comprehensive difference is greater than or equal to the difference threshold;

[0025] When the comprehensive difference is greater than or equal to the difference threshold, the comprehensive difference is mapped to a vector space to obtain the difference vector.

[0026] Optionally, determining the weight corresponding to the new category based on the difference vector, the update rate parameter, and the weight corresponding to the original category includes:

[0027] Mapping the difference vector to the weight space through a fully connected layer to generate a weight offset;

[0028] The update rate parameter is adjusted according to the user operation frequency and the convergence state of the deep learning model, the product of the weight offset and the update rate parameter is calculated, and the weight of the new category is obtained according to the product and the weight corresponding to the original category.

[0029] Optionally, adjusting the deep learning model according to the operation information includes:

[0030] When the user drags the vertex of the original annotation box, the mouse trajectory coordinate sequence is obtained and a new annotation box is generated using curve fitting;

[0031] Calculate a coordinate difference matrix between the original annotation box and the new annotation box, calculate a loss gradient based on the coordinate difference matrix, and adjust the deep learning model based on the loss gradient.

[0032] In a second aspect of the present application, an intelligent data annotation system is provided, comprising a setting module, a processing module, an annotation module, and an output module, wherein:

[0033] a settings module configured to match and load a pre-trained deep learning model based on fabric type and surface characteristics and preset model setting conditions, wherein the fabric type and surface characteristics include texture, color, and defect type, and the model setting conditions include recognition accuracy requirements and computing resource limitations;

[0034] A processing module configured to input the image to be annotated into the deep learning model and generate preliminary annotation results according to a preset confidence threshold;

[0035] a labeling module configured to receive user operation information on the preliminary labeling result, adjust the preliminary labeling result according to the operation information to obtain a labeling result, and adjust the deep learning model according to the operation information, the operation information including deleting, modifying categories, and adjusting bounding boxes;

[0036] The output module is configured to generate a marking file according to the marking result.

[0037] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device performs any of the methods described above.

[0038] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions. When the instructions are executed, any one of the methods described above is executed.

[0039] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0040] 1. Match and load pre-trained deep learning models based on fabric type and surface characteristics and preset model setting conditions. This makes the selected model highly adaptable to the labeling needs of specific fabric types and surfaces, improves the fit between the model and the labeling task, helps to more accurately identify fabric features, and lays a good foundation for subsequent labeling work; considering computing resource limitations, it can select the model that best suits the current resource situation while meeting the recognition accuracy requirements, avoiding resource waste, improving the operating efficiency of the labeling platform, and enabling the platform to work efficiently and stably under different hardware environments;

[0041] 2. Input the images to be annotated into the deep learning model, and leverage the model's powerful feature extraction and classification capabilities to quickly generate preliminary annotation results. Compared to traditional manual annotation methods, this greatly improves annotation speed and can process a large number of images in a short period of time, meeting the needs of large-scale data annotation. Preliminary annotation results are generated based on a preset confidence threshold, ensuring a certain degree of reliability. Only annotations whose model prediction confidence reaches the threshold are retained, reducing the number of incorrect annotations, improving the accuracy of preliminary annotations, and providing a good foundation for subsequent manual adjustments.

[0042] 3. Receive user operation information on the preliminary annotation results (deletion, modification of categories, and adjustment of bounding boxes), allowing users to make detailed adjustments to the annotation results based on actual conditions. This human-computer interaction method fully utilizes human subjective judgment and expertise in specific fields, can correct possible errors in the model, make the annotation results more in line with actual needs, and improve the accuracy and quality of annotation. Adjust the deep learning model according to the user's operation information to achieve dynamic learning and optimization of the model. As users continue to adjust the annotation results, the model can learn more accurate features and annotation rules, gradually improve its own performance and generalization ability, and generate more accurate preliminary annotation results in subsequent annotation tasks, forming a virtuous cycle;

[0043] 4. Generate annotation files based on the final annotation results, ensuring the standardization and consistency of the annotation data. Annotation files can use a universal format to facilitate subsequent data storage, management, and use. This facilitates model training and algorithm development based on this annotated data, helping to improve the efficiency and accuracy of the entire data processing and analysis process. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of the intelligent data annotation method disclosed in the embodiment of the present application;

[0045] Figure 2 It is a schematic diagram of the annotated picture disclosed in the embodiment of this application;

[0046] Figure 3 is a schematic diagram of the recognition results disclosed in the embodiment of this application;

[0047] Figure 4 This is a schematic diagram of the editing and annotation results disclosed in the embodiment of this application;

[0048] Figure 5 This is a schematic diagram of the modified threshold value disclosed in the embodiment of the present application;

[0049] Figure 6 It is a directly labeled initial schematic diagram disclosed in the embodiment of this application;

[0050] Figure 7 This is a schematic diagram of the direct marking process disclosed in the embodiment of this application;

[0051] Figure 8 This is a schematic diagram of the direct marking result disclosed in the embodiment of this application;

[0052] Figure 9 This is a module diagram of the intelligent data annotation system disclosed in the embodiment of the present application;

[0053] Figure 10 This is a structural diagram of an electronic device disclosed in an embodiment of the present application.

[0054] Explanation of the accompanying drawings: 701, setting module; 702, processing module; 703, annotation module; 704, output module; 801, processor; 802, communication bus; 803, user interface; 804, network interface; 805, memory. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0056] In the description of the embodiments of this application, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.

[0057] In the description of the embodiments of the present application, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0058] This embodiment discloses an intelligent data annotation method, which is applied to a data annotation platform. Figure 1 This is a flow chart of the intelligent data annotation method disclosed in the embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0059] S101. Matching and loading a pre-trained deep learning model based on fabric type and surface characteristics and preset model setting conditions, wherein the fabric type and surface characteristics include texture, color, and defect type, and the model setting conditions include recognition accuracy requirements and computing resource limitations;

[0060] S102: Input the image to be annotated into the deep learning model, and generate preliminary annotation results according to a preset confidence threshold;

[0061] S103, receiving user operation information on the preliminary annotation result, adjusting the preliminary annotation result according to the operation information to obtain an annotation result, and adjusting the deep learning model according to the operation information, wherein the operation information includes deleting, modifying a category, and adjusting a bounding box;

[0062] S104: Generate a marking file according to the marking result.

[0063] Different fabric types have unique texture characteristics, such as plain, twill, and satin. These textures exhibit distinct patterns and structures at both the micro and macro levels. For example, plain weave has a smooth surface and numerous warp and weft interlacing points, while twill has distinct diagonal lines. Deep learning models need to be able to recognize these texture differences to accurately distinguish between different fabric types. Fabric colors vary widely, from single solid colors to complex printed patterns. Color information is crucial for both fabric type identification and defect detection. For example, certain fabric types may have a specific color range, and color anomalies may indicate the presence of defects. Models need to be able to accurately perceive and process color information, including hue, saturation, and brightness. Fabric defects come in a wide variety of forms, such as holes, stains, yarn skipping, and color variations. Each defect has its own unique appearance and characteristics. Models need to be able to identify different defect types so that their location and type can be accurately marked during the annotation process. Recognition accuracy requirements: The required recognition accuracy of the model should be determined based on the actual application scenario and needs. For example, in high-end textile quality inspection, a model may require defect recognition accuracy exceeding 99% to ensure product quality. However, in applications with lower accuracy requirements, this accuracy can be appropriately lowered. Computing resource constraints: Consider the computing resources available on the annotation platform, such as the performance and number of CPUs and GPUs, and memory capacity. When selecting a pre-trained model, ensure that the model can run efficiently within the available computing resources. If computing resources are limited, a model with a smaller number of parameters and lower computational complexity may be appropriate to ensure real-time and stable annotation. Based on the fabric type and surface characteristics and model setup requirements, select a suitable pre-trained deep learning model from the model library. Models in the library may be pre-trained for different fabric types, defect types, and accuracy requirements. Once a matching model is found, it is loaded into the data annotation platform's runtime environment to prepare for subsequent image annotation. The fabric image to be annotated is then fed into the loaded deep learning model. Images may be stored in various formats (such as JPEG, PNG, etc.), and pre-processing such as resizing and normalizing pixel values ​​may be required before feeding them into the model to ensure they meet the model's input requirements. The deep learning model extracts and analyzes features from the input image. Using a multi-layered neural network structure, the model automatically learns characteristic patterns in the image and identifies information such as fabric type, color, and defects. For example, convolutional neural networks have powerful feature extraction capabilities in image processing, effectively capturing texture, edges, and other features in images. Based on the learned characteristic patterns, the model predicts objects in the image (such as fabric type and defect location) and assigns a confidence score to each prediction. The confidence score indicates the reliability of the model's predictions and typically ranges from 0 to 1. A preset confidence threshold is used; only when the model's prediction confidence exceeds this threshold is the prediction output as a preliminary annotation result.For example, if the confidence threshold is set to 0.8, only predictions with a confidence level greater than 0.8 will be retained. Preliminary annotation results may include information such as the object's category (e.g., defect type), location, and bounding box. Users can view these preliminary annotation results on the data annotation platform and perform operations based on their needs. Operations include deleting, modifying the category, and adjusting the bounding box. Deleting: If a user deems a preliminary annotation result to be incorrect or unnecessary, they can delete it. Modifying the category: If the model's prediction of the object's category is inaccurate, the user can manually modify the annotation category. For example, a defect misclassified as a "stain" can be modified to a "hole." Adjusting the bounding box: The bounding box predicted by the model may not be accurate. Users can adjust its position and size by dragging its corners or edges to more accurately enclose the object. Based on the user's operations, the preliminary annotation results are adjusted accordingly to produce the final annotation result. The adjusted annotation result is more consistent with the actual situation and accurately reflects the information in the fabric image. User operations reflect any deficiencies in the model's annotation process. This operation information is used to adjust and optimize the deep learning model. Specific methods may include: using user-adjusted annotated data as new training samples to incrementally train the model, enabling it to learn more accurate features and annotation rules; adjusting model parameters, such as weights and biases, based on user feedback to improve model performance; and continuously adjusting the model to better adapt it to the task of labeling fabric types and surfaces, thereby improving the accuracy and efficiency of subsequent labeling. The final annotation results are collated to ensure the completeness and standardization of the annotation information. The annotation results typically include information such as the image name, object category, location, and bounding box. Annotation files are generated based on the collated annotation results. Annotation files can be formatted in common formats, such as XML and JSON. These formats are highly readable and extensible, facilitating subsequent data storage, management, and use. For example, in XML-formatted annotation files, specific tags can be used to represent image information, object category, and bounding box coordinates. The generated annotation files can be used for training other machine learning models, data analysis, and visualization.

[0064] Figure 2 This is a schematic diagram of the annotated picture disclosed in the embodiment of this application. Figure 3 This is a schematic diagram of the recognition results disclosed in the embodiment of this application, such as Figure 3 As shown, the portion framed by a solid line is the identified bad needle. Figure 4 is a schematic diagram of the editing and annotation results disclosed in the embodiment of this application, wherein: Figure 4 (a) in the figure indicates that the labeling result is a horizontal bar. Figure 4 (b) in the figure indicates that the labeling result is flying flowers. Figure 5 is a schematic diagram of the modified threshold value disclosed in the embodiment of this application, Figure 5(a) in the figure shows the labeling result when the horizontal bar threshold is 0.25. Figure 5 (b) in the figure shows the labeling result with a horizontal bar threshold of 0.92. When the system cannot label, manual labeling can be performed. Figure 6 This is a directly labeled initial schematic diagram disclosed in the embodiment of this application, such as Figure 6 Indicates that directly marking can create new flying flowers; Figure 7 This is a schematic diagram of the direct marking process disclosed in the embodiment of this application; Figure 7 As shown, the green line is in the process of creating new flying flowers; Figure 8 This is a schematic diagram of the direct marking result disclosed in the embodiment of this application, such as Figure 8 As shown, the flying flower marking is completed. Figure 2-Figure 8 The following examples are described in detail.

[0065] Optionally, adjusting the preliminary annotation result according to the operation information to obtain the annotation result includes:

[0066] When the operation information is deletion, changing the border color of the original annotation box in the preliminary annotation result to a preset first color;

[0067] When the operation information is a category, the original category in the preliminary annotation result is replaced with the new category, and the border color of the original annotation box in the preliminary annotation result is changed to a preset second color;

[0068] When the operation information is editing, the boundary box of the marked area is redrawn according to the trajectory generated by the user dragging the mouse, and the border color of the original marked box in the preliminary marking result is changed to a preset third color.

[0069] The user determines that a box in the preliminary annotation results is incorrectly labeled or that the area does not need to be labeled, so they choose to delete the box. The border color of the original box in the preliminary annotation results is changed to a preset first color. This border color change visually indicates to the user that the box has been marked for deletion, distinguishing it from other normal or pending boxes. Furthermore, during subsequent data processing or model training, the system can identify this specific border color and ignore the corresponding annotation information. The preset first color is typically a prominent color that differs significantly from the color of normal boxes, such as red. Red is visually noticeable, allowing users to quickly identify deleted boxes. If the user believes that the category associated with a box in the preliminary annotation results is inaccurate, they can replace the original category in the preliminary annotation results with a new one. For example, if the preliminary annotation mistakenly identifies a defect as a "stain," the user can modify it to a "hole" based on the actual situation. The border color of the original box in the preliminary annotation results is changed to a preset second color. Changing the border color allows users to clearly see the annotation boxes whose categories have been modified, facilitating their review and confirmation. Furthermore, during subsequent data processing, the system can identify annotation boxes with modified categories based on the border color, potentially requiring special processing or further review. A preset second color option allows users to select a distinct color, such as blue, that is different from the color of deleted annotation boxes. Blue highlights the modified category without being confused with the red color of deleted items. A user may find that the boundaries of a certain annotation box in the preliminary annotation results are inaccurate and do not properly enclose the target area. They can then drag the mouse to reposition and resize the bounding box. The bounding box for the annotated area is redrawn based on the mouse movement trajectory. The system captures the mouse movement in real time and uses this trajectory information to accurately draw a new bounding box that more closely matches the actual boundaries of the target area. The border color of the original annotation box in the preliminary annotation results is changed to a preset third color. This border color change allows users to intuitively identify the edited annotation boxes. Furthermore, during subsequent data processing, the system can identify this specific border color and separately process or mark the edited annotation boxes for quality checks and model updates. The preset third color can be a color different from the deletion and category modification markup box colors, such as green. Green usually represents the modified or updated state and can clearly distinguish it from the markup boxes of other states.

[0070] Different color codes allow reviewers to quickly locate modified portions of the annotation results. When reviewing large amounts of annotated data, reviewers no longer need to examine the detailed information of each annotation box individually; they only need to focus on those that have changed color, significantly improving review efficiency and reducing review time and workload. By distinguishing different operation states by color, annotation managers can more conveniently monitor and manage the annotation process. For example, they can count the number of annotation boxes of different colors to understand the frequency of various operations (deletion, category modification, and bounding box editing), and analyze potential issues in the annotation process. For example, frequent modifications to certain categories may indicate a deficiency in the model's recognition of those categories, allowing targeted improvement measures. Color changes serve as intuitive cues, helping users perform operations more accurately. Color cues during category modification and bounding box editing also provide users with a clearer view of their operation results. Unified color coding rules help standardize user workflows. When users make annotation adjustments, they follow the established color change rules, making the entire annotation process more standardized and consistent. This helps improve the quality of the annotation data and reduce errors caused by improper operation. The annotation boxes of different colors record the user's adjustment information on the preliminary annotation results. This information can serve as an important basis for model optimization.

[0071] Optionally, adjusting the deep learning model according to the operation information includes:

[0072] When it is detected that the user performs a deletion operation on the original annotation box, extracting the image feature vector corresponding to the original annotation box, and calculating the cosine similarity between the image feature vector and the weight vector of each category in the classifier false detection weight matrix in the deep learning model to determine the most similar category;

[0073] The weight value of the most similar category in the classifier false detection weight matrix is ​​reduced according to a preset ratio.

[0074] During training, deep learning models learn various image feature representations. For the image region corresponding to the original bounding box, the model extracts a feature vector for that region through preceding structures such as convolutional layers. This feature vector is a highly abstract and generalized representation of the image region's characteristics, encompassing information such as color, texture, and shape. Typically, the feature vector is obtained at an intermediate layer in the model (such as before a fully connected layer). For example, when using a convolutional neural network (CNN), a series of convolution and pooling operations results in a feature map, which, when flattened, yields a feature vector. This feature vector represents the characteristics of the image region corresponding to the original bounding box. In a deep learning model's classifier, each class has a corresponding weight vector, which together form a false positive weight matrix. The weight vector reflects the model's sensitivity to features from different classes. The model performs classification by calculating the similarity between the input feature vector and the weight vectors for each class. Cosine similarity measures the similarity between two vectors, with values ​​ranging from -1 to 1. By calculating the cosine similarity between an image feature vector and the weight vectors for each class, we can determine the degree of similarity between the image feature and features from different classes. Compare the calculated cosine similarity values ​​and identify the category with the greatest similarity. This category is considered the most similar category. In other words, the image features corresponding to the original annotation box are closest to the weight vector for this category in feature space. The preset ratio is a pre-set parameter that controls the extent of weight reduction. It can be adjusted based on actual conditions. For example, if the model has a high rate of false positives for a certain category, the preset ratio can be increased to further reduce the weight of that category. Conversely, if the false positives are relatively minor, the preset ratio can be decreased. Lowering the weight of the most similar category in the false positive weight matrix means that the model will be less sensitive to this category in subsequent classification processes. Because the user deleted the original annotation box, the model may have misjudged this category. By lowering the weight, the model is less likely to misclassify the image area as belonging to this category again, thereby improving the model's classification accuracy.

[0075] When a user deletes an original annotation box, the system extracts the image feature vector corresponding to the original annotation box and calculates its cosine similarity with the weight vectors for each category in the classifier's false positive weight matrix in the deep learning model. Cosine similarity measures the directional similarity between two vectors. This method accurately identifies the category most similar to the features of the false positive annotation box. Identifying the most similar category clearly identifies the source of the model's false positives. This helps developers gain a deeper understanding of the categories the model is prone to misidentifying, providing a clear direction for subsequent model optimization. The weight of the most similar category in the classifier's false positive weight matrix is ​​reduced by a preset ratio. The false positive weight matrix reflects the model's likelihood of false positives for different categories. Lowering the weight of a category reduces the model's tendency to misclassify it as a different category. By reducing the false positive weight, the model makes more accurate classification decisions. This effectively mitigates misclassifications that might otherwise occur due to high false positive weights, enabling more accurate identification of defects and improving overall annotation accuracy and model reliability. This adjustment method, based on user feedback, enables dynamic model optimization. When users discover false positives during the annotation process and delete them, the model promptly captures this information and adjusts its weight matrix. This dynamic adjustment mechanism enables the model to adapt to different annotation scenarios and data characteristics, continuously learning and improving its performance. In real-world applications, fabric data can be diverse and complex. By continuously adjusting based on user feedback, the model can better adapt to fabric data of different types and characteristics, improving its generalization and adaptability.

[0076] Optionally, adjusting the deep learning model according to the operation information includes:

[0077] When it is detected that the user modifies the category label of the original annotation box, a mapping relationship between the original category and the new category is obtained, and a feature map of the area corresponding to the original annotation box is extracted;

[0078] Inputting the feature map into an incremental learning module of a classifier, generating a first prototype vector of a new category through sliding window mean pooling, and calculating a difference vector between the first prototype vector and the second prototype vector of the original category;

[0079] The weight corresponding to the new category is determined based on the difference vector, the update rate parameter, and the weight corresponding to the original category.

[0080] During the data annotation process, the system continuously monitors user actions. When a user modifies the category label of the original annotation bounding box, the system immediately detects this action. For example, in a fabric defect labeling task, a user may originally label a defect as "hole," but after further observation and analysis, they change it to "skipped yarn." The system accurately identifies this modification by monitoring user interactions (such as clicking the edit button or selecting a new category). When a modification is detected, the system obtains the mapping between the original and new categories. This mapping helps the model understand the transition logic between different categories. For example, in the example above, the mapping from "hole" to "skipped yarn" is recorded. In practical applications, a category mapping table may be constructed to store common category transitions for quick query and access when needed. The system extracts a feature map for the region corresponding to the original annotation bounding box. This feature map is extracted from the input image using the convolutional layers of a deep learning model. It contains rich image feature information about the region, such as texture, color, and shape. These features are crucial for the model's category recognition and classification. For example, for images of fabric defects, feature maps can capture the unique characteristics of the defects, helping the model distinguish between different types of defects. The extracted feature maps are input into the classifier's incremental learning module. Incremental learning is a method that allows a model to learn new knowledge without retraining the entire model. In data annotation scenarios, as users continuously modify category labels, the model needs to be able to quickly adapt to new category information. The incremental learning module processes this new feature data, enabling the model to dynamically update and expand its knowledge. In the incremental learning module, sliding window mean pooling is used to generate the first prototype vector for the new category. Sliding window mean pooling is a technique for dimensionality reduction and feature extraction on feature maps. Specifically, a fixed-size window is slid across the feature map, and the mean of all feature values ​​within the window is calculated as the new feature value. By sliding the window across the entire feature map, a more representative feature vector, the first prototype vector, is obtained. The first prototype vector represents the typical characteristics of the new category in the feature space. The difference vector between the first prototype vector (for the new category) and the second prototype vector (for the original category) is calculated. The second prototype vector is the typical feature vector of the original category that the model has learned during the previous learning process. The difference vector reflects the degree of difference between the new category and the original category in feature space. By calculating the difference vector, the model can understand the characteristic changes of the new category relative to the original category, providing a basis for subsequent weight adjustments. The update rate parameter controls the speed and magnitude of the weight update of the new category. Its value range is typically between 0 and 1. If the update rate parameter is set to a large value, the model will adapt to the new category more quickly, but may cause model instability. If the update rate parameter is set to a small value, the model will update more slowly but be more stable.In practical applications, the appropriate update rate parameter can be selected based on the characteristics of the data and the requirements of the labeling task. The weight corresponding to the new category is determined based on the difference vector, the update rate parameter, and the weight corresponding to the original category. The specific calculation method may vary depending on the model, but a weighted calculation method is usually adopted. For example, the difference vector can be multiplied by the update rate parameter, and then a part of the weight of the original category can be added to obtain the weight of the new category. The purpose of this is to make the weight of the new category reflect the characteristics of the new category while being integrated with the original weight system of the model to ensure the stability and consistency of the model.

[0081] When a user modifies the category label of the original annotation box, the model immediately determines the mapping between the original and new categories and extracts feature maps for the corresponding regions. By feeding these feature maps into the classifier's incremental learning module, the model can quickly generate the first prototype vector for the new category. This mechanism enables the model to efficiently learn and adapt to new categories without retraining the entire dataset, significantly saving training time and computing resources. In real-world annotation scenarios, new categories may appear continuously. This adjustment method allows the model to update in real time based on user actions, enabling it to promptly identify and process new categories, improving its adaptability and practicality in dynamic environments. Using sliding window mean pooling to generate the first prototype vector for the new category effectively extracts and summarizes the representative features of the new category in the feature space. Furthermore, calculating the difference vector between the first prototype vector and the second prototype vector of the original category accurately characterizes the feature differences between the new and original categories. This difference characterization helps the model better understand the boundaries and distinctions between different categories, improving the accuracy of category recognition. By clearly identifying the feature differences between the new and original categories, the model can more accurately classify samples into the correct category when making classification decisions. Especially when feature similarity between categories is high, calculating the difference vector provides the model with a clearer basis for classification, reducing the possibility of misclassification. The weights for the new categories are determined based on the difference vector, the update rate parameter, and the weights for the original categories. The update rate parameter provides flexible control over the extent of model updates. Users can adjust the update rate parameter based on actual needs and application scenarios.

[0082] Optionally, calculating the difference vector between the first prototype vector and the second prototype vector of the original category includes:

[0083] Calculating the cosine similarity and the Euclidean distance between the first prototype vector and the second prototype vector, and performing weighted fusion on the cosine similarity and the Euclidean distance to generate a comprehensive difference;

[0084] Determine a difference threshold based on the number of samples of the original category and the annotation confidence, and judge whether the comprehensive difference is greater than or equal to the difference threshold;

[0085] When the comprehensive difference is greater than or equal to the difference threshold, the comprehensive difference is mapped to a vector space to obtain the difference vector.

[0086] Cosine similarity is used to measure the degree of similarity between two vectors in direction. By calculating cosine similarity, we can understand the directional difference between the first prototype vector and the second prototype vector in the feature space, thereby reflecting the similarity between the two categories in feature distribution. Euclidean distance is used to measure the actual distance between two vectors in space. The larger the value of Euclidean distance, the farther the distance between the two vectors in space, that is, the greater the feature difference between the two categories. In order to comprehensively consider the differences in direction and distance between the two vectors, cosine similarity and Euclidean distance are weighted and fused. Let the weight of cosine similarity be , the weight of the Euclidean distance is (generally ), the comprehensive difference can be expressed as: , represents the cosine similarity, represents the Euclidean distance, where This is to convert cosine similarity into dissimilarity, with larger values ​​indicating greater dissimilarity. Through weighted fusion, a comprehensive metric can be obtained that more comprehensively reflects the difference between two prototype vectors. The number of samples in the original category affects the model's understanding of the characteristics of that category. A larger number of samples leads to a more accurate grasp of the characteristic distribution of that category. Annotation confidence reflects the reliability of the annotation results. Taking these two factors into consideration, a dissimilarity threshold can be determined. For example, when the original category has a large number of samples and high annotation confidence, a relatively high dissimilarity threshold can be set. This is because the model already has a relatively accurate understanding of the category and requires a larger difference to consider the new category significantly different from the original. Conversely, when the number of samples is small or the annotation confidence is low, a lower dissimilarity threshold can be set to make it easier to detect differences between the new category and the original. The calculated comprehensive dissimilarity is then compared with the dissimilarity threshold. If the comprehensive difference is greater than or equal to the difference threshold, the difference between the first and second prototype vectors is significant enough to indicate that the new category is clearly distinct from the original category. If the comprehensive difference is less than the difference threshold, the new category is considered similar in characteristics to the original category, and significant weight adjustments may not be necessary. When the comprehensive difference is greater than or equal to the difference threshold, the comprehensive difference is mapped into vector space to represent the difference information in vector form, facilitating subsequent model processing (such as weight adjustment). The specific mapping method can be designed based on actual needs. For example, the comprehensive difference can be multiplied by a unit vector or transformed into a vector with a specific direction and magnitude using some function. The resulting vector is the difference vector, which represents the difference between the new category and the original category in feature space. This difference vector is used to calculate the weight corresponding to the new category, allowing the model to appropriately adjust the weight of the new category based on the degree of difference between the new category and the original category, thereby better adapting to the characteristics of the new category.

[0087] By calculating the cosine similarity and Euclidean distance between the first and second prototype vectors and performing a weighted fusion to generate a comprehensive dissimilarity measure, the difference between the two vectors can be comprehensively measured from different perspectives. Cosine similarity focuses on measuring the directional similarity of vectors, while Euclidean distance focuses on the actual distance between vectors in space. Combining these two metrics can more accurately capture the difference between vectors and avoid misjudgments caused by the limitations of a single metric. The weighted fusion approach can adjust the weights of different metrics based on actual needs, making the comprehensive dissimilarity measure more reflective of the focus on vector differences in real applications. For example, in some cases, the directional difference of vectors may be more important, so the weight of cosine similarity can be appropriately increased; in other cases, the spatial distance of vectors may be more critical, so the weight of Euclidean distance can be increased. This flexibility helps improve the accuracy of dissimilarity judgment. The dissimilarity threshold is determined based on the number of samples in the original category, allowing the threshold setting to account for the differences in the number of samples across categories. For categories with a large number of samples, their prototype vectors may be more stable. Therefore, a relatively high difference threshold can be set to avoid misclassifying small fluctuations as large differences. For categories with a small number of samples, their prototype vectors may be less stable. In this case, a lower difference threshold can be set to more sensitively capture any differences. Annotation confidence reflects the reliability of the annotation results. Incorporating annotation confidence into the determination of the difference threshold can make the threshold setting more reasonable. If the annotation confidence of the original category is high, it indicates that the feature representation of that category is more reliable. In this case, a relatively high difference threshold can be set. Conversely, if the annotation confidence is low, the difference threshold should be lowered to more cautiously judge the differences between vectors. Adaptive difference threshold setting allows the model to automatically adjust its judgment criteria based on the characteristics of different categories and the annotation situation, improving the model's adaptability and robustness in various complex scenarios. When the comprehensive difference is greater than or equal to the difference threshold, the comprehensive difference is mapped into vector space to obtain a difference vector. This mapping method allows the difference vector to intuitively represent the difference between the two prototype vectors and can be further analyzed and processed in vector space. The representation of the difference vector makes it easier for the model to utilize this difference information in subsequent classification, recognition, and other tasks. The model can make more accurate classification decisions for new categories based on the difference vector, or use the difference vector to adjust the weights of existing categories, thereby improving the performance and accuracy of the model.

[0088] Optionally, determining the weight corresponding to the new category based on the difference vector, the update rate parameter, and the weight corresponding to the original category includes:

[0089] Mapping the difference vector to the weight space through a fully connected layer to generate a weight offset;

[0090] The update rate parameter is adjusted according to the user operation frequency and the convergence state of the deep learning model, the product of the weight offset and the update rate parameter is calculated, and the weight of the new category is obtained according to the product and the weight corresponding to the original category.

[0091] A fully connected layer is a commonly used neural network layer in deep learning models. It performs linear transformations and nonlinear activations on input data, thereby learning complex mappings between data. Mapping the difference vector to a weight space through a fully connected layer leverages the learning capabilities of the fully connected layer to convert the difference between the new category and the original category, represented by the difference vector, into a weight offset that guides weight adjustment. The fully connected layer calculates the input difference vector based on its internal weight and bias parameters, outputting a vector that matches the weight space. This vector is the weight offset. The weight offset indicates the direction and magnitude of the weight adjustment required for the new category relative to the original category. For example, if the difference vector indicates that the new category differs significantly from the original category in certain feature dimensions, the weight offset may have larger values ​​in these dimensions to guide the corresponding adjustment of the new category's weight in these dimensions. User operation frequency reflects the user's attention to a category and their active annotation activity. If users frequently modify the annotation box and category label for a category, it may indicate that there is a high degree of uncertainty in the category or that the model's recognition performance for that category is poor. At this point, the update rate parameter can be appropriately increased to enable the model to learn new categories more quickly and adapt to changes in user operations. For example, if users frequently adjust the labels for a new category, increasing the update rate parameter allows the model to adjust the weights of the new category more quickly, improving recognition accuracy for that category. The convergence state of a deep learning model indicates the model's stability and performance during training. If the model is close to convergence, it means that the model has fully learned the existing data. In this case, the update rate parameter can be appropriately reduced to avoid excessive fluctuations when introducing new categories and ensure model stability. Conversely, if the model has not yet converged, it means that the model is still in the process of continuous learning and optimization. In this case, the update rate parameter can be appropriately increased to accelerate the model's learning of new categories. By dynamically adjusting the update rate parameter based on user operation frequency and model convergence status, the model can update at an appropriate rate in different situations. This dynamic adjustment mechanism improves the model's adaptability and robustness, ensuring that the model can learn quickly and remain stable when introducing new categories. Calculating the product of the weight offset and the update rate parameter is actually scaling the weight offset. The update rate parameter controls the magnitude of weight adjustments. By multiplying the weights, the change in the weights of the new categories can be adjusted based on actual needs. A larger update rate amplifies the effect of the weight offset, resulting in a larger change in the weights of the new categories. Conversely, a smaller update rate minimizes the effect of the weight offset, resulting in a more gradual change in the weights of the new categories. The weights of the new categories are derived by multiplying the product with the weights corresponding to the original categories, typically by adding the product to the original weights.The purpose of this is to introduce the difference information between the new category and the original category while retaining the original category weight information, so as to achieve a reasonable adjustment of the new category weight. , the product of the weight offset and the update rate parameter is , then the new category weight In this way, the model can take into account the knowledge of the original category and adapt to the characteristics of the new category in the process of learning the new category, thereby improving the recognition ability of the new category.

[0092] The difference vector is mapped to the weight space through a fully connected layer to generate a weight offset. The fully connected layer has powerful nonlinear fitting capabilities and can learn the complex mapping between the difference vector and the weight offset. This mapping accurately converts the difference information in the vector space into an adjustment in the weight space, ensuring that the weight of the new category accurately reflects its difference from the original category. Different new categories differ from the original category in different ways, and the flexibility of the fully connected layer enables it to adapt to various difference patterns and generate appropriate weight offsets. Whether it is differences in feature dimensions, feature strength, or feature distribution, the fully connected layer can capture these differences through learning and convert them into effective adjustments to the weight of the new category. The update rate parameter is adjusted based on user operation frequency, allowing the model to adapt to user operation habits and needs. If users frequently modify or add new categories to a category, it indicates that the category is more important in the current task or there is more uncertainty. In this case, the update rate parameter can be appropriately increased to enable the model to learn and adapt to new category information more quickly. Conversely, if user operation frequency is low, the update rate parameter can be reduced to maintain model stability. The convergence state of a deep learning model reflects the model's current learning progress and stability. In the early stages of model training, when the model converges quickly, the update rate parameter can be appropriately increased to accelerate the learning of new class weights. In the later stages of model training, when the model is nearing convergence, the update rate parameter should be reduced to prevent the model from losing stability due to excessive updates. By adjusting the update rate parameter based on the model's convergence state, the model can maintain good learning performance and stability at all stages. The weight offset is multiplied by the update rate parameter and combined with the weight corresponding to the original class to obtain the weight for the new class. This calculation method considers the differences between the new class and the original class while retaining the existing weight information of the original class. This allows the weight of the new class to inherit the characteristics of the original class while also rationally incorporating the new class's characteristics. Combining the weight offset with the update rate parameter through multiplication enables smooth weight updates. The update rate parameter controls the update amplitude, avoiding drastic changes during the weight update process and ensuring model stability and continuity. This smooth update method also allows the model to better adapt to new class information, gradually adjusting weights and improving classification accuracy.

[0093] Optionally, adjusting the deep learning model according to the operation information includes:

[0094] When the user drags the vertex of the original annotation box, the mouse trajectory coordinate sequence is obtained and a new annotation box is generated using curve fitting;

[0095] Calculate a coordinate difference matrix between the original annotation box and the new annotation box, calculate a loss gradient based on the coordinate difference matrix, and adjust the deep learning model based on the loss gradient.

[0096] During deep learning model training, a visual interface is usually provided for users to adjust the original annotation box. When the user drags the vertices of the original annotation box, the system monitors the mouse movement trajectory in real time. The mouse coordinate information during the dragging process is recorded at regular intervals (for example, multiple times per second) to form a coordinate sequence. This coordinate sequence reflects the user's process and intention in adjusting the annotation box. The mouse trajectory coordinate sequence may contain some noise and irregular fluctuations. Directly using this sequence to generate a new annotation box may not be smooth and accurate. Curve fitting methods can be used to process the coordinate sequence to obtain a new annotation box that better meets the user's intention. Common curve fitting methods include polynomial fitting and spline curve fitting. For example, when using polynomial fitting, a suitable polynomial function can be found based on the data points in the coordinate sequence so that the function can best fit these data points. The fitted polynomial function can be used to calculate the vertex coordinates of the new annotation box. Based on the fitted coordinates, the position and size of the new annotation box are determined. The new annotation box should accurately reflect the user's intention to adjust the original annotation box. Both the original and new annotation boxes can be represented by the coordinates of four vertices. Assume that the vertex coordinates of the original annotation box are , , , , the vertex coordinates of the new annotation box are , , , For each corresponding vertex, calculate the coordinate difference in the x-axis and y-axis directions. For example, the coordinate difference of the first vertex is ( , ), and so on, the coordinate differences of all vertices can be obtained. These coordinate differences are organized into a matrix, namely the coordinate difference matrix. This matrix reflects the position difference between the original annotation box and the new annotation box. The loss function is used to measure the difference between the model prediction result and the true annotation. In this scenario, the difference between the original annotation box and the new annotation box can be used as part of the loss. The loss gradient is the partial derivative of the loss function with respect to the model parameters. By calculating the loss gradient, we can understand the direction of change of the model parameters in the current state so that the model can be adjusted. The loss gradient can be calculated using the backpropagation algorithm. The backpropagation algorithm starts with the loss function and calculates the gradient of each layer layer by layer until the gradient of the model parameters is obtained. In this process, the coordinate difference matrix, as part of the loss function, will participate in the calculation of the gradient. Common parameter update rules are gradient descent and its variants. Taking stochastic gradient descent (SGD) as an example, the update formula for model parameters is ,in, are the updated parameters of the model, are the current parameters of the model, is the learning rate, is the loss gradient. The learning rate controls the step size of parameter updates. A larger learning rate allows the model to converge faster, but may cause the model to oscillate around the optimal solution. A smaller learning rate allows the model to converge more stably, but at a slower rate. By adjusting model parameters based on the loss gradient, the model can learn the user's intention to adjust the annotation box. In subsequent annotation or prediction tasks, the annotation box generated by the model will be closer to the user's desired result. This adjustment method allows the model to continuously adapt to user feedback, improving annotation accuracy and model performance.

[0097] Mouse trajectories record the user's hand movements and reflect the user's desired adjustments to the size, shape, and position of the annotation box. Using curve fitting to generate a new annotation box avoids the irregular or rough appearance that can result from manual vertex dragging. The curve fitting algorithm generates a smoother and more accurate new annotation box based on the mouse trajectory coordinate sequence, ensuring it more closely matches the actual object boundaries, improving annotation accuracy and quality. Calculating the coordinate difference matrix between the original and new annotation boxes intuitively quantifies the differences between the two. Each element in the coordinate difference matrix represents the difference between the original and new annotation boxes at the corresponding coordinate position. Analysis of these differences reveals changes in the annotation boxes in various directions. This matrix provides a specific quantitative basis for subsequent model adjustments. The model can use the coordinate difference matrix to determine the magnitude and direction of the annotation box adjustments, thereby making targeted adjustments to its parameters to better accommodate the user's annotation operations. The loss gradient is calculated based on the coordinate difference matrix. This loss gradient indicates the direction and magnitude of adjustments required to reduce the difference between the original and new annotation boxes in the current state. By calculating the loss gradient, the model can identify areas where it is lacking, allowing it to learn and improve in a targeted manner. Adjusting the deep learning model based on the loss gradient can help the model learn more accurate feature representations and classification rules. For example, if a user frequently adjusts the annotation box of a specific area during the annotation process, the model can be guided by the loss gradient to pay more attention to the feature information of that area, improving its ability to recognize and classify objects in that area.

[0098] This embodiment also discloses an intelligent data labeling system. Figure 7 This is a module diagram of the intelligent data annotation system disclosed in the embodiment of this application. Figure 9 As shown, the system includes a setting module 701, a processing module 702, a marking module 703 and an output module 704, wherein:

[0099] A setup module 701 is configured to match and load a pre-trained deep learning model based on fabric type and surface characteristics and preset model setup conditions, wherein the fabric type and surface characteristics include texture, color, and defect type, and the model setup conditions include recognition accuracy requirements and computing resource limitations;

[0100] Processing module 702 is configured to input the image to be annotated into the deep learning model and generate preliminary annotation results according to a preset confidence threshold;

[0101] a labeling module 703 configured to receive user operation information on the preliminary labeling result, adjust the preliminary labeling result according to the operation information to obtain a labeling result, and adjust the deep learning model according to the operation information, wherein the operation information includes deleting, modifying categories, and adjusting bounding boxes;

[0102] The output module 704 is configured to generate a marking file according to the marking result.

[0103] Optionally, the marking module 703 is configured to:

[0104] When the operation information is deletion, changing the border color of the original annotation box in the preliminary annotation result to a preset first color;

[0105] When the operation information is a category, the original category in the preliminary annotation result is replaced with the new category, and the border color of the original annotation box in the preliminary annotation result is changed to a preset second color;

[0106] When the operation information is editing, the boundary box of the marked area is redrawn according to the trajectory generated by the user dragging the mouse, and the border color of the original marked box in the preliminary marking result is changed to a preset third color.

[0107] Optionally, the marking module 703 is configured to:

[0108] When it is detected that the user performs a deletion operation on the original annotation box, extracting the image feature vector corresponding to the original annotation box, and calculating the cosine similarity between the image feature vector and the weight vector of each category in the classifier false detection weight matrix in the deep learning model to determine the most similar category;

[0109] The weight value of the most similar category in the classifier false detection weight matrix is ​​reduced according to a preset ratio.

[0110] Optionally, the marking module 703 is configured to:

[0111] When it is detected that the user modifies the category label of the original annotation box, a mapping relationship between the original category and the new category is obtained, and a feature map of the area corresponding to the original annotation box is extracted;

[0112] Inputting the feature map into an incremental learning module of a classifier, generating a first prototype vector of a new category through sliding window mean pooling, and calculating a difference vector between the first prototype vector and the second prototype vector of the original category;

[0113] The weight corresponding to the new category is determined based on the difference vector, the update rate parameter, and the weight corresponding to the original category.

[0114] Optionally, the marking module 703 is configured to:

[0115] Calculating the cosine similarity and the Euclidean distance between the first prototype vector and the second prototype vector, and performing weighted fusion on the cosine similarity and the Euclidean distance to generate a comprehensive difference;

[0116] Determine a difference threshold based on the number of samples of the original category and the annotation confidence, and judge whether the comprehensive difference is greater than or equal to the difference threshold;

[0117] When the comprehensive difference is greater than or equal to the difference threshold, the comprehensive difference is mapped to a vector space to obtain the difference vector.

[0118] Optionally, the marking module 703 is configured to:

[0119] Mapping the difference vector to the weight space through a fully connected layer to generate a weight offset;

[0120] The update rate parameter is adjusted according to the user operation frequency and the convergence state of the deep learning model, the product of the weight offset and the update rate parameter is calculated, and the weight of the new category is obtained according to the product and the weight corresponding to the original category.

[0121] Optionally, the marking module 703 is configured to:

[0122] When the user drags the vertex of the original annotation box, the mouse trajectory coordinate sequence is obtained and a new annotation box is generated using curve fitting;

[0123] Calculate a coordinate difference matrix between the original annotation box and the new annotation box, calculate a loss gradient based on the coordinate difference matrix, and adjust the deep learning model based on the loss gradient.

[0124] It should be noted that the above embodiments provide devices that implement their functions using only the division of the above functional modules as examples. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0125] This embodiment also discloses an electronic device, referring to Figure 10 The electronic device may include: at least one processor 801 , at least one communication bus 802 , a user interface 803 , a network interface 804 , and at least one memory 805 .

[0126] The communication bus 802 is used to implement the connection and communication between these components.

[0127] The user interface 803 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 803 may also include a standard wired interface and a wireless interface.

[0128] The network interface 804 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0129] The processor 801 may include one or more processing cores. Using various interfaces and circuits, the processor 801 connects to various components within the server. It executes instructions, programs, code sets, or instruction sets stored in the memory 805, as well as accesses data stored in the memory 805, to perform various server functions and process data. Optionally, the processor 801 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 801 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing content displayed on the display screen; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 801 but implemented as a separate chip.

[0130] Among them, the memory 805 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 805 includes a non-transitory computer-readable storage medium. The memory 805 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 805 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 805 may also be optionally at least one storage device located away from the aforementioned processor 801. As Figure 10 As shown, the memory 805 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program of the intelligent data annotation system.

[0131] exist Figure 10 In the electronic device shown, the user interface 803 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 801 can be used to call the application of the intelligent data labeling system stored in the memory 805. When executed by one or more processors 801, the electronic device executes one or more methods in the above embodiments.

[0132] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0133] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0135] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0136] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0137] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory 805. Based on this understanding, the technical solution of this application, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 805 and includes several instructions for causing a computer device (such as a personal computer, server, or network device) to execute all or part of the steps of the various embodiments of the method of this application. The aforementioned memory 805 includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a magnetic disk, or an optical disk.

[0138] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. An intelligent data annotation method, characterized in that: Applied to a data annotation platform, the method includes: Matching and loading a pre-trained deep learning model based on fabric type and surface characteristics, including texture, color, and defect type, and preset model setting conditions, including recognition accuracy requirements and computing resource limitations; Input the image to be annotated into the deep learning model and generate preliminary annotation results based on a preset confidence threshold; Receive user operation information on the preliminary annotation result, adjust the preliminary annotation result according to the operation information to obtain an annotation result, and adjust the deep learning model according to the operation information, wherein the operation information includes deleting, modifying categories, and adjusting bounding boxes; Generate a marking file according to the marking results, The adjusting the deep learning model according to the operation information includes: When it is detected that the user performs a deletion operation on the original annotation box, extracting the image feature vector corresponding to the original annotation box, and calculating the cosine similarity between the image feature vector and the weight vector of each category in the classifier false detection weight matrix in the deep learning model to determine the most similar category; reducing the weight value of the most similar category in the classifier false detection weight matrix according to a preset ratio; When it is detected that the user modifies the category label of the original annotation box, a mapping relationship between the original category and the new category is obtained, and a feature map of the area corresponding to the original annotation box is extracted; Inputting the feature map into an incremental learning module of a classifier, generating a first prototype vector of the new category through sliding window mean pooling, and calculating a difference vector between the first prototype vector and the second prototype vector of the original category; Determining a weight corresponding to the new category based on the difference vector, an update rate parameter, and a weight corresponding to the original category; When the user drags the vertex of the original annotation box, the mouse trajectory coordinate sequence is obtained and a new annotation box is generated using curve fitting; Calculate a coordinate difference matrix between the original annotation box and the new annotation box, calculate a loss gradient based on the coordinate difference matrix, and adjust the deep learning model based on the loss gradient.

2. The intelligent data labeling method according to claim 1, characterized in that: The adjusting the preliminary annotation result according to the operation information to obtain the annotation result includes: When the operation information is deletion, changing the border color of the original annotation box in the preliminary annotation result to a preset first color; When the operation information is a category, the original category in the preliminary annotation result is replaced with the new category, and the border color of the original annotation box in the preliminary annotation result is changed to a preset second color; When the operation information is editing, the boundary box of the marked area is redrawn according to the trajectory generated by the user dragging the mouse, and the border color of the original marked box in the preliminary marking result is changed to a preset third color.

3. The intelligent data labeling method according to claim 1, characterized in that: The calculating of the difference vector between the first prototype vector and the second prototype vector of the original category includes: Calculating the cosine similarity and the Euclidean distance between the first prototype vector and the second prototype vector, and performing weighted fusion on the cosine similarity and the Euclidean distance to generate a comprehensive difference; Determine a difference threshold based on the number of samples of the original category and the annotation confidence, and judge whether the comprehensive difference is greater than or equal to the difference threshold; When the comprehensive difference is greater than or equal to the difference threshold, the comprehensive difference is mapped to a vector space to obtain the difference vector.

4. The intelligent data labeling method according to claim 3, characterized in that: Determining the weight corresponding to the new category based on the difference vector, the update rate parameter, and the weight corresponding to the original category includes: Mapping the difference vector to the weight space through a fully connected layer to generate a weight offset; The update rate parameter is adjusted according to the user operation frequency and the convergence state of the deep learning model, the product of the weight offset and the update rate parameter is calculated, and the weight of the new category is obtained according to the product and the weight corresponding to the original category.

5. An intelligent data annotation system, characterized in that: It includes setting module, processing module, annotation module and output module, among which: a settings module configured to match and load a pre-trained deep learning model based on fabric type and surface characteristics and preset model setting conditions, wherein the fabric type and surface characteristics include texture, color, and defect type, and the model setting conditions include recognition accuracy requirements and computing resource limitations; A processing module configured to input the image to be annotated into the deep learning model and generate preliminary annotation results according to a preset confidence threshold; a labeling module configured to receive user operation information on the preliminary labeling result, adjust the preliminary labeling result according to the operation information to obtain a labeling result, and adjust the deep learning model according to the operation information, the operation information including deleting, modifying categories, and adjusting bounding boxes; An output module configured to generate a marking file according to the marking result, The annotation module is configured to: When it is detected that the user performs a deletion operation on the original annotation box, extracting the image feature vector corresponding to the original annotation box, and calculating the cosine similarity between the image feature vector and the weight vector of each category in the classifier false detection weight matrix in the deep learning model to determine the most similar category; reducing the weight value of the most similar category in the classifier false detection weight matrix according to a preset ratio; When it is detected that the user modifies the category label of the original annotation box, a mapping relationship between the original category and the new category is obtained, and a feature map of the area corresponding to the original annotation box is extracted; Inputting the feature map into an incremental learning module of a classifier, generating a first prototype vector of the new category through sliding window mean pooling, and calculating a difference vector between the first prototype vector and the second prototype vector of the original category; Determining a weight corresponding to the new category based on the difference vector, an update rate parameter, and a weight corresponding to the original category; When the user drags the vertex of the original annotation box, the mouse trajectory coordinate sequence is obtained and a new annotation box is generated using curve fitting; Calculate a coordinate difference matrix between the original annotation box and the new annotation box, calculate a loss gradient based on the coordinate difference matrix, and adjust the deep learning model based on the loss gradient.

6. An electronic device, characterized in that: The electronic device comprises a processor, a memory, a user interface and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 4 is executed.

Images