Intelligent data labeling method and system, electronic equipment and storage medium

Abstract

The invention discloses an intelligent data labeling method and system, electronic equipment and a storage medium. The method comprises the steps that a pre-trained deep learning model is matched and loaded according to fabric characteristics and preset model setting conditions, the fabric characteristics comprise textures, colors and flaw types, and the model setting conditions comprise recognition precision requirements and calculation resource limits; inputting the to-be-labeled picture into the deep learning model, and generating a preliminary labeling result according to a preset confidence threshold; operation information of a user on the preliminary labeling result is received, the preliminary labeling result is adjusted according to the operation information to obtain a labeling result, the deep learning model is adjusted according to the operation information, and the operation information comprises deletion, category modification and bounding box adjustment; and generating an annotation file according to the annotation result. According to the technical scheme provided by the embodiment of the invention, the accuracy and efficiency of data annotation are effectively improved.

Description

Technical Field

[0001] This application relates to the technical field of data annotation, and particularly to an intelligent data annotation method, system, electronic device, and storage medium. Background Art

[0002] Data annotation technology plays a crucial role in the field of artificial intelligence. Its main task is to classify, label, or annotate data such as images, texts, or audios for subsequent machine learning model training. With the rapid development of deep learning technology, high-quality data annotation has become one of the key factors to improve model performance. Data annotation not only directly affects the accuracy of the model but also largely determines the generalization ability of the model, so it has received extensive attention and application in both the industrial and academic fields.

[0003] Currently, to achieve efficient data annotation, the industry usually adopts manual annotation or rule-based automated annotation. Manual annotation is the most traditional method, where professional personnel manually annotate data one by one. Although the annotation accuracy is relatively high, the efficiency is low and the cost is high. The rule-based automated annotation method automatically completes annotation through pre-set rules and is applicable to specific scenarios, but its flexibility is poor and it is difficult to adapt to complex and diverse data types. Existing data annotation methods often have difficulty meeting high-precision requirements when dealing with cloth images with complex textures, diverse colors, and defects, and are limited by computing resources, resulting in a decrease in annotation accuracy or low annotation efficiency.

[0004] Therefore, there is an urgent need for an efficient data annotation method that can adapt to multiple scenarios. Summary of the Invention

[0005] This application provides an intelligent data annotation method, system, electronic device, and storage medium. By combining user interaction and model adaptive adjustment, it effectively improves the accuracy and efficiency of data annotation, while reducing the cost of manual intervention, and is applicable to data annotation tasks in complex scenarios.

[0006] In the first aspect of this application, an intelligent data annotation method is provided, which is applied to a data annotation platform. The method includes: According to the characteristics of the cloth type and cloth surface and the preset model setting conditions, match and load a pre-trained deep learning model. The characteristics of the cloth type and cloth surface include texture, color, and defect type, and the model setting conditions include recognition accuracy requirements and computing resource limitations; Input the image to be annotated into the deep learning model, and generate a preliminary annotation result according to the preset confidence threshold; Receive the operation information of the user on the preliminary annotation result, adjust the preliminary annotation result according to the operation information to obtain the annotation result, and adjust the deep learning model according to the operation information, where the operation information includes deletion, modification of category, and adjustment of the bounding box; Generate an annotation file according to the annotation result.

[0007] Optionally, the adjusting the preliminary annotation result according to the operation information to obtain the annotation result includes: When the operation information is deletion, change the border color of the original annotation box in the preliminary annotation result to a preset first color; When the operation information is category, replace the original category in the preliminary annotation result with a new category, and change the border color of the original annotation box in the preliminary annotation result to a preset second color; When the operation information is editing, redraw the bounding box of the annotation area according to the trajectory generated by the user dragging the mouse, and change the border color of the original annotation box in the preliminary annotation result to a preset third color.

[0008] Optionally, the adjusting the deep learning model according to the operation information includes: When it is detected that the user performs a deletion operation on the original annotation box, extract the image feature vector corresponding to the original annotation box, calculate the cosine similarity between the image feature vector and the weight vectors of each category in the misdetection weight matrix of the classifier in the deep learning model to determine the most similar category; Reduce the weight value of the most similar category in the misdetection weight matrix of the classifier according to a preset ratio.

[0009] Optionally, the adjusting the deep learning model according to the operation information includes: When it is detected that the user modifies the category label of the original annotation box, obtain the mapping relationship between the original category and the new category, and extract the feature map of the area corresponding to the original annotation box; Input the feature map into the incremental learning module of the classifier, generate the first prototype vector of the new category through sliding window average pooling, and calculate the difference vector between the first prototype vector and the second prototype vector of the original category; Determine the weight corresponding to the new category based on the difference vector, the update rate parameter, and the weight corresponding to the original category.

[0010] Optionally, the calculating the difference vector between the first prototype vector and the second prototype vector of the original category includes: Calculate the cosine similarity and the Euclidean distance between the first prototype vector and the second prototype vector, and perform weighted fusion on the cosine similarity and the Euclidean distance to generate a comprehensive difference degree; Determine the difference threshold according to the number of samples in the original category and the annotation confidence, and judge whether the comprehensive difference is greater than or equal to the difference threshold; When the comprehensive difference is greater than or equal to the difference threshold, map the comprehensive difference to a vector space to obtain the difference vector.

[0011] Optionally, determining the weight corresponding to the new category based on the difference vector, the update rate parameter, and the weight corresponding to the original category includes: Map the difference vector to a weight space through a fully connected layer to generate a weight offset; Adjust the update rate parameter according to the user operation frequency and the convergence state of the deep learning model, calculate the product of the weight offset and the update rate parameter, and obtain the weight of the new category according to the product and the weight corresponding to the original category.

[0012] Optionally, adjusting the deep learning model according to the operation information includes: When the user drags the vertex of the original annotation box, obtain the mouse trajectory coordinate sequence and generate a new annotation box by curve fitting; Calculate the coordinate difference matrix between the original annotation box and the new annotation box, calculate the loss gradient according to the coordinate difference matrix, and adjust the deep learning model according to the loss gradient.

[0013] In a second aspect of the present application, an intelligent data annotation system is provided, including a setting module, a processing module, an annotation module, and an output module, where: The setting module is configured to match and load a pre-trained deep learning model according to the cloth type and surface characteristics and the preset model setting conditions, where the cloth type and surface characteristics include texture, color, and defect type, and the model setting conditions include recognition accuracy requirements and computing resource limitations; The processing module is configured to input the image to be annotated into the deep learning model and generate a preliminary annotation result according to a preset confidence threshold; The annotation module is configured to receive the operation information of the user on the preliminary annotation result, adjust the preliminary annotation result according to the operation information to obtain an annotation result, and adjust the deep learning model according to the operation information, where the operation information includes deletion, modification of categories, and adjustment of bounding boxes; The output module is configured to generate an annotation file according to the annotation result.

[0014] In a third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. Both the user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory, so that the electronic device executes the method described in any one of the above.

[0015] In a fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions, and when the instructions are executed, the method described in any one of the above is executed.

[0016] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Match and load a pre-trained deep learning model according to the characteristics of the sowing cloth surface and the preset model setting conditions. This enables the selected model to highly adapt to the annotation requirements of a specific sowing cloth surface, improves the fit between the model and the annotation task, helps to more accurately identify the cloth surface features, and lays a good foundation for subsequent annotation work; Considering the computing resource limitations, it is possible to select the most suitable model for the current resource situation on the premise of meeting the recognition accuracy requirements, avoid resource waste, improve the operation efficiency of the annotation platform, and enable the platform to work efficiently and stably under different hardware environments; 2. Input the picture to be annotated into the deep learning model, and quickly generate a preliminary annotation result by virtue of the powerful feature extraction and classification capabilities of the model. Compared with the traditional manual annotation method, the annotation speed is greatly improved, and a large number of pictures can be processed in a short time to meet the needs of large-scale data annotation. The preliminary annotation result is generated through a preset confidence threshold, which ensures that the annotation result has a certain reliability. Only the annotations with the model prediction confidence reaching the threshold will be retained, reducing the number of incorrect annotations and improving the accuracy of the preliminary annotation, providing a good basis for subsequent manual adjustment; 3. Receive the operation information of the user on the preliminary annotation result (deleting, modifying the category, and adjusting the bounding box), and allow the user to make detailed adjustments to the annotation result according to the actual situation. This human-computer interaction method gives full play to the human subjective judgment ability and professional knowledge in a specific field, can correct the possible errors of the model, make the annotation result more in line with the actual needs, and improve the accuracy and quality of the annotation. Adjust the deep learning model according to the user's operation information, realizing the dynamic learning and optimization of the model. As the user continuously adjusts the annotation result, the model can learn more accurate features and annotation rules, gradually improve its own performance and generalization ability, and generate more accurate preliminary annotation results in subsequent annotation tasks, forming a virtuous cycle; 4. Generate an annotation file based on the final annotation results, ensuring the standardization and consistency of the annotated data. The annotation file can adopt a common format, facilitating subsequent data storage, management, and use, providing convenience for tasks such as model training and algorithm development based on these annotated data, and helping to improve the efficiency and accuracy of the entire data processing and analysis process. Description of the Drawings

[0017] Figure 1 is a schematic flowchart of the intelligent data annotation method disclosed in the embodiments of the present application; Figure 2 is a schematic diagram of an annotated picture disclosed in the embodiments of the present application; Figure 3 is a schematic diagram of the recognition result disclosed in the embodiments of the present application; Figure 4 is a schematic diagram of the edited annotation result disclosed in the embodiments of the present application; Figure 5 is a schematic diagram of the modified threshold disclosed in the embodiments of the present application; Figure 6 is a schematic diagram of the direct annotation initial stage disclosed in the embodiments of the present application; Figure 7 is a schematic diagram of the direct annotation process disclosed in the embodiments of the present application; Figure 8 is a schematic diagram of the direct annotation result disclosed in the embodiments of the present application; Figure 9 is a schematic diagram of the modules of the intelligent data annotation system disclosed in the embodiments of the present application; Figure 10 is a schematic diagram of the structure of an electronic device disclosed in the embodiments of the present application.

[0018] Description of the reference numerals: 701, setting module; 702, processing module; 703, annotation module; 704, output module; 801, processor; 802, communication bus; 803, user interface; 804, network interface; 805, memory. Detailed Embodiments

[0019] To enable those skilled in the art to better understand the technical solutions in this specification, the following will clearly and completely describe the technical solutions in the embodiments of this specification in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments.

[0020] In the description of the embodiments of the present application, words such as "for example" or "for illustration" are used to give examples, illustrations or explanations. Any embodiment or design solution described as "for example" or "for illustration" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "for example" or "for illustration" is intended to present relevant concepts in a specific manner.

[0021] In the description of the embodiments of the present application, the term "a plurality of" means two or more. For example, a plurality of systems means two or more systems, and a plurality of screen terminals means two or more screen terminals. In addition, the terms "first" and "second" are used only for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The terms "comprise", "include", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0022] This embodiment discloses an intelligent data annotation method, which is applied to a data annotation platform. Figure 1 It is a schematic flowchart of the intelligent data annotation method disclosed in the embodiments of the present application, as Figure 1 shown. The method includes the following steps: S101. According to the characteristics of the fabric surface of the sowing and the preset model setting conditions, match and load a pre-trained deep learning model, where the characteristics of the fabric surface of the sowing include texture, color and defect type, and the model setting conditions include the recognition accuracy requirement and the computing resource limitation; S102. Input the picture to be annotated into the deep learning model, and generate a preliminary annotation result according to a preset confidence threshold; S103. Receive the operation information of the user on the preliminary annotation result, adjust the preliminary annotation result according to the operation information to obtain an annotation result, and adjust the deep learning model according to the operation information, where the operation information includes deletion, modification of categories and adjustment of bounding boxes; S104. Generate an annotation file according to the annotation result.

[0023] Different types of fabrics have unique texture features, such as plain, twill, and satin. These textures show different patterns and structures at both the micro and macro levels. For example, plain fabric has a flat surface and many interweaving points of warp and weft; twill fabric has obvious diagonal lines. Deep learning models need to be able to recognize these texture differences in order to accurately distinguish different types of fabrics. Fabrics come in a variety of colors, from single solid colors to complex printed patterns. Color information is very important for both fabric identification and defect detection. For example, some fabrics may have a specific color range, and color anomalies may indicate the presence of defects. The model needs to be able to accurately perceive and process color information, including color hue, saturation, and brightness. There are many types of fabric defects, such as holes, stains, skipped yarns, and color differences. Each defect has its own unique shape and characteristics. The model needs to be able to recognize different types of defects so that the location and type of defects can be accurately marked during the annotation process. Recognition accuracy requirements: Determine the recognition accuracy required by the model based on the actual application scenario and needs. For example, in high-end textile quality inspection, the model may be required to have a defect recognition accuracy of more than 99% to ensure product quality; while in some occasions where the accuracy requirements are relatively low, the recognition accuracy can be appropriately reduced. Computing resource limitations: Consider the computing resources available on the annotation platform, such as the performance and number of CPUs and GPUs, memory size, etc. When selecting a pre-trained model, you need to ensure that the model can run efficiently under the existing computing resources. If computing resources are limited, you may need to select a model with a small number of parameters and low computational complexity to ensure the real-time and stability of the annotation process. According to the fabric type and fabric characteristics and model setting conditions, select a suitable pre-trained deep learning model in the model library. The models in the model library may be pre-trained for different fabric types, defect types, and accuracy requirements. Once a matching model is found, it is loaded into the operating environment of the data annotation platform to prepare for subsequent image annotation work. Input the fabric image to be annotated into the loaded deep learning model. The image may be stored in different formats (such as JPEG, PNG, etc.). Before entering the model, some pre-processing operations may be required, such as resizing the image, normalizing the pixel values, etc., to ensure that the image meets the input requirements of the model. The deep learning model extracts and analyzes the features of the input image. The model automatically learns the feature patterns in the image through a multi-layer neural network structure, and identifies information such as fabric type, color, and defects. For example, convolutional neural networks have powerful feature extraction capabilities in image processing and can effectively capture features such as texture and edges in images. Based on the learned feature patterns, the model predicts the target in the image (such as fabric type area, defect location, etc.) and gives the confidence of each prediction. The confidence indicates the reliability of the model's prediction results, and the value range is usually between 0 and 1. A confidence threshold is preset, and the prediction result is output as a preliminary annotation result only when the confidence of the model prediction is higher than the threshold.For example, if the confidence threshold is set to 0.8, only predictions with a confidence greater than 0.8 will be retained. The preliminary annotation results may include information such as the category of the target (e.g., defect type), location, and bounding box. The user views the preliminary annotation results on the data annotation platform and operates on the annotation results according to the actual situation. The operation information includes deletion, modification of the category, and adjustment of the bounding box, etc. Deletion: If the user believes that a preliminary annotation result is incorrect or unnecessary, they can choose to delete the annotation. Modifying the category: When the model's prediction of the target category is inaccurate, the user can manually modify the annotated category. For example, modify a defect misjudged as "stain" to "hole". Adjusting the bounding box: The bounding box predicted by the model may not be accurate enough. The user can adjust its position and size by dragging the corner points or edges of the bounding box to make it more accurately enclose the target. According to the user's operation information, the preliminary annotation results are adjusted accordingly to obtain the final annotation results. The adjusted annotation results are more in line with the actual situation and can accurately reflect the information in the fabric image. The user's operation information reflects the deficiencies of the model during the annotation process. Using this operation information, the deep learning model is adjusted and optimized. The specific methods may include: using the annotated data adjusted by the user as new training samples to incrementally train the model so that the model learns more accurate features and annotation rules; adjusting the model's parameters, such as weights and biases, according to the user's operation feedback to improve the model's performance; continuously adjusting the model to make it better adapt to the fabric surface annotation task and improve the accuracy and efficiency of subsequent annotation. The final annotation results are sorted out to ensure the integrity and standardization of the annotation information. The annotation results usually include information such as the name of the image, the category of the target, location, and bounding box. According to the sorted annotation results, an annotation file is generated. The annotation file can adopt a common format, such as XML, JSON, etc. These formats have good readability and scalability, facilitating subsequent data storage, management, and use. For example, in an XML-format annotation file, specific tags can be used to represent image information, target category, and bounding box coordinates, etc. The generated annotation file can be used for training other machine learning models, data analysis, visualization, and other tasks.

[0024] Figure 2 is a schematic diagram of an annotated image disclosed in an embodiment of the present application. Figure 3 is a schematic diagram of an identification result disclosed in an embodiment of the present application, as Figure 3 shown, the part enclosed by the solid line is the identified bad needle. Figure 4 is a schematic diagram of editing annotation results disclosed in an embodiment of the present application, where Figure 4 (a) in represents that the annotation result is a horizontal bar, Figure 4 (b) in represents that the annotation result is flying flowers. Figure 5 is a schematic diagram of modifying the threshold disclosed in an embodiment of the present application, Figure 5In (a), it represents the annotation result with a horizontal bar threshold of 0.25. Figure 5 In (b), it represents the annotation result with a horizontal bar threshold of 0.92. When the system is unable to perform annotation, direct manual annotation can be carried out. Figure 6 This is the initial schematic diagram of direct annotation disclosed in the embodiments of the present application. As Figure 6 shown, direct annotation can create a new flying flower. Figure 7 This is the schematic diagram of the direct annotation process disclosed in the embodiments of the present application. As Figure 7 shown, the green line is during the process of creating a new flying flower. Figure 8 This is the schematic diagram of the direct annotation result disclosed in the embodiments of the present application. As Figure 8 shown, the flying flower annotation is completed. In combination with Figures 2 - 8 the following embodiments are specifically described.

[0025] Optionally, the adjusting the preliminary annotation result according to the operation information to obtain an annotation result includes: When the operation information is "delete", change the border color of the original annotation box in the preliminary annotation result to a preset first color. When the operation information is "category", replace the original category in the preliminary annotation result with a new category, and change the border color of the original annotation box in the preliminary annotation result to a preset second color. When the operation information is "edit", redraw the bounding box of the annotation area according to the trajectory generated by the user dragging the mouse, and change the border color of the original annotation box in the preliminary annotation result to a preset third color.

[0026] The user determines that a certain bounding box in the preliminary annotation result is mis-annotated or that area does not need to be annotated, so the user chooses to delete the bounding box. The border color of the original bounding box in the preliminary annotation result is changed to a preset first color. By changing the border color, it is visually shown to the user that the bounding box has been marked as deleted, distinguishing it from other normal or pending bounding boxes. At the same time, in subsequent data processing or model training, the system can recognize the bounding boxes with this specific color border and thus ignore the corresponding annotation information. The preset first color usually selects a color that is more prominent and has a large color difference from the normal bounding box color, such as red. Red is visually easy to notice and can enable the user to quickly identify the deleted bounding box. If the user believes that the category corresponding to a certain bounding box in the preliminary annotation result is inaccurate, the original category in the preliminary annotation result can be replaced with a new category. For example, the preliminary annotation misjudges a certain defect as "stain", and the user modifies it to "hole" according to the actual situation. The border color of the original bounding box in the preliminary annotation result is changed to a preset second color. Changing the border color can, on the one hand, allow the user to clearly see the bounding box whose category has been modified, facilitating the user's inspection and confirmation; on the other hand, in subsequent data processing, the system can identify the bounding boxes with modified categories based on the border color, and these annotations may need to be specially processed or further reviewed. The preset second color can be selected as a color that is different from the color of the deleted bounding box and has a certain degree of distinguishability, such as blue. Blue can not only highlight the state of category modification but also not be confused with the red color of the deleted state. The user finds that the boundary of a certain bounding box in the preliminary annotation result is inaccurate and cannot well enclose the target area, so the user drags the mouse to re-adjust the position and size of the bounding box. The boundary box of the annotation area is redrawn according to the trajectory generated by the user dragging the mouse. The system will capture the movement trajectory of the mouse in real-time and accurately draw a new bounding box based on this trajectory information to make it more conform to the actual boundary of the target area. The border color of the original bounding box in the preliminary annotation result is changed to a preset third color. By changing the border color, the user can visually see the bounding box that has been edited and adjusted. At the same time, in subsequent data processing, the system can recognize the bounding boxes with this specific color border, and process or mark the edited bounding boxes separately for quality inspection and model update. The preset third color can be selected as a color that is different from the colors of both the deleted and category-modified bounding boxes, such as green. Green usually represents the modified or updated state and can be clearly distinguished from the bounding boxes in other states.

[0027] Different color identifications enable the reviewers to quickly locate the modified parts in the annotation results. In the review work of a large amount of annotation data, the reviewers do not need to check the detailed information of each annotation box one by one, but only need to pay attention to the annotation boxes with color changes, thus greatly improving the review efficiency and reducing the review time and workload. By distinguishing different operation states through colors, the annotation managers can more conveniently monitor and manage the annotation process. For example, the number of annotation boxes with different colors can be counted to understand the frequencies of various operations (deletion, category modification, bounding box editing), analyze possible problems in the annotation process. For instance, that certain categories are often modified may imply deficiencies in the model's recognition of these categories, and thus targeted measures can be taken for improvement. As an intuitive prompt, the color change can help users perform operations more accurately during the operation process. In the operations of category modification and bounding box editing, the color prompt can also enable users to more clearly confirm their operation results. The unified color identification rule helps to standardize the user's operation process. When making annotation adjustments, users will operate according to the established color change rule, making the entire annotation process more standardized and consistent. This helps to improve the quality of the annotation data and reduce errors caused by non-standard operations. The annotation boxes of different colors record the adjustment information of the user to the preliminary annotation results, and this information can be used as an important basis for model optimization.

[0028] Optionally, adjusting the deep learning model according to the operation information includes: When it is detected that the user performs a deletion operation on the original annotation box, extract the image feature vector corresponding to the original annotation box, calculate the cosine similarity between the image feature vector and the weight vectors of each category in the misdetection weight matrix of the classifier in the deep learning model to determine the most similar category; Reduce the weight value of the most similar category in the misdetection weight matrix of the classifier according to a preset ratio.

[0029] During the training process, the deep learning model learns various feature representations of images. For the image region corresponding to the original annotation box, the model can extract the feature vector of this region through structures such as the previous convolutional layers. This feature vector is a highly abstract and generalized representation of the features of this image region, containing information about color, texture, shape, etc. Usually, the feature vector is obtained at a certain intermediate layer of the model (such as before the fully connected layer). For example, when using a convolutional neural network (CNN), after a series of convolutional and pooling operations, a feature map is obtained, and flattening it can obtain a feature vector. This feature vector can represent the features of the image region corresponding to the original annotation box. In the classifier of the deep learning model, each category has a corresponding weight vector, and these weight vectors form the misdetection weight matrix. The weight vector reflects the sensitivity of the model to the features of different categories, and the model classifies by calculating the similarity between the input feature vector and the weight vectors of each category. Cosine similarity is a method to measure the similarity between two vectors, and its value ranges from -1 to 1. By calculating the cosine similarity between the image feature vector and the weight vectors of each category, the similarity degree between the image features and the features of different categories can be obtained. Comparing the calculated cosine similarity values, finding the category with the largest similarity, this category is the most similar category. That is to say, the image features corresponding to the original annotation box are closest to the weight vector of this category in the feature space. The preset ratio is a pre-set parameter used to control the magnitude of weight reduction. It can be adjusted according to the actual situation. For example, if the misdetection situation of the model in a certain category is relatively serious, the preset ratio can be appropriately increased to reduce the weight of this category by a larger margin; conversely, if the misdetection situation is relatively light, the preset ratio can be appropriately reduced. Reducing the weight value of the most similar category in the misdetection weight matrix means that the sensitivity of the model to this category will be reduced in the subsequent classification process. Since the user deletes the original annotation box, it indicates that the model's judgment of this category may be incorrect. By reducing the weight, the possibility that the model misclassifies this image region as this category again can be reduced, thereby improving the classification accuracy of the model.

[0030] When it is detected that the user performs a deletion operation on the original annotation box, extract the image feature vector corresponding to the original annotation box, and calculate the cosine similarity between it and the weight vectors of each category in the misdetection weight matrix of the classifier in the deep learning model. Cosine similarity can measure the similarity degree of two vectors in terms of direction. Through this method, the category most similar to the features of the misdetected annotation box can be accurately found. After determining the most similar category, the root cause of the model's misdetection is clarified. This helps developers deeply understand the categories that the model is prone to misidentify, providing a clear direction for subsequent model optimization. Reduce the weight value of the most similar category in the misdetection weight matrix of the classifier according to a preset ratio. The misdetection weight matrix reflects the possibility of the model generating misdetections in different categories. Reducing the weight value of a certain category means reducing the tendency of the model to misclassify this category as other categories. By reducing the misdetection weight, the model will be more accurate in classification decisions. The misclassification situations that might originally occur due to a high misdetection weight will be effectively curbed, making the model more accurate in identifying various defects, improving the overall annotation accuracy and the reliability of the model. This adjustment method is based on the user's operation feedback and realizes the dynamic optimization of the model. When the user discovers a misdetection situation during the annotation process and performs a deletion operation, the model can promptly capture this information and adjust its own weight matrix. This dynamic adjustment mechanism enables the model to adapt to different annotation scenarios and data characteristics, continuously learning and improving its own performance. In practical applications, the data of the sowing cloth surface may be diverse and complex. By continuously adjusting according to the user feedback, the model can better adapt to different types and characteristics of cloth surface data, improving the generalization ability and adaptability of the model.

[0031] Optionally, adjusting the deep learning model according to the operation information includes: When it is detected that the user modifies the category label of the original annotation box, obtain the mapping relationship between the original category and the new category, and extract the feature map of the region corresponding to the original annotation box; Input the feature map into the incremental learning module of the classifier, generate the first prototype vector of the new category through sliding window average pooling, and calculate the difference vector between the first prototype vector and the second prototype vector of the original category; Determine the weight corresponding to the new category based on the difference vector, the update rate parameter, and the weight corresponding to the original category.

[0032] During the data annotation process, the system continuously monitors the user's operation behavior. When the user modifies the category label of the original annotation box, the system immediately detects this operation. For example, in the task of annotating fabric surface defects, the user originally marked a certain defect as "hole", and after further observation and analysis, changed it to "float". The system accurately identifies this modification operation by listening to the user's interaction events (such as clicking the modification button, selecting a new category, etc.). When the modification operation is detected, the system obtains the mapping relationship between the original category and the new category. This mapping relationship helps the model understand the conversion logic between different categories. For example, in the above example, the mapping relationship from "hole" to "float" will be recorded. In practical applications, a category mapping table may be constructed to store common category conversion situations for quick query and use when needed. The system extracts the feature map corresponding to the area of the original annotation box. The feature map is extracted from the input image through the convolutional layer of the deep learning model, and it contains rich image feature information of this area, such as texture, color, shape, etc. These features are crucial for the model's category recognition and classification. For example, for fabric surface defect images, the feature map can capture the unique features of the defects to help the model distinguish different types of defects. The extracted feature map is input into the incremental learning module of the classifier. Incremental learning is a method that allows the model to learn new knowledge without retraining the entire model. In the data annotation scenario, as the user continuously modifies the category label, the model needs to be able to quickly adapt to the new category information. The incremental learning module can process these new feature data, enabling the model to dynamically update and expand its knowledge. In the incremental learning module, the method of sliding window mean pooling is used to generate the first prototype vector of the new category. Sliding window mean pooling is a technique for dimensionality reduction and feature extraction of the feature map. The specific operation is to slide a fixed-size window on the feature map, calculate the mean of all feature values within the window, and use this mean as a new feature value. By sliding the window over the entire feature map, a more representative feature vector, that is, the first prototype vector, can be obtained. The first prototype vector represents the typical features of the new category in the feature space. Calculate the difference vector between the first prototype vector (new category) and the second prototype vector (original category). The second prototype vector is the typical feature vector of the original category that the model has learned during the previous learning process. The difference vector reflects the degree of difference between the new category and the original category in the feature space. By calculating the difference vector, the model can understand the feature changes of the new category relative to the original category, providing a basis for subsequent weight adjustment. The update rate parameter is a parameter that controls the update speed and amplitude of the new category weight. Its value range is usually between 0 and 1. If the update rate parameter is set larger, the model will adapt to the new category faster, but it may cause the model to be unstable; if the update rate parameter is set smaller, the model update speed will be slower, but it will be more stable.In practical applications, the appropriate update rate parameter can be selected according to the characteristics of the data and the requirements of the annotation task. Based on the difference vector, the update rate parameter, and the weight corresponding to the original category, the weight corresponding to the new category is determined. The specific calculation method may vary depending on the model, but usually a weighted calculation method is adopted. For example, the difference vector can be multiplied by the update rate parameter and then added to a part of the original category weight to obtain the weight of the new category. The purpose of doing this is to make the weight of the new category able to reflect the characteristics of the new category and at the same time be integrated with the original weight system of the model to ensure the stability and coherence of the model.

[0033] When it is detected that the user modifies the category label of the original annotation box, the model can immediately obtain the mapping relationship between the original category and the new category and extract the feature maps of the corresponding regions. By inputting these feature maps into the incremental learning module of the classifier, the model can quickly generate the first prototype vector of the new category. This mechanism enables the model to efficiently learn and adapt to new categories without having to retrain the entire dataset, greatly saving training time and computing resources. In actual annotation scenarios, new categories may continuously appear. This adjustment method allows the model to be updated in real time according to the user's operations, enabling the model to timely identify and process new categories, and improving the adaptability and practicality of the model in a dynamic environment. Using sliding window mean pooling to generate the first prototype vector of the new category can effectively extract and summarize the typical features of the new category in the feature space. At the same time, calculating the difference vector between the first prototype vector and the second prototype vector of the original category can accurately characterize the feature differences between the new category and the original category. This difference characterization helps the model better understand the boundaries and differences between different categories and improve the accuracy of category recognition. By clarifying the feature differences between the new category and the original category, the model can more accurately classify samples into the correct categories during the classification decision. Especially in the case where the feature similarities between categories are relatively high, the calculation of the difference vector provides a clearer classification basis for the model and reduces the possibility of misclassification. Determining the weight corresponding to the new category based on the difference vector, the update rate parameter, and the weight corresponding to the original category, where the update rate parameter plays a role in flexibly controlling the update amplitude of the model. The user can adjust the size of the update rate parameter according to the actual needs and application scenarios.

[0034] Optionally, calculating the difference vector between the first prototype vector and the second prototype vector of the original category includes: Calculating the cosine similarity and Euclidean distance between the first prototype vector and the second prototype vector, and performing weighted fusion on the cosine similarity and the Euclidean distance to generate a comprehensive difference degree; Determining a difference degree threshold according to the number of samples and annotation confidence of the original category, and judging whether the comprehensive difference degree is greater than or equal to the difference degree threshold; When the comprehensive difference degree is greater than or equal to the difference degree threshold, map the comprehensive difference degree to a vector space to obtain the difference vector.

[0035] Cosine similarity is used to measure the similarity degree of two vectors in terms of direction. By calculating the cosine similarity, the direction difference between the first prototype vector and the second prototype vector in the feature space can be understood, thereby reflecting the similarity degree of the feature distributions of the two categories. Euclidean distance is used to measure the actual distance between two vectors in space. The larger the value of the Euclidean distance, the farther the distance between the two vectors in space, that is, the greater the feature difference between the two categories. In order to comprehensively consider the differences in direction and distance between the two vectors, the cosine similarity and the Euclidean distance are weighted and fused. Let the weight of the cosine similarity be , and the weight of the Euclidean distance be (usually ), then the comprehensive difference degree can be expressed as: , represents the cosine similarity, represents the Euclidean distance, where It is to convert the cosine similarity into a difference degree, where a larger value indicates a greater difference. Through weighted fusion, a comprehensive index that more comprehensively reflects the difference between two prototype vectors can be obtained. The number of samples in the original category affects the model's understanding of the characteristics of that category. The more samples there are, the more accurately the model grasps the feature distribution of that category. The annotation confidence reflects the reliability of the annotation result. Considering these two factors comprehensively, a difference degree threshold can be determined. For example, when the number of samples in the original category is large and the annotation confidence is high, a relatively high difference degree threshold can be set because the model already has a relatively accurate understanding of that category, and a larger difference is required to consider that the new category is significantly different from the original category; conversely, when the number of samples is small or the annotation confidence is low, a lower difference degree threshold is set to more easily detect the difference between the new category and the original category. The calculated comprehensive difference degree is compared with the difference degree threshold. If the comprehensive difference degree is greater than or equal to the difference degree threshold, it indicates that the difference between the first prototype vector and the second prototype vector is significant enough, and it can be considered that the new category is significantly different from the original category; if the comprehensive difference degree is less than the difference degree threshold, it is considered that the new category is relatively similar to the original category in terms of features, and perhaps no major weight adjustment is required. When the comprehensive difference degree is greater than or equal to the difference degree threshold, in order to represent the difference degree information in the form of a vector for convenient subsequent processing in the model (such as weight adjustment), the comprehensive difference degree is mapped to the vector space. The specific mapping method can be designed according to actual needs. For example, the comprehensive difference degree can be multiplied by a unit vector, or it can be transformed into a vector with a specific direction and magnitude through a certain function transformation. The vector obtained after mapping is the difference vector, which represents the difference information between the new category and the original category in the feature space. This difference vector can be used to calculate the weight corresponding to the new category in subsequent calculations, enabling the model to reasonably adjust the weight of the new category according to the difference degree between the new category and the original category, so as to better adapt to the features of the new category.

[0036] By calculating the cosine similarity and Euclidean distance between the first prototype vector and the second prototype vector, and performing weighted fusion to generate the comprehensive difference degree, the difference between the two vectors can be comprehensively measured from different perspectives. The cosine similarity focuses on measuring the similarity of vectors in terms of direction, while the Euclidean distance focuses on the actual distance of vectors in space. Combining these two measurement methods can more accurately capture the difference information between vectors and avoid misjudgments caused by the limitations of a single measurement method. The weighted fusion method can adjust the weights of different measurement methods according to actual needs, making the comprehensive difference degree better reflect the key points of attention to vector differences in practical applications. For example, in some cases, the direction difference of vectors may be more important, and at this time, the weight of the cosine similarity can be appropriately increased; in other cases, the spatial distance of vectors may be more critical, and then the weight of the Euclidean distance can be increased. This flexibility helps to improve the accuracy of difference judgment. Determine the difference degree threshold according to the number of samples in the original category, so that the setting of the threshold can take into account the differences in the number of samples in different categories. For categories with a large number of samples, their prototype vectors may be more stable, so a relatively high difference degree threshold can be set to avoid misjudging as a large difference due to small fluctuations; for categories with a small number of samples, their prototype vectors may not be stable enough, and at this time, a lower difference degree threshold can be set to more sensitively capture possible differences. The annotation confidence reflects the reliability of the annotation result. Incorporating the annotation confidence into the determination process of the difference degree threshold can make the setting of the threshold more reasonable. If the annotation confidence of the original category is high, it means that the feature representation of this category is relatively reliable, and at this time, a relatively high difference degree threshold can be set; on the contrary, if the annotation confidence is low, the difference degree threshold needs to be reduced to more carefully judge the difference between vectors. The adaptive setting of the difference degree threshold enables the model to automatically adjust the judgment criteria according to the characteristics and annotation situations of different categories, improving the adaptability and robustness of the model in various complex scenarios. When the comprehensive difference degree is greater than or equal to the difference degree threshold, map the comprehensive difference degree to the vector space to obtain the difference vector. This mapping method enables the difference vector to intuitively represent the difference between the two prototype vectors and can be further analyzed and processed in the vector space. The representation method of the difference vector is convenient for the model to utilize the difference information in subsequent classification, recognition and other tasks. The model can make more accurate classification decisions on new categories according to the difference vector, or adjust the weights of the original categories using the difference vector, thereby improving the performance and accuracy of the model.

[0037] Optionally, determining the weight corresponding to the new category based on the difference vector, the update rate parameter, and the weight corresponding to the original category includes: Mapping the difference vector to the weight space through a fully connected layer to generate a weight offset; Adjust the update rate parameter according to the user operation frequency and the convergence state of the deep learning model, calculate the product of the weight offset and the update rate parameter, and obtain the weight of the new category based on the product and the weight corresponding to the original category.

[0038] The fully connected layer is a commonly used neural network layer in deep learning models. It can perform linear transformation and nonlinear activation on the input data to learn the complex mapping relationship between the data. Mapping the difference vector to the weight space through the fully connected layer means using the learning ability of the fully connected layer to convert the difference information between the new category represented by the difference vector and the original category into a weight offset that is instructive for weight adjustment. The fully connected layer calculates the input difference vector based on its internal weight parameters and bias parameters, and outputs a vector that matches the weight space. This vector is the weight offset. The weight offset indicates the direction and magnitude of the weight adjustment required for the new category relative to the original category. For example, if the difference vector indicates that the new category is significantly different from the original category in some feature dimensions, then the weight offset may have a larger value in these dimensions to guide the weight of the new category to be adjusted accordingly in these dimensions. The frequency of user operations reflects the user's attention to a certain category and the degree of active annotation. If the user frequently modifies the annotation box category label of a certain category, it means that there may be more uncertainty in this category or the model's recognition effect on this category is not good. At this time, the update rate parameter can be appropriately increased so that the model can learn new category information faster and adapt to user operation changes. For example, when the user frequently adjusts the labeling of a new category, increasing the update rate parameter can allow the model to adjust the weight of the new category faster and improve the recognition accuracy of the category. The convergence state of the deep learning model indicates the stability and performance of the model during the training process. If the model is close to convergence, it means that the model has learned the existing data sufficiently. At this time, the update rate parameter can be appropriately reduced to avoid excessive fluctuations in the model when introducing new category information and ensure the stability of the model. On the contrary, if the model has not converged, it means that the model is still in the process of continuous learning and optimization. At this time, the update rate parameter can be appropriately increased to speed up the model's learning of new category information. Taking into account the user operation frequency and the model convergence state, the update rate parameter is dynamically adjusted so that the model can be updated at an appropriate speed in different situations. This dynamic adjustment mechanism can improve the adaptability and robustness of the model, ensuring that the model can learn quickly and remain stable when introducing new category information. Calculating the product of the weight offset and the update rate parameter is actually scaling the weight offset. The update rate parameter plays a role in controlling the range of weight adjustment. Through product calculation, the change in the weight of the new category can be adjusted according to actual needs. If the update rate parameter is large, the impact of the weight offset will be amplified, and the weight of the new category will change greatly; conversely, if the update rate parameter is small, the impact of the weight offset will be reduced, and the weight change of the new category will be relatively smooth. The weight of the new category is obtained based on the product and the weight corresponding to the original category, usually by adding the product to the weight of the original category.The purpose of this is to introduce the difference information between the new category and the original category while preserving the weight information of the original category, so as to achieve a reasonable adjustment of the new category weight. For example, the original category weight is... ... and the product of the weight offset and the update rate parameter is... ... then the new category weight... In this way, during the learning process of the new category, the model can not only consider the knowledge of the original category but also adapt to the characteristics of the new category, improving the recognition ability of the new category.

[0039] The difference vector is mapped to the weight space through the fully connected layer to generate the weight offset. The fully connected layer has a strong nonlinear fitting ability and can learn the complex mapping relationship between the difference vector and the weight offset. Through this mapping, the difference information in the vector space can be accurately converted into the adjustment amount in the weight space, so that the weight of the new category can accurately reflect the difference between it and the original category. The difference pattern between different new categories and the original category is different. The flexibility of the fully connected layer enables it to adapt to various difference patterns and generate appropriate weight offsets. Whether it is the difference in feature dimension, feature intensity or feature distribution, the fully connected layer can capture these differences through learning and convert them into effective adjustments to the weight of the new category. Adjust the update rate parameter according to the user's operation frequency so that the model can adapt to the user's operation habits and needs. If the user frequently modifies or labels a category, it means that the category is more important in the current task or there is more uncertainty. At this time, the update rate parameter can be appropriately increased to enable the model to learn and adapt to the new category information faster. On the contrary, if the user's operation frequency is low, the update rate parameter can be reduced to maintain the stability of the model. The convergence state of the deep learning model reflects the current learning progress and stability of the model. In the early stage of model training, the model converges quickly. At this time, the update rate parameter can be appropriately increased to speed up the learning speed of the new category weights; in the later stage of model training, the model is close to convergence. At this time, the update rate parameter should be reduced to avoid the model from losing stability due to excessive updates. By adjusting the update rate parameter in combination with the model convergence state, the model can maintain good learning performance and stability at different stages. The product of the weight offset and the update rate parameter is calculated, and it is combined with the weight corresponding to the original category to obtain the weight of the new category. This calculation method not only takes into account the difference information between the new category and the original category, but also retains the existing weight information of the original category, so that the weight of the new category can reasonably integrate the characteristics of the new category on the basis of inheriting the characteristics of the original category. Combining the weight offset with the update rate parameter by multiplication can achieve smooth update of the weight. The update rate parameter plays a role in controlling the update amplitude, avoiding drastic changes in the weight update process, and ensuring the stability and continuity of the model. At the same time, this smooth update method also enables the model to better adapt to the new category information, gradually adjust the weight, and improve the accuracy of classification.

[0040] Optionally, adjusting the deep learning model according to the operation information includes: When the user drags the vertex of the original annotation box, the mouse trajectory coordinate sequence is obtained, and a new annotation box is generated by curve fitting; Calculate a coordinate difference matrix between the original annotation box and the new annotation box, calculate a loss gradient according to the coordinate difference matrix, and adjust the deep learning model according to the loss gradient.

[0041] During the training process of a deep learning model, a visualization interface is usually provided for users to adjust the original annotation boxes. When the user drags the vertices of the original annotation box, the system will monitor the movement trajectory of the mouse in real time. The coordinate information of the mouse during the dragging process is recorded at a certain time interval (for example, recorded multiple times per second), forming a coordinate sequence. This coordinate sequence reflects the process and intention of the user to adjust the annotation box. The mouse trajectory coordinate sequence may contain some noise and irregular fluctuations, and the new annotation box directly generated using this sequence may not be smooth and accurate enough. The method of curve fitting can be used to process the coordinate sequence to obtain a new annotation box that better conforms to the user's intention. Common curve fitting methods include polynomial fitting, spline curve fitting, etc. For example, when using polynomial fitting, a suitable polynomial function can be found according to the data points in the coordinate sequence, so that the function can best fit these data points. Through the polynomial function obtained by fitting, the vertex coordinates of the new annotation box can be calculated. According to the fitted coordinates, the position and size of the new annotation box are determined. The new annotation box should be able to accurately reflect the user's intention to adjust the original annotation box. Both the original annotation box and the new annotation box can be represented by the coordinates of four vertices. Suppose the vertex coordinates of the original annotation box are , , , , and the vertex coordinates of the new annotation box are , , , . For each corresponding vertex, calculate the coordinate differences in the x-axis and y-axis directions. For example, the coordinate difference of the first vertex is ( , ), and so on, and the coordinate differences of all vertices can be obtained. These coordinate differences are organized into a matrix, that is, the coordinate difference matrix. This matrix reflects the position difference between the original annotation box and the new annotation box. The loss function is used to measure the difference between the model prediction result and the true annotation. In this scenario, the difference between the original annotation box and the new annotation box can be used as part of the loss. The loss gradient is the partial derivative of the loss function with respect to the model parameters. By calculating the loss gradient, the change direction of the model parameters in the current state can be understood in order to adjust the model. The backpropagation algorithm can be used to calculate the loss gradient. The backpropagation algorithm starts from the loss function and calculates the gradient of each layer layer by layer until the gradient of the model parameters is obtained. In this process, the coordinate difference matrix, as part of the loss function, will participate in the calculation of the gradient. The common parameter update rule is the gradient descent method and its variants. Taking the stochastic gradient descent method (SGD) as an example, the update formula for the model parameters is , where is the parameter after the model update, is the current parameter of the model, is the learning rate, and is the loss gradient. The learning rate controls the step size of parameter updates. A larger learning rate allows the model to converge faster, but may cause the model to oscillate near the optimal solution; a smaller learning rate allows the model to converge more stably, but the convergence speed is slower. By adjusting the model parameters according to the loss gradient, the model can learn the user's intention to adjust the annotation box. In subsequent annotation or prediction tasks, the annotation box generated by the model will be closer to the result expected by the user. This adjustment method enables the model to continuously adapt to the user's operation feedback, improving the accuracy of annotation and the performance of the model.

[0042] The mouse trajectory records the user's hand movement information, reflecting the expected adjustment directions of the size, shape, and position of the annotation box. Using curve fitting to generate a new annotation box can avoid irregular or rough annotation boxes that may be generated when the user manually drags the vertices. The curve fitting algorithm can generate a smoother and more accurate new annotation box based on the mouse trajectory coordinate sequence, making it more conform to the actual boundary of the target and improving the accuracy and quality of annotation. Calculating the coordinate difference matrix between the original annotation box and the new annotation box can intuitively quantify the difference between the two annotation boxes. Each element in the coordinate difference matrix represents the difference between the original annotation box and the new annotation box at the corresponding coordinate position. By analyzing these differences, the changes in the annotation box in each direction can be understood. This matrix provides a specific quantitative basis for subsequent model adjustment. The model can judge the adjustment amplitude and direction of the annotation box based on the coordinate difference matrix, and then adjust its own parameters accordingly to better adapt to the user's annotation operation. Calculating the loss gradient based on the coordinate difference matrix, the loss gradient indicates the direction and amplitude that the model needs to adjust to reduce the difference between the original annotation box and the new annotation box in the current state. By calculating the loss gradient, the model can identify its deficiencies in which aspects, and then learn and improve targeted. Adjusting the deep learning model based on the loss gradient can prompt the model to learn more accurate feature representations and classification rules. For example, if the user often adjusts the annotation box in a specific area during the annotation process, the model can pay more attention to the feature information in this area under the guidance of the loss gradient, improving the recognition and classification ability of the target in this area.

[0043] This embodiment also discloses an intelligent data annotation system. Figure 7 is a schematic diagram of the modules of the intelligent data annotation system disclosed in the embodiments of the present application, as Figure 9 shown. The system includes a setting module 701, a processing module 702, an annotation module 703, and an output module 704, where: A setting module 701, configured to match and load a pre-trained deep learning model according to the characteristics of the fabric type and fabric surface and preset model setting conditions, where the fabric type and fabric surface characteristics include texture, color, and defect type, and the model setting conditions include recognition accuracy requirements and computing resource limitations; A processing module 702, configured to input an image to be labeled into the deep learning model and generate a preliminary labeling result according to a preset confidence threshold; A labeling module 703, configured to receive operation information of a user on the preliminary labeling result, adjust the preliminary labeling result according to the operation information to obtain a labeling result, and adjust the deep learning model according to the operation information, where the operation information includes deletion, modification of categories, and adjustment of bounding boxes; An output module 704, configured to generate a labeling file according to the labeling result.

[0044] Optionally, the labeling module 703 is configured to: When the operation information is deletion, change the border color of the original labeling box in the preliminary labeling result to a preset first color; When the operation information is a category, replace the original category in the preliminary labeling result with a new category, and change the border color of the original labeling box in the preliminary labeling result to a preset second color; When the operation information is editing, redraw the bounding box of the labeling area according to the trajectory generated by the user dragging the mouse, and change the border color of the original labeling box in the preliminary labeling result to a preset third color.

[0045] Optionally, the labeling module 703 is configured to: When it is detected that the user performs a deletion operation on the original labeling box, extract the image feature vector corresponding to the original labeling box, calculate the cosine similarity between the image feature vector and the weight vectors of each category in the misdetection weight matrix of the classifier in the deep learning model to determine the most similar category; Reduce the weight value of the most similar category in the misdetection weight matrix of the classifier according to a preset ratio.

[0046] Optionally, the labeling module 703 is configured to: When it is detected that the user modifies the category label of the original labeling box, obtain the mapping relationship between the original category and the new category, and extract the feature map of the area corresponding to the original labeling box; Input the feature map into the incremental learning module of the classifier, generate a first prototype vector of the new category through sliding window average pooling, and calculate the difference vector between the first prototype vector and the second prototype vector of the original category; Determine the weight corresponding to the new category based on the difference vector, the update rate parameter, and the weight corresponding to the original category.

[0047] Optionally, the annotation module 703 is configured to: Calculate the cosine similarity and the Euclidean distance between the first prototype vector and the second prototype vector, and perform weighted fusion on the cosine similarity and the Euclidean distance to generate a comprehensive difference degree; Determine a difference degree threshold according to the number of samples in the original category and the annotation confidence level, and determine whether the comprehensive difference degree is greater than or equal to the difference degree threshold; When the comprehensive difference degree is greater than or equal to the difference degree threshold, map the comprehensive difference degree to a vector space to obtain the difference vector.

[0048] Optionally, the annotation module 703 is configured to: Map the difference vector to a weight space through a fully connected layer to generate a weight offset; Adjust the update rate parameter according to the user operation frequency and the convergence state of the deep learning model, calculate the product of the weight offset and the update rate parameter, and obtain the weight of the new category according to the product and the weight corresponding to the original category.

[0049] Optionally, the annotation module 703 is configured to: When the user drags the vertex of the original annotation box, obtain the mouse trajectory coordinate sequence, and generate a new annotation box by curve fitting; Calculate the coordinate difference matrix between the original annotation box and the new annotation box, calculate the loss gradient according to the coordinate difference matrix, and adjust the deep learning model according to the loss gradient.

[0050] It should be noted that: when the device provided in the above embodiment realizes its functions, only the above-mentioned division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process can be seen in the method embodiment, which will not be repeated here.

[0051] This embodiment also discloses an electronic device. Refer to Figure 10 , the electronic device may include: at least one processor 801, at least one communication bus 802, a user interface 803, a network interface 804, and at least one memory 805.

[0052] Among them, the communication bus 802 is used to realize the connection and communication between these components.

[0053] Among them, the user interface 803 may include a display screen and a camera. Optionally, the user interface 803 may further include a standard wired interface and a wireless interface.

[0054] Among them, the network interface 804 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0055] Among them, the processor 801 may include one or more processing cores. The processor 801 connects various parts within the entire server through various interfaces and lines. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 805, and by calling the data stored in the memory 805, the processor 801 performs various functions of the server and processes data. Optionally, the processor 801 may be implemented in at least one of the following hardware forms: digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 801 may integrate one or a combination of several of the following: a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 801 and may be implemented separately by a single chip.

[0056] Among them, the memory 805 may include random access memory (RAM) and may also include read-only memory. Optionally, the memory 805 includes a non-transitory computer-readable storage medium. The memory 805 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 805 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store the data involved in the above-mentioned method embodiments. Optionally, the memory 805 may further be at least one storage device located far from the aforementioned processor 801. For example Figure 10As shown in the figure, the memory 805, which is a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program of the intelligent data annotation system.

[0057] In Figure 10 In the electronic device shown in the figure, the user interface 803 is mainly used to provide an interface for the user to input data and obtain the data input by the user; while the processor 801 can be used to call the application program of the intelligent data annotation system stored in the memory 805. When executed by one or more processors 801, the electronic device is caused to execute the method of one or more of the above embodiments.

[0058] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be adopted in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0059] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0060] In the several embodiments provided by this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some service interfaces. The indirect couplings or communication connections of the devices or units can be in electrical or other forms.

[0061] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0062] In addition, in each embodiment of this application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0063] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory 805. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 805 and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned memory 805 includes various media that can store program codes, such as USB flash drives, mobile hard disks, magnetic disks, or optical discs.

[0064] The foregoing are only exemplary embodiments of the present disclosure and should not be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure still fall within the scope covered by the present disclosure. After considering the disclosure of the specification, those skilled in the art will readily think of other implementation manners of the present disclosure. The present application aims to cover any variations, uses, or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include the common general knowledge or conventional technical means in the technical field not recorded in the present disclosure. The specification and the embodiments are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. An intelligent data annotation method, characterized in that, Applied to a data annotation platform, the method includes: According to the characteristics of the fabric type and surface and the preset model setting conditions, match and load a pre-trained deep learning model, where the fabric type and surface characteristics include texture, color, and defect type, and the model setting conditions include recognition accuracy requirements and computing resource limitations; Input the image to be annotated into the deep learning model, and generate a preliminary annotation result according to a preset confidence threshold; Receive the operation information of the user on the preliminary annotation result, adjust the preliminary annotation result according to the operation information to obtain an annotation result, and adjust the deep learning model according to the operation information, where the operation information includes deletion, modification of categories, and adjustment of bounding boxes; Generate an annotation file according to the annotation result.

2. The intelligent data annotation method according to claim 1, wherein The adjusting the preliminary annotation result according to the operation information to obtain an annotation result includes: When the operation information is deletion, change the border color of the original annotation box in the preliminary annotation result to a preset first color; When the operation information is about categories, replace the original category in the preliminary annotation result with a new category, and change the border color of the original annotation box in the preliminary annotation result to a preset second color; When the operation information is editing, redraw the bounding box of the annotation area according to the trajectory generated by the user dragging the mouse, and change the border color of the original annotation box in the preliminary annotation result to a preset third color.

3. The intelligent data annotation method according to claim 1, wherein The adjusting the deep learning model according to the operation information includes: When it is detected that the user performs a deletion operation on the original annotation box, extract the image feature vector corresponding to the original annotation box, calculate the cosine similarity between the image feature vector and the weight vectors of each category in the misdetection weight matrix of the classifier in the deep learning model to determine the most similar category; Reduce the weight value of the most similar category in the misdetection weight matrix of the classifier according to a preset ratio.

4. The intelligent data annotation method according to claim 1, wherein The adjusting the deep learning model according to the operation information includes: When it is detected that the user modifies the category label of the original annotation box, obtain the mapping relationship between the original category and the new category, and extract the feature map of the area corresponding to the original annotation box; Input the feature map into the incremental learning module of the classifier, generate a first prototype vector of the new category through sliding window average pooling, and calculate the difference vector between the first prototype vector and the second prototype vector of the original category; Determine the weight corresponding to the new category based on the difference vector, the update rate parameter, and the weight corresponding to the original category.

5. The intelligent data annotation method according to claim 4, wherein The calculating the difference vector between the first prototype vector and the second prototype vector of the original category includes: Calculate the cosine similarity and Euclidean distance between the first prototype vector and the second prototype vector, and perform weighted fusion on the cosine similarity and the Euclidean distance to generate a comprehensive difference degree; Determine a difference degree threshold according to the sample quantity and annotation confidence of the original category, and judge whether the comprehensive difference degree is greater than or equal to the difference degree threshold; When the comprehensive difference degree is greater than or equal to the difference degree threshold, map the comprehensive difference degree to the vector space to obtain the difference vector.

6. The intelligent data annotation method according to claim 5, wherein Determining the weight corresponding to the new category based on the difference vector, the update rate parameter, and the weight corresponding to the original category includes: Mapping the difference vector to a weight space through a fully connected layer to generate a weight offset; Adjusting the update rate parameter according to the user operation frequency and the convergence state of the deep learning model, calculating the product of the weight offset and the update rate parameter, and obtaining the weight of the new category according to the product and the weight corresponding to the original category.

7. The intelligent data annotation method according to claim 1, wherein, Adjusting the deep learning model according to the operation information includes: When the user drags the vertex of the original annotation box, obtaining the mouse trajectory coordinate sequence and generating a new annotation box by curve fitting; Calculating the coordinate difference matrix between the original annotation box and the new annotation box, calculating the loss gradient according to the coordinate difference matrix, and adjusting the deep learning model according to the loss gradient.

8. An intelligent data annotation system, characterized in that, Including a setting module, a processing module, an annotation module, and an output module, where: The setting module is configured to match and load a pre-trained deep learning model according to the fabric type and surface characteristics and the preset model setting conditions. The fabric type and surface characteristics include texture, color, and defect type, and the model setting conditions include the recognition accuracy requirement and the computing resource limitation; The processing module is configured to input the image to be annotated into the deep learning model and generate a preliminary annotation result according to a preset confidence threshold; The annotation module is configured to receive the operation information of the user on the preliminary annotation result, adjust the preliminary annotation result according to the operation information to obtain an annotation result, and adjust the deep learning model according to the operation information. The operation information includes deletion, modification of the category, and adjustment of the bounding box; The output module is configured to generate an annotation file according to the annotation result.

9. An electronic device, characterized in that, Including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. Both the user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, execute the method according to any one of claims 1-7.

Images