Picture data annotation and quality detection method, system and device and medium

Abstract

The invention provides a picture data annotation and quality detection method, system and device and a medium, and belongs to the technical field of picture data processing.The method comprises the following steps that a picture data annotation model is constructed based on an annotation task, and a pre-annotated public data set is used for training; obtaining a set number of picture data, and marking the picture data by using the picture data marking model to obtain a picture quality detection data set; constructing a machine learning model according to a predefined picture quality label, and training by using the picture quality detection data set to obtain a picture quality detection model; the method comprises the following steps: acquiring to-be-processed picture related data, performing data preprocessing, identifying a labeling task type by using a picture data labeling model, performing automatic labeling of the corresponding task type, performing automatic quality detection by using a picture quality detection model, and generating a picture quality detection report. According to the invention, automatic annotation and quality evaluation of the picture are realized, and the picture processing efficiency and processing effect are improved.

Description

Background Art

[0002] In the field of artificial intelligence, data labeling and image data quality testing are key links to ensure the quality of model training and image analysis. Traditional data labeling and image data quality testing are not only inefficient, but also cannot guarantee accuracy. For example, with the explosive growth of data volume, manual labeling can no longer meet the needs of large-scale data processing. In addition, manual labeling has strong subjectivity and poor consistency, which further affects the quality and reliability of data labeling. The traditional manual inspection method for image data quality testing often only evaluates a single image quality indicator, which lacks comprehensiveness and systematicness. In particular, with the diversification of image analysis applications, the requirements for image data quality are getting higher and higher. Traditional quality inspection methods can no longer meet the needs of modern applications.

[0003] Although some AI-based data annotation and image quality detection solutions have emerged, these solutions still have shortcomings in many aspects. First, due to unreasonable model design or insufficient training data, some solutions have low accuracy in annotation and detection results. Secondly, in terms of automation level, some solutions still require manual intervention and cannot achieve truly fully automated processing. Finally, some solutions only support a single type of annotation task, such as image classification, and cannot cover other more complex tasks. Some solutions only focus on a single image quality indicator in terms of quality detection, such as clarity, brightness, etc., but lack an in-depth understanding and analysis of image content, resulting in incomplete and inaccurate quality detection results, which cannot provide a reliable basis for subsequent image analysis.

[0004] This is a shortcoming of the prior art. Therefore, in view of the above-mentioned defects in the prior art, it is very necessary to provide a method, system, device and medium for image data annotation and quality detection. Summary of the invention

[0005] In view of the above-mentioned traditional image data annotation and quality detection in the prior art cannot meet the requirements of modern applications, the current artificial intelligence-based data annotation and image quality detection methods have low accuracy, cannot be truly automated, and only support a single annotation task type and a single image quality indicator. The present invention provides a method, system, device and medium for image data annotation and quality detection to solve the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a method for image data annotation and quality detection, comprising the following steps: S1. Build an image data annotation model based on the annotation task and use the pre-annotated public dataset for training; S2. Obtain a set amount of image data and annotate them using the image data annotation model to obtain an image quality detection dataset; S3. Build a machine learning model based on the predefined image quality labels, and use the image quality detection dataset for training to obtain an image quality detection model; S4. After obtaining the image-related data to be processed and performing data preprocessing, use the image data annotation model to identify the annotation task type and perform automatic annotation of the corresponding task type. Then use the image quality detection model to perform automatic quality detection and generate an image quality detection report.

[0007] Furthermore, the specific steps of step S1 are as follows: S11. Obtain image data, annotation type, and annotation data from a public dataset to generate an image annotation dataset; S12. Divide the image annotation dataset into annotated training data and annotated test data; S13. Obtain the type of annotation task to be processed, wherein the annotation task type includes image classification, object detection, and semantic segmentation; S14. Construct an image classification sub-model, an object detection sub-model, and a semantic segmentation sub-model, and connect each sub-model through a labeling task recognition channel to generate a network structure of a picture data labeling model; S15. Use the labeled training data in the image labeling dataset to train the network structure of the image data labeling model, adjust the hyperparameters until the image data labeling model is optimal, and then use the test dataset for testing until the model test results meet the requirements.

[0008] Furthermore, the specific steps of step S14 are as follows: S141. Build an image classification sub-model based on the VGG model architecture, build an object detection sub-model based on the YOLO architecture, and build a semantic segmentation sub-model based on the UNet architecture; S142. Establish a labeling task recognition channel, connect the back end of the task recognition channel with the front end of the image classification sub-model, the object detection sub-model and the semantic segmentation sub-model, and obtain a picture data labeling model network structure with the front end of the task recognition channel as input and the back end of each sub-model as output.

[0009] Furthermore, the specific steps of step S2 are as follows: S21. Obtain a set amount of image data and determine the type of image data annotation task; S22. Input the image data and the corresponding annotation task type into the image data annotation model, and output the annotation data corresponding to the annotation task type; S23. Use each image data and the corresponding annotation data as an image quality detection data set.

[0010] Furthermore, the specific steps of step S3 are as follows: S31. Define key indicators for image quality detection, including clarity, color accuracy, noise level, and contrast; S32. Constructing a feature extraction module for extracting features related to image data quality based on an image reasoning algorithm and a convolutional neural network; S33. Based on the defined key indicators of image quality detection and the constructed feature extraction module, an evaluation module for scoring based on image features is introduced to build a machine learning model; S34. Dividing the image quality detection data set into quality detection training data and quality detection test data; S35. Use the quality detection training data in the image quality detection dataset to train the machine learning model, learn the mapping relationship between image features and image quality, and optimize the hyperparameters and algorithms of the model during the training process; S36. Use the quality detection test data to test the trained model until it meets the requirements to obtain an image quality detection model.

[0011] Furthermore, the specific steps of step S4 are as follows: S41. Obtain the image-related data to be processed and perform preprocessing; S42. Input the preprocessed image-related data into the image data annotation model, identify the annotation task type, select the corresponding sub-model to annotate the corresponding type of image data, and obtain the annotation type; S43. Input the image-related data and the corresponding annotation type into the image quality detection model, extract the image features, and then score the image quality based on the defined key indicators; S44. Compare the image quality scoring result with the preset standard to determine whether the image meets the requirements. If it does not meet the requirements, output the problem description and suggested correction plan to generate an image quality inspection report.

[0012] Further, the picture-related data in step S41 includes pictures, videos or multimedia data related to pictures; Data preprocessing includes cleaning, format conversion, and data enhancement.

[0013] In a second aspect, the present invention provides a system for image data annotation and quality detection, comprising: The annotation model construction and training module is used to build an image data annotation model based on the annotation task and use pre-annotated public data sets for training; The quality inspection data set construction module is used to obtain a set amount of image data and annotate them using the image data annotation model to obtain an image quality inspection data set; The quality detection model training module is used to build a machine learning model based on predefined image quality labels and use the image quality detection dataset for training to obtain an image quality detection model; The image annotation and quality detection module is used to obtain the image-related data to be processed and perform data preprocessing, then use the image data annotation model to identify the annotation task type and perform automatic annotation of the corresponding task type, and then use the image quality detection model to perform automatic quality detection and generate an image quality detection report.

[0014] In a third aspect, the present invention provides an electronic device, comprising a processor and a memory; The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the electronic device executes the method described in the first aspect above.

[0015] In a fourth aspect, the present invention provides a storage medium, The storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method described in the first aspect.

[0016] The beneficial effects of the present invention are: The method, system, device and medium for image data annotation and quality detection provided by the present invention realize fast and accurate annotation and quality detection of image data by constructing an image data annotation model and an image quality detection model based on artificial intelligence; compared with traditional manual annotation and manual inspection methods, the processing time is greatly shortened and the overall work efficiency is improved; and different elements in the image can be accurately identified and annotated. At the same time, the image quality detection model can comprehensively and accurately evaluate the quality of the image by defining clear quality indicators and constructing a feature extraction module; the accuracy of annotation and detection is improved.

[0017] The present invention does not require human intervention. The entire process from image data acquisition, preprocessing, labeling to quality inspection can be completed automatically by the system, which not only reduces labor costs, but also avoids problems such as subjectivity and poor consistency that may be caused by manual operations.

[0018] The image data annotation model of the present invention can handle various annotation task types such as image classification, object detection and semantic segmentation, meeting the needs of different application scenarios; at the same time, the image quality detection model also covers multiple key quality indicators such as clarity, color accuracy, noise level and contrast, providing strong support for the comprehensive evaluation of image data.

[0019] It can be seen that compared with the prior art, the present invention has outstanding substantive features and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 It is a flow chart of the method for image data annotation and quality detection of the present invention.

[0022] Figure 2 It is a schematic diagram of the system for image data annotation and quality detection of the present invention. DETAILED DESCRIPTION

[0023] The method for image data annotation and quality detection can be applied to one or more electronic devices, wherein the electronic device is an electronic device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0024] An electronic device may be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), an interactive network television (IPTV), etc. The network in which the electronic device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] See also Figure 1 The following is a method for image data annotation and quality detection in a specific implementation, including the following steps: S1. Build an image data annotation model based on the annotation task and use the pre-annotated public dataset for training; S2. Obtain a set amount of image data and annotate them using the image data annotation model to obtain an image quality detection dataset; S3. Build a machine learning model based on the predefined image quality labels, and use the image quality detection dataset for training to obtain an image quality detection model; S4. After obtaining the image-related data to be processed and performing data preprocessing, use the image data annotation model to identify the annotation task type and perform automatic annotation of the corresponding task type. Then use the image quality detection model to perform automatic quality detection and generate an image quality detection report.

[0027] In this embodiment, the specific steps of step S1 are as follows: S11. Obtain image data, annotation type, and annotation data from a public dataset to generate an image annotation dataset; S12. Divide the image annotation dataset into annotated training data and annotated test data; S13. Obtain the type of annotation task to be processed, wherein the annotation task type includes image classification, object detection, and semantic segmentation; S14. Construct an image classification sub-model, an object detection sub-model, and a semantic segmentation sub-model, and connect each sub-model through the annotation task recognition channel to generate a network structure of the image data annotation model; the specific steps of step S14 are as follows: S141. Build an image classification sub-model based on the VGG model architecture, build an object detection sub-model based on the YOLO architecture, and build a semantic segmentation sub-model based on the UNet architecture; S142. Establish a labeling task recognition channel, connect the back end of the task recognition channel with the front end of the image classification sub-model, the object detection sub-model and the semantic segmentation sub-model, and obtain a picture data labeling model network structure with the front end of the task recognition channel as input and the back end of each sub-model as output; Specifically, load a pre-trained VGG16 model through the torchvision.models module of PyTorch, use the pre-trained VGG16 model as the image classification sub-model, and modify the last classification layer to adapt to the required categories as follows; num_classes = 10 # Assume there are 10 categories vgg_model = models.vgg16(pretrained=True) vgg_model.classifier[6] = nn.Linear(vgg_model.classifier[6].in_features, num_classes) The object detection sub-model based on the YOLO architecture is constructed as follows: class YOLOv3Detector(nn.Module): def __init__(self, num_classes=80, anchors=None, ...):super(YOLOv3Detector, self).__init__() pass def forward(self, x): pass The labeling task recognition channel is established and the sub-models are integrated in the following way: class ImageAnnotationModel(nn.Module): def __init__(self, vgg_classifier, yolov3_detector, unet_segmenter): super(ImageAnnotationModel, self).__init__() self.task_recognition_channel = nn.Sequential( nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1), nn.ReLU(inplace=True), nn.MaxPool2d(kernel_size=2, stride=2), #Finally output a vector representing the task type ) self.vgg_classifier = vgg_classifier self.yolov3_detector = yolov3_detector self.unet_segmenter = unet_segmenter S15. Use the labeled training data in the image labeling dataset to train the network structure of the image data labeling model, adjust the hyperparameters until the image data labeling model is optimal, and then use the test dataset to test until the model test results meet the requirements; During the training process, the loss function and optimizer of the image classification sub-model are defined as follows criterion = nn.CrossEntropyLoss() optimizer = torch.optim.SGD(vgg_model.parameters(), lr=0.001,momentum=0.9) The specific steps of step S2 are as follows: S21. Obtain a set amount of image data and determine the type of image data annotation task; S22. Input the image data and the corresponding annotation task type into the image data annotation model, and output the annotation data corresponding to the annotation task type; def forward(self, x, task_type): # Select the corresponding sub-model for processing according to the task type # Here task_type should be an integer or tensor representing the task type # For example, 0 means classification, 1 means detection, and 2 means segmentation if task_type == 0: # Classify the image return self.vgg_classifier(x) elif task_type == 1: # Detect objects return self.yolov3_detector(x) elif task_type == 2: # Perform semantic segmentation on the image return self.unet_segmenter(x) else: raise ValueError("Unknown task type") # Instantiate the model vgg_classifier = VGGClassifier(num_classes=10) # Assume there are 10 classification categories # yolov3_detector = YOLOv3Detector(num_classes=20)# Assume there are 20 detection categories and need to be actually implemented # unet_segmenter = UNetSegmenter(in_channels=3, out_channels=1)# Assuming it is binary semantic segmentation, actual implementation is required S23. Using each image data and corresponding annotation data as an image quality detection data set; The specific steps of step S3 are as follows: S31. Define key indicators for image quality detection, including clarity, color accuracy, noise level, and contrast; S32. Constructing a feature extraction module for extracting features related to image data quality based on an image reasoning algorithm and a convolutional neural network; S33. Based on the defined key indicators of image quality detection and the constructed feature extraction module, an evaluation module for scoring based on image features is introduced to build a machine learning model; S34. Dividing the image quality detection data set into quality detection training data and quality detection test data; S35. Use the quality detection training data in the image quality detection dataset to train the machine learning model, learn the mapping relationship between image features and image quality, and optimize the hyperparameters and algorithms of the model during the training process; S36. Use the quality detection test data to test the trained model until it meets the requirements to obtain an image quality detection model; The specific steps of step S4 are as follows: S41. Obtaining image-related data to be processed and performing preprocessing; image-related data includes images, videos, or multimedia data related to images; data preprocessing includes cleaning, format conversion, and data enhancement; S42. Input the preprocessed image-related data into the image data annotation model, identify the annotation task type, select the corresponding sub-model to annotate the corresponding type of image data, and obtain the annotation type; S43. Input the image-related data and the corresponding annotation type into the image quality detection model, extract the image features, and then score the image quality based on the defined key indicators; S44. Compare the image quality scoring result with the preset standard to determine whether the image meets the requirements. If it does not meet the requirements, output the problem description and suggested correction plan to generate an image quality inspection report.

[0028] It should be understood that the order of execution of the steps in the above embodiment does not mean that the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0029] The following is an embodiment of a system for image data labeling and quality detection provided by an embodiment of the present disclosure. The system and the methods for image data labeling and quality detection in the above-mentioned embodiments belong to the same inventive concept. For details not fully described in the embodiments of the system for image data labeling and quality detection, reference can be made to the embodiments of the methods for image data labeling and quality detection in the above-mentioned embodiments.

[0030] like Figure 2 As shown, the system includes: The annotation model construction and training module is used to build an image data annotation model based on the annotation task and use pre-annotated public data sets for training; The quality inspection data set construction module is used to obtain a set amount of image data and annotate them using the image data annotation model to obtain an image quality inspection data set; The quality detection model training module is used to build a machine learning model based on predefined image quality labels and use the image quality detection dataset for training to obtain an image quality detection model; The image annotation and quality detection module is used to obtain the image-related data to be processed and perform data preprocessing, then use the image data annotation model to identify the annotation task type and perform automatic annotation of the corresponding task type, and then use the image quality detection model to perform automatic quality detection and generate an image quality detection report.

[0031] The method for image data annotation and quality detection provided by the present invention is a combination of the modules and algorithm steps of each example described in the embodiments disclosed herein, and can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0032] Those skilled in the art will appreciate that various aspects of the method for image data annotation and quality detection provided by the present invention can be implemented as a system, method or program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which can be collectively referred to as "circuit", "module" or "system" here.

[0033] In the non-transitory computer-readable storage medium storing the method of image data annotation and quality detection, it can be but not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0034] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.

Claims

1. A method for image data annotation and quality detection, characterized in that: The steps include: S1. Build an image data annotation model based on the annotation task and use the pre-annotated public dataset for training; S2. Obtain a set amount of image data and annotate them using the image data annotation model to obtain an image quality detection dataset; S3. Build a machine learning model based on the predefined image quality labels, and use the image quality detection dataset for training to obtain an image quality detection model; S4. After obtaining the image-related data to be processed and performing data preprocessing, use the image data annotation model to identify the annotation task type and perform automatic annotation of the corresponding task type. Then use the image quality detection model to perform automatic quality detection and generate an image quality detection report.

2. The method for image data labeling and quality detection according to claim 1, characterized in that: The specific steps of step S1 are as follows: S11. Obtain image data, annotation type, and annotation data from a public dataset to generate an image annotation dataset; S12. Divide the image annotation dataset into annotated training data and annotated test data; S13. Obtain the type of annotation task to be processed, wherein the annotation task type includes image classification, object detection, and semantic segmentation; S14. Construct an image classification sub-model, an object detection sub-model, and a semantic segmentation sub-model, and connect each sub-model through a labeling task recognition channel to generate a network structure of a picture data labeling model; S15. Use the labeled training data in the image labeling dataset to train the network structure of the image data labeling model, adjust the hyperparameters until the image data labeling model is optimal, and then use the test dataset for testing until the model test results meet the requirements.

3. The method for image data labeling and quality detection according to claim 2, characterized in that: The specific steps of step S14 are as follows: S141. Build an image classification sub-model based on the VGG model architecture, build an object detection sub-model based on the YOLO architecture, and build a semantic segmentation sub-model based on the UNet architecture; S142. Establish a labeling task recognition channel, connect the back end of the task recognition channel with the front end of the image classification sub-model, the object detection sub-model and the semantic segmentation sub-model, and obtain a picture data labeling model network structure with the front end of the task recognition channel as input and the back end of each sub-model as output.

4. The method for image data labeling and quality detection according to claim 2, characterized in that: The specific steps of step S2 are as follows: S21. Obtain a set amount of image data and determine the type of image data annotation task; S22. Input the image data and the corresponding annotation task type into the image data annotation model, and output the annotation data corresponding to the annotation task type; S23. Use each image data and the corresponding annotation data as an image quality detection data set.

5. The method for image data labeling and quality detection according to claim 4, characterized in that: The specific steps of step S3 are as follows: S31. Define key indicators for image quality detection, including clarity, color accuracy, noise level, and contrast; S32. Constructing a feature extraction module for extracting features related to image data quality based on an image reasoning algorithm and a convolutional neural network; S33. Based on the defined key indicators of image quality detection and the constructed feature extraction module, an evaluation module for scoring based on image features is introduced to build a machine learning model; S34. Dividing the image quality detection data set into quality detection training data and quality detection test data; S35. Use the quality detection training data in the image quality detection dataset to train the machine learning model, learn the mapping relationship between image features and image quality, and optimize the hyperparameters and algorithms of the model during the training process; S36. Use the quality detection test data to test the trained model until it meets the requirements to obtain an image quality detection model.

6. The method for image data annotation and quality detection according to claim 5, characterized in that: The specific steps of step S4 are as follows: S41. Obtain the image-related data to be processed and perform preprocessing; S42. Input the preprocessed image-related data into the image data annotation model, identify the annotation task type, select the corresponding sub-model to annotate the corresponding type of image data, and obtain the annotation type; S43. Input the image-related data and the corresponding annotation type into the image quality detection model, extract the image features, and then score the image quality based on the defined key indicators; S44. Compare the image quality scoring result with the preset standard to determine whether the image meets the requirements. If it does not meet the requirements, output the problem description and suggested correction plan to generate an image quality inspection report.

7. The method for image data labeling and quality detection according to claim 6, characterized in that: The picture-related data in step S41 includes pictures, videos or multimedia data related to pictures; Data preprocessing includes cleaning, format conversion, and data enhancement.

8. A system for image data annotation and quality detection, characterized in that: include: The annotation model construction and training module is used to build an image data annotation model based on the annotation task and use pre-annotated public data sets for training; The quality inspection data set construction module is used to obtain a set amount of image data and annotate them using the image data annotation model to obtain an image quality inspection data set; The quality detection model training module is used to build a machine learning model based on predefined image quality labels and use the image quality detection dataset for training to obtain an image quality detection model; The image annotation and quality detection module is used to obtain the image-related data to be processed and perform data preprocessing, then use the image data annotation model to identify the annotation task type and perform automatic annotation of the corresponding task type, and then use the image quality detection model to perform automatic quality detection and generate an image quality detection report.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the electronic device executes the method described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method described in any one of claims 1 to 7.

Images