Deep learning-based data set preprocessing and label management method and system

Abstract

The invention belongs to the technical field of batch image processing. The data set preprocessing and label management method based on deep learning comprises the steps that a plurality of initial images are acquired and preprocessed, a plurality of target images with main objects are obtained, and the main objects represent main targets or objects needing to be recognized and concerned in the target images; performing label classification on the target image with the main body object by using a pre-training model to obtain a plurality of labels corresponding to the target image; according to the modification of the plurality of tags, adjusting a credibility threshold to obtain a plurality of target tags; and screening the plurality of target images according to the target labels to obtain a plurality of to-be-processed images. By means of the method, the processing efficiency of the image data can be improved, the credibility threshold value is adjusted to generate multiple types of labels, the diversity of the image data is improved, and particularly the work efficiency of style migration and personalized content creation is remarkably improved.

Description

Technical Field

[0001] The present application belongs to the technical field of batch image processing, and in particular, relates to a method and system for data set preprocessing and label management based on deep learning. Background Art

[0002] In the field of artificial intelligence, especially in image generation and style transfer tasks, LoRA (Low-Rank Adaptation) technology has attracted much attention due to its efficiency and flexibility. LoRA allows large pre-trained models (such as Stable Diffusion) to be quickly adapted to specific domains or tasks under limited computing resources. However, existing technologies have significant limitations in data preprocessing and label management for LoRA model training:

[0003] Insufficient data diversity: Traditional data preprocessing methods make it difficult to efficiently collect and screen data sets of specific styles or themes, limiting the adaptability of the LoRA model in diverse application scenarios.

[0004] Low label accuracy: Accurate style description and attribute labels are essential for style transfer and personalized content creation. However, manual labeling is not only time-consuming but also highly subjective, making it difficult to ensure the consistency and accuracy of labels.

[0005] Based on the above reasons, there is an urgent need for a dataset preprocessing and label management method based on deep learning. Summary of the invention

[0006] Based on this, it is necessary to provide a dataset preprocessing and label management method based on deep learning to address the above technical problems.

[0007] In a first aspect, the present application provides a method for data set preprocessing and label management based on deep learning, comprising:

[0008] Acquire a plurality of initial images, and pre-process the plurality of initial images to obtain the plurality of target images having subject objects, wherein the subject objects represent main targets or objects that need to be identified and focused on in the target images;

[0009] Using the pre-trained model, label classification is performed on the target image having the subject object to obtain a number of labels corresponding to the target image;

[0010] According to the modification of the plurality of labels, the credibility threshold is adjusted to obtain a plurality of target labels;

[0011] According to the target label, a plurality of the target images are screened to obtain a plurality of images to be processed.

[0012] In some practicable manners, the step of acquiring a plurality of initial images and preprocessing the plurality of initial images to obtain a plurality of target images having a subject object includes:

[0013] Adjusting the initial images to obtain a plurality of standard images, wherein the standard images represent images obtained after color correction is performed on the initial images;

[0014] Processing the standard image to obtain an image of optimal size;

[0015] The main object in the optimal size image is identified to obtain the target image having the main object.

[0016] In some practicable manners, the step of adjusting the initial images to obtain the standard images includes:

[0017] Using the ResNet50 deep learning model, extracting features from the initial image to obtain image features of the initial image;

[0018] Analyze the initial image according to the image features of the initial image to obtain the main object of the initial image;

[0019] The standard image is determined according to the subject object.

[0020] In some practicable manners, the step of using a pre-trained model to classify labels of the target image having the subject object to obtain a plurality of labels corresponding to the target image includes:

[0021] Get the label credibility value;

[0022] A multi-label classification model of the Transformer architecture is used to combine with an image feature vector to obtain a predicted probability label, wherein the image feature vector represents the image feature converted into a vector, and the predicted probability label represents the credibility probability of each label;

[0023] Compare the predicted probability label with the label credibility value, and if they are greater than or equal to each other, obtain the label;

[0024] The labels are classified using a multi-label classification model of a Transformer architecture to obtain a number of labels corresponding to the target image.

[0025] In some practicable manners, the step of adjusting the credibility threshold according to the modification of the plurality of labels to obtain the plurality of target labels includes:

[0026] Acquire a modification tag, wherein the modification tag is at least one of the tags corresponding to the target image;

[0027] According to the modified label, the label credibility value is adjusted:

[0028] If a number of labels corresponding to the target image modified by the user are greater than or equal to a preset value, the label credibility value is reduced to obtain more target labels;

[0029] If a number of labels corresponding to the target image modified by the user is less than a preset value, the label credibility value is increased to obtain fewer target labels.

[0030] In some practicable manners, the step of screening a plurality of target images according to the target tag to obtain a plurality of images to be processed includes:

[0031] Acquire a target image data set, wherein the target image data set includes a plurality of the target images;

[0032] According to the target label, as a screening condition, screening is performed in the target image data set to obtain a number of images to be processed.

[0033] In some practicable manners, after the step of screening a plurality of target images according to the target tag to obtain a plurality of images to be processed, the method further includes:

[0034] The U-Net deep learning architecture is used to segment the images to be processed to obtain images of the main objects.

[0035] In a second aspect, the present application provides a data set preprocessing and label management system based on deep learning, which is applied to the aforementioned data set preprocessing and label management method based on deep learning, and the system includes:

[0036] A preprocessing unit, used to acquire a plurality of initial images, and preprocess the plurality of initial images to obtain the plurality of target images having subject objects, wherein the subject objects represent main targets or objects that need to be identified and focused on in the target images;

[0037] A label generation unit, used to classify the target image with the subject object by using the pre-trained model, and obtain a plurality of labels corresponding to the target image;

[0038] The label generation unit is further used to adjust the credibility threshold according to the modification of the plurality of labels to obtain a plurality of target labels;

[0039] The result unit is used to screen the target images according to the target label to obtain a number of images to be processed.

[0040] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the aforementioned deep learning-based data set preprocessing and label management method when executing the computer program.

[0041] In a fourth aspect, the present application provides a computer program, which, when executed by a processor, implements the steps of the aforementioned deep learning-based data set preprocessing and label management method.

[0042] Beneficial effects:

[0043] The present application provides a data set preprocessing and label management method based on deep learning. First, a number of initial images are acquired, and the several initial images are preprocessed to obtain the several target images with subject objects, wherein the subject objects represent the main targets or objects that need to be identified and paid attention to in the target images; then, the target images with subject objects are labeled and classified using a pretrained model to obtain a number of labels corresponding to the target images; next, according to the modification of the several labels, the credibility threshold is adjusted to obtain a number of target labels; finally, according to the target labels, the several target images are screened to obtain a number of images to be processed. Through the above method, it is possible to preprocess several initial images, obtain target images, label the target images, modify the content of the labels according to user needs, adjust the credibility threshold of the generated labels according to the frequency, generate target labels according to the adjusted credibility threshold, use the target labels to screen the target images, and finally obtain several images to be processed. Next, operations such as cropping can be performed on the images to be processed. Using the above method, the processing efficiency of image data can be improved, the credibility threshold can be adjusted to generate various types of labels, and the diversity of image data can be increased, especially the work efficiency of style transfer and personalized content creation can be significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 is a flowchart of a data set preprocessing and label management method based on deep learning in one embodiment;

[0046] Figure 2 A logic diagram of a data set preprocessing and label management method based on deep learning in one embodiment;

[0047] Figure 3 A schematic diagram of automatic labeling results of a data set preprocessing and label management method based on deep learning in one embodiment;

[0048] Figure 4 The present invention is a schematic diagram of the operation of batch processing of images according to a method for dataset preprocessing and label management based on deep learning in one embodiment. DETAILED DESCRIPTION

[0049] In order to facilitate understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. Embodiments of the present application are provided in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all couplings of one or more related listed items.

[0051] It can be understood that the terms "first", "second", etc. used in the present application can be used in this article to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from another element.

[0052] Some terms involved in this application are explained below to facilitate understanding of this application:

[0053] OpenCV library, Open Source Computer Vision Library, is an open source computer vision and machine learning software library that provides a wide range of image processing functions, including image reading, writing, conversion, filtering, geometric transformation, etc.

[0054] MS COCO, Microsoft Common Objects in Context, is a widely used computer vision dataset, mainly used for image recognition, object detection, segmentation, key point detection and other tasks. It aims to provide a multi-class dataset with rich scenes and complex backgrounds to promote the performance of deep learning models in real scenes.

[0055] The Transformer architecture is a deep learning architecture for processing sequence data and is widely used in natural language processing (NLP) tasks such as machine translation, text generation, and text classification.

[0056] U-Net deep learning architecture is a deep learning architecture for image segmentation tasks and has been widely used in various image segmentation tasks.

[0057] React is an open source JavaScript library for building user interfaces, mainly used to build the view layer in single page applications (SPA).

[0058] like Figure 1 and Figure 2 As shown, in the first aspect, the present application provides a data set preprocessing and label management method based on deep learning, the method comprising:

[0059] S100, acquiring a plurality of initial images, and preprocessing the plurality of initial images to obtain the plurality of target images having a main object.

[0060] The main object refers to the main target or object in the target image that needs to be identified and focused on.

[0061] Specifically, obtaining the plurality of target images having the subject object comprises the following steps:

[0062] S101, adjusting a plurality of the initial images to obtain a plurality of standard images.

[0063] The standard image refers to an image obtained by color-correcting the initial image.

[0064] S102, processing the standard image to obtain an image of optimal size.

[0065] S103, identifying the main object in the optimal size image to obtain the target image having the main object.

[0066] Specifically, the uploaded initial image is adjusted, that is, the original image data, so that the adjusted image can be convenient for subsequent processing. Exemplarily, the ResNet50 deep learning model is used to adjust several of the initial images, that is, to optimize the target size, and obtain several standard images, including the following steps:

[0067] S1011, using the ResNet50 deep learning model, perform feature extraction on the initial image to obtain image features of the initial image.

[0068] S1012: Analyze the initial image according to the image features of the initial image to obtain the main object of the initial image.

[0069] S1013: Determine the standard image according to the main object.

[0070] Specifically, the ResNet50 deep learning model is used to extract features of the standard image to obtain the image features of the standard image. ResNet50 gradually extracts image features through multi-layer convolution operations. In ResNet50, starting from the input layer, the network first extracts low-level features of the initial image (such as edges, textures, etc.). As the number of network layers increases, the features become more complex, and finally high-level features are extracted in the deep layer (such as the shape and category information of the main object, and high-level features refer to features with the main object, that is, according to the image features of the initial image, the initial image is analyzed to obtain the main object features). The output of each layer can be regarded as the feature map extracted by the layer, and these feature maps are obtained by performing convolution operations on the feature maps of the previous layer.

[0071] After obtaining the high-level features, the standard image is processed according to the high-level features to obtain an image of optimal size. That is, when the high-level features represent the main object, the standard image is determined according to the main object.

[0072] Among them, the optimal size image means selecting the most suitable resolution under the preset size resolution. For example, the size standards of the picture are first unified as 512×512px (pixels), 768×768px, 512×768px, 768×512px and 1024×1024px. Next, the features obtained according to the deep learning model are judged. The judgment criteria can be pre-set, such as detail level, composition, color richness, etc., which constitute the detail complexity of the picture. Next, the detail complexity judgment can be performed by setting multiple thresholds. For images with rich details (such as high-resolution product images), 1024×1024px is recommended. For images of medium complexity (such as portraits), 768×768px is recommended. For simple images or scenes with limited storage space, 512×512px is recommended. In other words, a suitable resolution, that is, the optimal size, is selected according to the complexity of the initial image. In other words, in subsequent steps, the optimal size is used to perform the corresponding steps.

[0073] It should be noted that the skip connections of residual networks (such as ResNet50) can effectively transfer useful features and prevent useful information from being lost in deep layers. With this design, deep learning models are better able to retain high-level features and filter out irrelevant low-level features. This helps to accurately identify and process important parts of the image, thereby determining the optimal size of the image region. During the training process of the deep learning model, through the back-propagation algorithm, the model automatically learns which features are most useful for tasks (such as image classification). Low-level features are gradually merged and transformed into more complex high-level features. Features that are not important for the classification task will be "ignored" during the training process, and the model will pay more attention to useful features.

[0074] For trained models, high-level features can be screened by analyzing the network's feature maps. The degree of activation in the feature map can indicate which image areas the network is more sensitive to. By observing and analyzing the feature maps of different layers, feature maps with higher activation values ​​can be screened out. These feature maps usually represent the network's response to important features in the image. In addition, global average pooling is often used for feature screening. The values ​​of each feature map are aggregated into a single value through average pooling. These aggregated values ​​represent the importance of the network to different features in the entire image. Through global average pooling, the features that contribute most to the standard image can be screened out, thereby retaining high-level features.

[0075] It should also be noted that the high-level features can be used to input into the YOLO v5 algorithm (a conventional target detection algorithm). Next, the high-level features are used as the focus of the image in the YOLO v5 algorithm, and the image is cropped with this as the center, i.e., center cropping. For example, in step S1011, the high-level features can be input into the YOLO v5 algorithm to obtain the detection result.

[0076] In some embodiments, processing the standard image to obtain an image of optimal size may include the following method:

[0077] For standard image cropping, the YOLO v5 algorithm can be used to focus the image, retain the focus position, and crop around the focus position to improve the visual focus effect, namely, focus cropping, which includes the following steps:

[0078] S1014, using the YOLO v5 algorithm to perform target detection on the input standard image, identify the main object in the standard image, and obtain a detection result.

[0079] It should also be noted that in order to ensure high accuracy, the YOLO v5 model was trained on the widely recognized MS COCO dataset, and its mean average precision (mAP) reached 0.89, with excellent target detection performance. In other words, YOLO v5 is a trained model. Use the OpenCV library to read the input standard image to ensure that the standard image format is suitable for the model input. Specifically, the read standard image is converted to the input format required by YOLO v5 (such as adjusting the image size to the input size of the model, such as 640x640, and normalizing it). Next, the processed standard image is input into YOLOv5, and YOLOv5 will output the detection result, which includes the category of each detected subject object (such as people, cars, animals, etc.), confidence (Confidence Score) and the position of the bounding box (boundingbox). Parse the output results of YOLOv5 to extract all detected subject objects in the standard image. This subject object information will be used for subsequent image cropping operations.

[0080] S1015, determining a cropping area of ​​the standard image according to the detection result, to ensure that the main object occupies a prominent position in the cropped standard image.

[0081] It should be noted that the detection results of YOLOv5 are analyzed to determine which main objects in the standard image need to be retained. Select detection results with higher confidence (the confidence threshold can be set, and those above the confidence threshold are determined as detection results with higher confidence) to ensure that the selected objects are accurate and reliable. According to the bounding box of each main object, a minimum rectangular box (minimum circumscribed rectangle) surrounding these objects is calculated, which will be used as the area for standard image cropping. If there are multiple main objects in the standard image, you can select the minimum rectangular box surrounding all the main objects, or select only one of the main objects for cropping according to specific needs. In order to ensure that the cropping area is as large as possible to include the integrity of the main object, while removing unnecessary background as much as possible.

[0082] S1016, cropping the standard image to remove unnecessary background and other elements and highlight the main object.

[0083] It should be noted that the standard image is adjusted (e.g., the initial image is scaled) using the cropping function of the YOLOv5 library (i.e., the specified area in the initial image is intercepted by array indexing. YOLOv5 itself does not have a cropping function, but the YOLOv5 library can be used to implement cropping). The cropping area determined in step S1015 is retained, and other unnecessary parts in the standard image are removed. Through cropping, the main object in the standard image will be more prominent, and the removed background and other irrelevant elements will help to highlight the main object, thereby enhancing the visual effect.

[0084] It should also be noted that the OpenCV library can also be used to perform color correction on the standard image in order to reduce the amount of calculation in subsequent steps.

[0085] Use the OpenCV library to read images or video frames and perform color correction. Color correction refers to adjusting the color balance of an image to make the colors in the image more natural or meet specific requirements, including adjusting white balance, brightness, contrast, saturation, etc., in order to improve the visual effect of the image or restore the true color of the image.

[0086] S200, using a pre-trained model, classifying the target image with the subject object by label, and obtaining a plurality of labels corresponding to the target image.

[0087] Specifically, obtaining a number of labels corresponding to the target image includes the following steps:

[0088] S201, obtaining a tag credibility value.

[0089] The label credibility value can be understood as confidence, and the value range is between 0 and 1, which indicates the reliability of the label. The larger the value, the higher the reliability. It should be noted that the credibility value can be a predefined value, or it can be expressed based on historical data or the score output by the model.

[0090] Specifically, the scoring of historical data output to represent the label includes the following steps:

[0091] S2011, according to the occurrence frequency of each tag in the historical data, obtain a first tag credibility value of the tag. For example, if a tag appears 80 times in 100 samples, its credibility can be set to 0.8.

[0092] S2012, obtaining a label data set according to the label information in the historical data. Exemplarily, the historical data represents a plurality of historical images, each of which has a plurality of labels. And at least one label in one historical image may be the same as at least one label in another historical image.

[0093] S2013, extract unique tags from the tag data set to form a tag list. Exemplarily, the number of tags in the tag data set is N.

[0094] S2014: construct a co-occurrence matrix according to the tag list. Exemplarily, an N*N co-occurrence matrix is ​​initialized, all tag values ​​are set to 0, and the co-occurrence matrix is ​​used to store the co-occurrence times of each pair of tags.

[0095] S2015, traverse multiple historical images, record the occurrence of label pairs in the co-occurrence matrix, and obtain the relationship between labels. A label pair indicates that label A and label B appear in a sample, and label A and label B constitute a label pair (A, B). Exemplarily, if the label set of the sample (historical image) is [A, B][A, B][A, B], then increase the count of the corresponding position in the matrix: for the number of co-occurrences of label A and label B, increase the values ​​of (A, B)(A, B)(A, B) and (B, A)(B, A)(B, A) in the co-occurrence matrix (because co-occurrence is symmetrical). By analyzing the values ​​in the co-occurrence matrix, it is possible to determine which labels often appear together, and then infer the relationship between the labels. For example, if the number of co-occurrences of label A and label B is high, it can be considered that there is a strong correlation between them.

[0096] S2016, obtaining a second tag credibility value based on the relationship between the tags. Exemplarily, based on the co-occurrence frequency between the tags, a threshold can be pre-set to determine which tag combinations can improve credibility. For example, if the number of co-occurrences of tag A and tag B exceeds a pre-set threshold, it can be considered that the credibility is high. In addition, the co-occurrence matrix is ​​normalized so that the co-occurrence value of each pair of tags is within a certain range (such as 0 to 1).

[0097] S2016, assigning a first weight to the first tag credibility value, assigning a second weight to the second tag credibility value, performing weighted calculation, and obtaining a weighted average tag credibility value.

[0098] Through the above steps, a tag co-occurrence matrix can be constructed to calculate the number of co-occurrences of each pair of tags in the historical data. This analysis can reveal the relationship between tags, thereby providing useful information in tag credibility assessment. In addition, based on the steps of S2015, combined with S2016, the tag credibility value is made more accurate.

[0099] More specifically, the score output by the model to represent the label credibility value includes using, for example, a deep learning classifier, and the prediction result on the validation set or the test set as a credibility score. The deep learning classifier usually outputs the predicted probability of each label as the credibility value of the label directly. For example, if the model predicts that the probability of label A is 0.85, its credibility value can be set to 0.85. The deep learning classifier is a trained classifier. This application does not limit how to train the classifier.

[0100] S202, using the multi-label classification model of the Transformer architecture and combining it with the image feature vector to obtain a predicted probability label.

[0101] The image feature vector represents the image feature converted into a vector, and the predicted probability label represents the credibility probability of each label.

[0102] like Figure 3 As shown, specifically, a multi-label classification model based on the Transformer architecture is used for prediction, and the image feature vector is combined with the input of the Transformer. The image feature vector represents the image features extracted by the aforementioned convolutional neural network (such as ResNet50) feature extraction model, and is a high-dimensional vector that can represent the key information of the image. The Transformer architecture takes the image feature vector as input, processes it through a multi-layer self-attention mechanism, and generates a predicted probability for each label. These probability values ​​represent the credibility of the Transformer architecture for each label, and are usually output through a sigmoid activation function, ranging from 0 to 1. Exemplarily, in the image captioning task, the image feature vector can be extracted by CNN (convolutional neural network), and then input into the Transformer to generate the corresponding text description, such as a label, to form automatic labeling.

[0103] The Transformer architecture takes the image feature vector as input and generates the predicted probability of each label. The predicted probability of each label represents the confidence of the Transformer architecture for each label, usually output by the sigmoid activation function, ranging from 0 to 1.

[0104] S203, comparing the predicted probability label with the label credibility value, if they are greater than or equal to each other, obtaining a label.

[0105] Specifically, the predicted probability of each label is compared with the pre-defined label credibility value to determine whether the label predicted by the model is reliable enough so that it can be considered a valid label. If the predicted probability of a label is greater than or equal to its corresponding credibility value, the label is considered a valid label. This process ensures that only labels with high enough credibility are included in the final result. By comparison, it is determined which labels are accepted as valid labels. The accepted label set will serve as the final output label of the model for the target image. These labels should reflect the main features and content of the image to ensure that the predictions of the Transformer architecture have good credibility.

[0106] S204, using a multi-label classification model of a Transformer architecture to classify the labels to obtain a plurality of labels corresponding to the target image.

[0107] Specifically, after valid labels are confirmed, these labels can be further classified. This step can be achieved by reusing the Transformer architecture model to more finely classify the determined labels or give them more information for easier management. In other words, the Transformer model may process the relationship between labels to more accurately provide a corresponding set of labels for each target image.

[0108] Through the above steps, combined with the multi-label classification model of the Transformer architecture, useful information can be effectively extracted from the image and reliable labels can be generated. This process not only improves the prediction ability of the multi-label classification model of the Transformer architecture, but also ensures the reliability of the output labels, which can provide strong support for image recognition, classification and other tasks in practical applications.

[0109] S300, adjusting the credibility threshold according to the modification of the plurality of labels to obtain a plurality of target labels.

[0110] Specifically, obtaining several target labels includes the following steps:

[0111] S301, obtaining a modification tag.

[0112] The modified label is at least one of the labels corresponding to the target image.

[0113] S302: adjusting the credibility value of the tag according to the modified tag:

[0114] If a number of labels corresponding to the target image modified by the user are greater than or equal to a preset value, the label credibility value is reduced to obtain more target labels;

[0115] If a number of labels corresponding to the target image modified by the user is less than a preset value, the label credibility value is increased to obtain fewer target labels.

[0116] It should be noted that modified labels refer to new labels generated after the user modifies or confirms the label of the target image. These labels may be manually added or deleted by the user during actual use, or they may be generated by other automated methods. For example, if the target image is automatically labeled as "cat", "pet", and "animal", the user may choose to delete the "animal" label or add the "house cat" label. These actions will generate modified labels.

[0117] The credibility of a label is a measure of the system's reliability or accuracy of the label. When users modify certain labels, this indicates that the system's automatic labeling may be biased, or that the user's needs have changed, so it is necessary to adjust the credibility values ​​of these labels. The purpose of adjusting the credibility values ​​is to make the system predict labels more accurately in similar scenarios in the future.

[0118] Exemplarily, if the user has made a large number of modifications to the labels of the target image (exceeding the preset modification threshold), this indicates that there may be problems with the prediction of these labels. The credibility of these labels needs to be re-evaluated to avoid the same errors in the future. Reduce credibility. By reducing the credibility value of the relevant labels, you can be more conservative in future label predictions, allowing more labels to appear to increase flexibility and adaptability. After reducing the credibility, more labels may be marked to ensure that all possible labels are covered. This strategy is suitable for scenarios where users expect to get more comprehensive label suggestions in the future. If the user has made a small number of modifications to the labels of the target image, it means that the labels are basically accurate. At this time, the credibility of the relevant labels can be further improved to enhance the certainty in future predictions. Improve credibility. By increasing the credibility value of the relevant labels, you can be more confident in future label predictions and only recommend those labels that are more likely to accurately reflect the content of the image. Get fewer target labels: After increasing the credibility, you may tend to choose fewer but more accurate labels in the future, thereby reducing unnecessary annotations. This is suitable for scenarios where users want to get more streamlined and accurate label recommendations in the future.

[0119] By obtaining user-modified tags and adjusting the tag credibility value based on the amount of modification, it can adapt to user needs more intelligently. When users modify tags a lot, the credibility will be reduced to obtain more tag coverage, while when users only make a few modifications, the credibility will be increased to obtain more accurate tag recommendations. This dynamic adjustment mechanism can effectively improve the quality of tag prediction and user experience.

[0120] As mentioned above, the multi-label classification model based on the Transformer architecture automatically generates labels based on the label credibility value. The label credibility threshold adopts a dynamic adjustment mechanism, with an initial value of 0.35, but it will be automatically adjusted based on user feedback and the performance of the multi-label classification model. The specific logic is as follows:

[0121] When users frequently modify automatically generated tags, the threshold is lowered and more candidate tags are generated.

[0122] When users rarely modify labels, the threshold is increased to generate more accurate labels.

[0123] S400: screening a plurality of target images according to the target tag to obtain a plurality of images to be processed.

[0124] Specifically, obtaining a number of images to be processed includes the following steps:

[0125] S401, obtaining a target image dataset.

[0126] The target image data set includes a plurality of target images.

[0127] S402, screening is performed in the target image data set according to the target label as a screening condition to obtain a plurality of images to be processed.

[0128] Specifically, a target image dataset refers to a collection containing a large number of target images. The images in the dataset have consistent formats and resolutions to ensure effective comparison and analysis during subsequent processing. Target tags refer to tags determined by users or systems for filtering target images. These tags usually represent specific subject objects in the target images. According to the target tags, filtering is performed in the target image dataset. The logic of filtering is to select target images containing the specified target tags. For example, if the target tag is "cat", then all images with the tag "cat" are selected in the target image dataset. In this way, the target tags can be used to filter out the target images that you want to find as images to be processed, so that the images to be processed can be processed in subsequent steps.

[0129] like Figure 4 As shown, it should be noted that after obtaining a number of steps of images to be processed, the following steps are also included:

[0130] S501, using the U-Net deep learning architecture, segmenting the plurality of images to be processed to obtain images of a plurality of main objects.

[0131] Specifically, the U-Net deep learning architecture is a trained image segmentation model. The U-Net deep learning architecture can be used to realize image segmentation, automatically identify and segment the main objects in the image, and realize functions including but not limited to portrait extraction, product image background removal, and extraction of specific objects in complex scenes, namely intelligent cutout.

[0132] Exemplarily, after the U-Net deep learning architecture performs segmentation processing on the image to be processed, it performs binarization processing, that is, each pixel is classified as a main object or background according to a preset threshold, and then the image of each main object is extracted from the segmentation result. Each main object is represented as a continuous area, which may correspond to the main content in the image, such as human organs, vehicles, buildings, etc. Further, the segmentation results are post-processed, such as removing noise areas, connecting broken parts, and filling small holes to ensure that the outline of the main object is clear and complete.

[0133] To remove the noise area, the area or number of pixels of each segmented area can be calculated. Usually, smaller areas are considered noise and can be removed by setting an area threshold. For example, the number of pixels in each connected area can be calculated. If it is less than the set threshold, the area is marked as background to remove the noise. In addition, morphological operations such as opening can be used. This is an operation of erosion followed by dilation, which can effectively remove small noise points. Specifically, the erosion operation shrinks the foreground area (i.e., the object) in the image and removes small noise points, while the subsequent dilation operation restores the size of the foreground area.

[0134] The broken connections can be in the binary image, and the foreground area can be analyzed for connected components. Adjacent and connected foreground pixels are grouped into the same group, so that it is possible to detect which parts need to be connected and merge these areas when necessary.

[0135] To fill small holes, you can use the dilation and closing operations in morphological operations to fill small holes inside the object. In particular, the closing operation can effectively fill small holes inside the object without changing the overall shape of the object by first dilating and then corroding.

[0136] In one embodiment, a method for preprocessing and labeling data based on deep learning further includes:

[0137] Front-end fast reasoning:

[0138] Through WebAssembly and ONNX, fast front-end reasoning is achieved, providing real-time image segmentation and cutout functions.

[0139] user interface:

[0140] Interaction design: Developed using the React framework, it provides an intuitive user interface and supports users to perform batch processing, such as reviewing, editing, and deleting tags.

[0141] Intelligent recommendation: Integrates natural language processing technology to provide intelligent recommendation functions to assist users in batch modification and optimization of tags.

[0142] Data processing and storage:

[0143] Data cleaning: After batch preprocessing of the original data of multiple images is completed, data cleaning is automatically performed to ensure data consistency and availability.

[0144] Format conversion: Supports conversion of multiple data formats and improves data sets to meet different model training requirements.

[0145] System integration and optimization:

[0146] Multi-language support: Python, Javascript and Java are used as the main development languages ​​to ensure the flexibility and scalability of the system.

[0147] Deep learning framework: Use PyTorch to train and deploy deep learning models, providing powerful computing power and model optimization.

[0148] In some embodiments, a method for preprocessing and labeling data based on deep learning further includes the following steps:

[0149] The regions of the target labels extracted from the target image are formed into corresponding relationships;

[0150] Use the target label to locate the area corresponding to the target label;

[0151] The U-Net deep learning architecture is used to process the area corresponding to the target label to obtain the image of the main object.

[0152] In a second aspect, the present application provides a data set preprocessing and label management system based on deep learning, which is applied to the aforementioned data set preprocessing and label management method based on deep learning, and the system includes:

[0153] A preprocessing unit, used to acquire a plurality of initial images, and preprocess the plurality of initial images to obtain the plurality of target images having subject objects, wherein the subject objects represent main targets or objects that need to be identified and focused on in the target images;

[0154] A label generation unit, used to classify the target image with the subject object by using the pre-trained model, and obtain a plurality of labels corresponding to the target image;

[0155] The label generation unit is further used to adjust the credibility threshold according to the modification of the plurality of labels to obtain a plurality of target labels;

[0156] The result unit is used to screen the target images according to the target label to obtain a number of images to be processed.

[0157] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the aforementioned deep learning-based data set preprocessing and label management method when executing the computer program.

[0158] In a fourth aspect, the present application provides a computer program, which, when executed by a processor, implements the steps of the aforementioned deep learning-based data set preprocessing and label management method.

[0159] In summary, the present application provides a data set preprocessing and label management method and system based on deep learning, which has the following beneficial effects:

[0160] Data processing efficiency is significantly improved: Compared with traditional methods, this application improves data processing efficiency by 300%. In the experiment, the batch processing time of 1,000 pictures of raw data was reduced from 10 hours to 3 hours. This improvement greatly shortened the data preparation cycle and accelerated the progress of model training.

[0161] The label accuracy rate is greatly improved: The automatic labeling of this application optimizes the accuracy of label generation through a dynamic adjustment mechanism, which increases the label accuracy rate from 85% to 95%. This improvement significantly enhances the effect of model training, especially in tasks such as style transfer and personalized content creation, and improves the recognition and classification capabilities of the model.

[0162] Enhanced real-time adaptability: The system of this application can respond to changes in creative needs in a timely manner, quickly update and expand data sets through intelligent algorithms, enhance the real-time adaptability of the model, and ensure that the model maintains high performance in a constantly changing application environment.

[0163] More efficient use of computing resources: With limited computing resources, this application improves the efficiency of resource utilization by optimizing the data preprocessing process. Experimental data shows that the computing resources required to preprocess the same amount of data are reduced by 20%, while maintaining processing speed and quality.

[0164] Improved user experience: By providing an intuitive tag management interface and intelligent recommendation functions, this application improves the user experience during tag editing and review. Users can manage tags more easily, reduce manual intervention, and improve overall workflow efficiency.

[0165] These beneficial effects not only demonstrate the innovation of this application at the technical level, but also reflect the significant value it brings in practical applications.

[0166] This application's outlook for the future:

[0167] With the continuous advancement of artificial intelligence technology, this application has broad expansion prospects and application potential. The following are some possible future development directions and application scenarios:

[0168] Video data preprocessing:

[0169] The image preprocessing technology of this system is extended to the video field to realize automatic cropping, feature extraction and target detection of video frames. This will provide support for applications such as video content analysis, video editing and video enhancement, and improve the processing efficiency and quality of video data.

[0170] 3D Image Processing:

[0171] Use deep learning technology to pre-process and analyze 3D images, including 3D model reconstruction, texture mapping, and feature extraction. This will provide powerful technical support for fields such as virtual reality (VR), augmented reality (AR), and 3D printing.

[0172] Federated Learning Integration:

[0173] Explore the integration of federated learning technology with this system to achieve multi-party collaborative data processing and model training while protecting data privacy. This will provide a safer and more efficient data processing solution for data-sensitive industries such as finance and healthcare.

[0174] Augmented Reality (AR) Applications:

[0175] Applying the image segmentation and object detection technology of this system to the AR field can achieve seamless integration of the real world and virtual information. This will provide innovative user experience for industries such as education, entertainment and retail.

[0176] Automated data annotation:

[0177] Further optimize the automatic labeling system to improve the accuracy and diversity of label generation. By integrating more advanced natural language processing technology, more complex semantic understanding and label generation can be achieved.

[0178] User customized service:

[0179] Provide customized data preprocessing and label management services according to users’ specific needs. For example, provide customized product image processing and label management solutions for e-commerce platforms to improve product display effects and user shopping experience.

[0180] Through these expansion directions, the system of this application can not only improve the efficiency and quality of existing applications, but also open up new application areas and promote the further development of artificial intelligence technology.

[0181] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0182] The various embodiments in the present disclosure are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0183] The protection scope of the present disclosure is not limited to the above-mentioned embodiments. Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the scope and spirit of the present disclosure. If these changes and modifications fall within the scope of the claims of the present disclosure and their equivalents, the intention of the present disclosure also includes these changes and modifications.

Claims

1. A data set preprocessing and label management method based on deep learning, characterized in that: Methods include: Acquire a plurality of initial images, and pre-process the plurality of initial images to obtain the plurality of target images having subject objects, wherein the subject objects represent main targets or objects that need to be identified and focused on in the target images; Using the pre-trained model, label classification is performed on the target image having the subject object to obtain a number of labels corresponding to the target image; According to the modification of the plurality of labels, the credibility threshold is adjusted to obtain a plurality of target labels; According to the target label, a plurality of the target images are screened to obtain a plurality of images to be processed.

2. The method for data set preprocessing and label management based on deep learning according to claim 1, characterized in that: The step of acquiring a plurality of initial images and preprocessing the initial images to obtain a plurality of target images having the subject object comprises: Adjusting the initial images to obtain a plurality of standard images, wherein the standard images represent images obtained after color correction is performed on the initial images; Processing the standard image to obtain an image of optimal size; The main object in the optimal size image is identified to obtain the target image having the main object.

3. The method for data set preprocessing and label management based on deep learning according to claim 2, characterized in that: The step of adjusting the initial images to obtain a plurality of standard images comprises: Using the ResNet50 deep learning model, extracting features from the initial image to obtain image features of the initial image; Analyze the initial image according to the image features of the initial image to obtain the main object of the initial image; The standard image is determined according to the subject object.

4. The method for data set preprocessing and label management based on deep learning according to claim 3, characterized in that: The step of using the pre-trained model to classify the target image with the subject object and obtaining a plurality of labels corresponding to the target image includes: Get the label credibility value; A multi-label classification model of the Transformer architecture is used to combine with an image feature vector to obtain a predicted probability label, wherein the image feature vector represents the image feature converted into a vector, and the predicted probability label represents the credibility probability of each label; Compare the predicted probability label with the label credibility value, and if they are greater than or equal to each other, obtain the label; The labels are classified using a multi-label classification model of a Transformer architecture to obtain a number of labels corresponding to the target image.

5. The method for data set preprocessing and label management based on deep learning according to claim 4, characterized in that: The step of adjusting the credibility threshold according to the modification of the plurality of labels to obtain a plurality of target labels comprises: Acquire a modification tag, wherein the modification tag is at least one of the tags corresponding to the target image; According to the modified label, the label credibility value is adjusted: If a number of labels corresponding to the target image modified by the user are greater than or equal to a preset value, the label credibility value is reduced to obtain more target labels; If a number of labels corresponding to the target image modified by the user is less than a preset value, the label credibility value is increased to obtain fewer target labels.

6. The method for data set preprocessing and label management based on deep learning according to claim 1, characterized in that: The step of screening a plurality of target images according to the target label to obtain a plurality of images to be processed comprises: Acquire a target image data set, wherein the target image data set includes a plurality of the target images; According to the target label, as a screening condition, screening is performed in the target image data set to obtain a number of images to be processed.

7. The method for data set preprocessing and label management based on deep learning according to claim 1, characterized in that: After the step of screening the target images according to the target label to obtain the images to be processed, the method further includes: The U-Net deep learning architecture is used to segment the images to be processed to obtain images of the main objects.

8. A data set preprocessing and label management system based on deep learning, characterized in that: The method for preprocessing and labeling data based on deep learning applied to any one of claims 1 to 7, wherein the system comprises: A preprocessing unit, used for acquiring a plurality of initial images, and preprocessing the plurality of initial images to obtain the plurality of target images having subject objects, wherein the subject objects represent main targets or objects that need to be identified and focused on in the target images; A label generation unit, used to classify the target image with the subject object by using the pre-trained model, and obtain a plurality of labels corresponding to the target image; The label generation unit is further used to adjust the credibility threshold according to the modification of the plurality of labels to obtain a plurality of target labels; The result unit is used to screen the target images according to the target label to obtain a number of images to be processed.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the data set preprocessing and label management method based on deep learning described in any one of claims 1 to 7 are implemented.

10. A computer program, characterized in that When the computer program is executed by a processor, the steps of the data set preprocessing and label management method based on deep learning described in any one of claims 1 to 7 are implemented.

Images