Image data set construction method, apparatus and device, and computer program product
By filtering and classifying image data from the original data and building high-quality image data sets, the problem of difficulty in constructing high-quality data sets in the prior art is solved, and efficient training and high-quality image generation of image generation models are realized.
Patent Information
- Application Number
- CN202411999548.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-05-02
AI Technical Summary
It is difficult to construct high-quality image data sets in the prior art, especially for the training of deep learning models, and a large number of high-quality data sets are required to achieve the ability to generate high-quality images.
By obtaining candidate image data from the original data, obtaining its basic image information and quality information, filtering the candidate image data based on this information, dividing it into multiple categories, and then filtering the data in each category based on the quality information, and finally building a high-quality image data set.
It realizes the construction of a category-balanced, high-quality image data set, meets the training needs of deep learning models, and improves the quality and richness of image generation.
Smart Images
Figure CN119919724A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technology, and in particular to a method for constructing an image data set, a device for constructing an image data set, an electronic device, and a computer program product. Background Art
[0002] With the rapid development of deep learning models, large models have gradually emerged due to their excellent performance. For example, large language models, image processing and generation models have been widely used in various fields.
[0003] Compared with traditional small models, large models have more parameters and higher computational complexity, and often require more data sets for training to reach convergence. At the same time, the generation effect of large models such as image generation based on diffusion models is strongly related to the training data set. In order to obtain the ability to generate high-quality images, a high-quality data set is usually required as a training set.
[0004] In view of this, there is an urgent need in this field for a method to construct a high-quality image dataset.
[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention
[0006] The purpose of the present disclosure is to provide a method for constructing an image dataset, an apparatus for constructing an image dataset, an electronic device and a computer program product, so as to construct a high-quality image dataset at least to a certain extent.
[0007] According to a first aspect of the present disclosure, a method for constructing an image dataset is provided, comprising:
[0008] Obtaining candidate image data from original data;
[0009] Acquiring basic image information and quality information of the candidate image data, and screening the candidate image data according to the basic image information and the quality information to obtain first image data;
[0010] dividing the first image data into a plurality of categories, and filtering the first image data in each category according to the quality information to obtain second image data;
[0011] A first image data set is constructed based on the second image data.
[0012] In an exemplary embodiment of the present disclosure, the original data includes original image data and original video data, and obtaining candidate image data from the original data includes:
[0013] Extracting original video frames from the original video data;
[0014] The candidate image data is obtained according to the original image data and the original video frame.
[0015] In an exemplary embodiment of the present disclosure, the acquiring basic image information and quality information of the candidate image data includes:
[0016] Obtaining basic image information of the candidate image data according to image size data and average pixel bit number of the candidate image data;
[0017] The candidate image data is input into a plurality of pre-trained quality assessment models to obtain a plurality of quality information of the candidate image data.
[0018] In an exemplary embodiment of the present disclosure, screening the candidate image data according to the basic image information and the quality information to obtain the first image data includes:
[0019] Determine first threshold conditions corresponding to each of the basic image information and the quality information, and screen the candidate image data according to the first threshold conditions to obtain first image data.
[0020] In an exemplary embodiment of the present disclosure, dividing the first image data into multiple categories includes:
[0021] Extracting semantic features of the first image data by using a pre-trained multimodal feature extraction model;
[0022] According to the semantic features of the first image data, the first image data is divided into multiple categories using an unsupervised clustering method.
[0023] In an exemplary embodiment of the present disclosure, the filtering of the first image data in each category according to the quality information to obtain the second image data includes:
[0024] Obtaining weighted quality information of the first image data according to the plurality of quality information of the first image data;
[0025] sorting the first image data in each category respectively according to the weighted quality information to obtain a sorting result of the first image data in each category;
[0026] According to the sorting result in each category and the preset number of category images, the first image data in each category is screened respectively to obtain second image data.
[0027] In an exemplary embodiment of the present disclosure, the method further includes:
[0028] Determine second threshold conditions corresponding to each of the basic image information and the quality information, and filter the second image data according to the second threshold conditions to obtain third image data;
[0029] Sampling the third image data to obtain fourth image data, and marking the fourth image data as a positive sample or a negative sample;
[0030] According to the fourth image data marked as the positive sample and the fourth image data marked as the negative sample, adjusting the quality assessment model to obtain a quality decision model;
[0031] The second image data is rescreened according to the quality decision model to obtain fifth image data, and a second image data set is constructed based on the fifth image data.
[0032] According to a second aspect of the present disclosure, a device for constructing an image data set is provided, comprising:
[0033] A candidate data acquisition module is configured to acquire candidate image data from the original data;
[0034] A first data screening module is configured to obtain basic image information and quality information of the candidate image data, and screen the candidate image data according to the basic image information and the quality information to obtain first image data;
[0035] a second data screening module configured to divide the first image data into a plurality of categories, and screen the first image data in each category according to the quality information to obtain second image data;
[0036] The first data set construction module is configured to construct a first image data set based on the second image data.
[0037] In an exemplary embodiment of the present disclosure, the candidate data acquisition module includes:
[0038] A video frame extraction unit, configured to extract original video frames from the original video data;
[0039] The basic data acquisition unit is configured to obtain the candidate image data according to the original image data and the original video frame.
[0040] In an exemplary embodiment of the present disclosure, the first data screening module includes:
[0041] A basic image information determining unit is configured to obtain basic image information of the candidate image data according to image size data and average pixel bit number of the candidate image data;
[0042] The quality information determining unit is configured to input the candidate image data into a plurality of pre-trained quality assessment models to obtain a plurality of quality information of the candidate image data.
[0043] In an exemplary embodiment of the present disclosure, the first data screening module further includes:
[0044] The first threshold condition screening unit is configured to determine the first threshold conditions corresponding to each of the basic image information and the quality information, and screen the candidate image data according to the first threshold conditions to obtain the first image data.
[0045] In an exemplary embodiment of the present disclosure, the second data screening module includes:
[0046] A semantic feature extraction unit, configured to extract semantic features of the first image data by using a pre-trained multimodal feature extraction model;
[0047] The image category classification unit is configured to perform an unsupervised clustering method to classify the first image data into multiple categories according to the semantic features of the first image data.
[0048] In an exemplary embodiment of the present disclosure, the second data screening module further includes:
[0049] a weighted score determining unit, configured to obtain weighted quality information of the first image data according to the plurality of quality information of the first image data;
[0050] an image data sorting unit, configured to sort the first image data in each category according to the weighted quality information, to obtain a sorting result of the first image data in each category;
[0051] The category image screening unit is configured to screen the first image data in each category according to the sorting result in each category and the preset number of category images to obtain second image data.
[0052] In an exemplary embodiment of the present disclosure, the apparatus for constructing the image data set further includes a second data set construction module, and the second data set construction module includes:
[0053] a third image data screening unit, configured to determine the second threshold conditions corresponding to each of the basic image information and the quality information, and screen the second image data according to the second threshold conditions to obtain third image data;
[0054] a positive and negative sample labeling unit, configured to perform sampling on the third image data to obtain fourth image data, and label the fourth image data as a positive sample or a negative sample;
[0055] a quality decision model determining unit, configured to adjust the quality assessment model according to the fourth image data marked as the positive sample and the fourth image data marked as the negative sample, so as to obtain a quality decision model;
[0056] The second image set construction unit is configured to re-screen the second image data according to the quality decision model to obtain fifth image data, and construct a second image data set based on the fifth image data.
[0057] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement any one of the above-described methods for constructing an image data set.
[0058] According to a fourth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method for constructing an image data set described in any one of the above is implemented.
[0059] The exemplary embodiments of the present disclosure may have the following beneficial effects:
[0060] In the method for constructing an image dataset in the example implementation of the present disclosure, the basic image information and quality information of the candidate image data are obtained, and the candidate image data are screened according to the basic image information and quality information to obtain the first image data, and then the first image data is divided into multiple categories, and the first image data in each category is screened according to the quality information to obtain the second image data, and finally the first image dataset is constructed based on the second image data. The method for constructing an image dataset in the example implementation of the present disclosure provides a complete set of dataset construction links, evaluates the image quality based on multiple basic image information and quality information, and then performs clustering processing on the evaluated image data. The image dataset finally screened not only meets the requirements of high image quality, but also contains balanced and rich content and scenes, thereby constructing a high-quality image dataset with balanced categories.
[0061] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.
[0063] Figure 1 A schematic diagram showing a flow chart of a method for constructing an image data set according to an exemplary embodiment of the present disclosure;
[0064] Figure 2 A schematic diagram showing a process of screening and obtaining second image data in an exemplary embodiment of the present disclosure is shown;
[0065] Figure 3 A schematic diagram of a process for constructing a second image data set according to an exemplary embodiment of the present disclosure is shown;
[0066] Figure 4 A schematic diagram showing a process of collecting candidate image data in a specific embodiment of the present disclosure is shown;
[0067] Figure 5 A schematic diagram showing a process flow of an image quality screening stage in a specific embodiment of the present disclosure is shown;
[0068] Figure 6 A schematic diagram showing a flow chart of a category balancing stage in a specific embodiment of the present disclosure is shown;
[0069] Figure 7 A block diagram showing an apparatus for constructing an image data set according to an exemplary embodiment of the present disclosure;
[0070] Figure 8 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0071] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.
[0072] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein.
[0073] The following example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.
[0074] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.
[0075] This example implementation first provides a method for constructing an image dataset. Figure 1 As shown, the method for constructing the above image dataset may include the following steps:
[0076] Step S110: Obtain candidate image data from the original data.
[0077] Step S120: Obtain basic image information and quality information of candidate image data, and screen the candidate image data according to the basic image information and the quality information to obtain first image data.
[0078] Step S130: Divide the first image data into multiple categories, and filter the first image data in each category according to the quality information to obtain the second image data.
[0079] Step S140: Construct a first image data set based on the second image data.
[0080] In the method for constructing an image dataset in the example implementation of the present disclosure, the basic image information and quality information of the candidate image data are obtained, and the candidate image data are screened according to the basic image information and quality information to obtain the first image data, and then the first image data is divided into multiple categories, and the first image data in each category is screened according to the quality information to obtain the second image data, and finally the first image dataset is constructed based on the second image data. The method for constructing an image dataset in the example implementation of the present disclosure provides a complete set of dataset construction links, evaluates the image quality based on multiple basic image information and quality information, and then performs clustering processing on the evaluated image data. The image dataset finally screened not only meets the requirements of high image quality, but also contains balanced and rich content and scenes, thereby constructing a high-quality image dataset with balanced categories.
[0081] Next, combine Figures 2 to 6 The above steps of this exemplary embodiment are described in more detail.
[0082] In step S110 , candidate image data is acquired from original data.
[0083] In this example implementation, the original data is a large amount of data obtained from the Internet or existing data sets, and may include original image data and original video data. In the process of collecting the original data, in order to ensure the richness of the scene content, free data without copyright can be downloaded from a large number of image and video websites.
[0084] In this example implementation, the original video frame can be extracted from the original video data, and then the candidate image data can be obtained based on the original image data and the original video frame. The original video data is randomly extracted, and the original video frame obtained by the extraction is used together with the original image data as the candidate image data to be screened.
[0085] In step S120, basic image information and quality information of candidate image data are obtained, and the candidate image data are screened according to the basic image information and the quality information to obtain first image data.
[0086] In this example implementation, the basic image information of an image refers to an index related to basic information of the image, and may include basic information related to resolution and file size. The quality information of an image refers to a quality score index related to image quality.
[0087] The candidate image data can be screened by the basic image information and quality information of the candidate image data. In the quality screening stage, a fully automated model evaluation method can be used to reduce the cost of manual intervention.
[0088] In this example implementation, basic image information of the candidate image data can be obtained based on the image size data and the average pixel bit number of the candidate image data. The basic image information may include the length, width and average pixel bit number of the image, wherein the average pixel bit number is the file bit number divided by the total number of pixels.
[0089] In this example implementation, the candidate image data may be input into a plurality of pre-trained quality assessment models to obtain a plurality of quality information of the candidate image data.
[0090] Through the pre-trained image quality assessment model, multiple quality information of candidate image data can be obtained. The quality information may include NIQE (Natural Image Quality Evaluator), MUSIQ (Multi-scale Image Quality), KVQ (Kwai Visual Quality), MANIQA (Multi-dimension Attention Network for No-Reference Image Quality), CLIPIQA (CLIP Image Quality Assessment), aesthetic prediction scores, etc.
[0091] By counting and calculating the above-mentioned basic image information and quality information and saving them in the attribute file, they can be used for subsequent directional screening. In addition, other types of indicators can also be selected for image quality screening, which is not specifically limited in this example implementation.
[0092] In this example implementation, after acquiring basic image information and quality information of candidate image data, first threshold conditions corresponding to respective basic image information and quality information may be determined, and candidate image data may be screened according to the first threshold conditions to obtain first image data.
[0093] Considering that each image quality assessment model has different applicable scenarios and different training data, each model may have its own preferences and is not completely universal. In addition, the average number of pixel bits also indicates the degree of image compression to a certain extent. The heavier the image encoding compression, the fewer bits the file occupies and the worse the image quality. Therefore, in the image quality screening stage, the image quality can be evaluated by referring to the average number of pixel bits and multiple quality information at the same time. Among them, in order to speed up the operation, the model can calculate the six quality information in the mixed precision mode commonly used in the industry. Then, based on prior knowledge, the corresponding first threshold conditions are formulated for each basic image information and quality information. When all the first threshold conditions are met, the candidate image data is judged as a high-quality image. Otherwise, it is judged as a low-quality image.
[0094] In step S130, the first image data is divided into a plurality of categories, and the first image data in each category is screened according to the quality information to obtain the second image data.
[0095] Image data collected from the Internet may carry content-related information such as labels, but the labels themselves are not necessarily accurate, and most images may not have labels. Therefore, it is necessary to classify the collected image data by content to balance the image data categories.
[0096] In this example implementation, the semantic features of the first image data may be extracted using a pre-trained multimodal feature extraction model, and then the first image data may be divided into multiple categories using an unsupervised clustering method based on the semantic features of the first image data.
[0097] In the category balancing stage, a pre-trained multimodal feature extraction model, such as the multimodal model CLIP (Contrastive Language–Image Pre-training) based on contrastive learning, can be used to extract high-quality semantic features of the first image data, and then a preset unsupervised clustering method, such as the kmeans (k-means) unsupervised clustering method, is used to adaptively divide the first image data into multiple categories, so as to quickly complete the content classification of the first image data, and perform further screening based on the classification.
[0098] In this example implementation, Figure 2 As shown, the first image data in each category is screened according to the quality information to obtain the second image data, which may specifically include the following steps:
[0099] Step S210: Obtain weighted quality information of the first image data according to multiple quality information of the first image data.
[0100] In order to keep the number of each category balanced, the weighted quality information of multiple quality information can be calculated, and this score can measure the overall quality of the image. Taking the above six types of quality information as an example, the calculation formula of weighted quality information is avg_score = a*(1 / niqe_score)+b*maniqa_score+c*musiq_score+d*clipiqa_score+e*kvq_score+f*aesthetic_score, where a, b, c, d, e, f are the weighting coefficients of each quality information, and avg_score is the final weighted quality information.
[0101] Step S220: Sort the first image data in each category according to the weighted quality information to obtain a sorting result of the first image data in each category.
[0102] The calculated weighted quality information is sorted from high to low in each category to obtain a sorting result of the first image data in each category.
[0103] Step S230: According to the sorting result in each category and the preset number of category images, the first image data in each category is screened to obtain the second image data.
[0104] For the first image data in each category, the top N data with the highest weighted quality information may be extracted, thereby obtaining high-quality second image data with balanced categories.
[0105] In step S140 , a first image data set is constructed based on the second image data.
[0106] Finally, the first image data set is constructed based on the high-quality second image data with balanced categories, thereby constructing a high-quality image data set with balanced categories.
[0107] In this example implementation, a second image data set with higher quality can be constructed based on the first image data set. Figure 3 As shown, the method for constructing the second image data set may specifically include the following steps:
[0108] Step S310: Determine the second threshold conditions corresponding to each basic image information and quality information, and filter the second image data according to the second threshold conditions to obtain third image data.
[0109] In this example implementation, the second threshold condition is a screening condition higher than the first threshold condition. By increasing the screening thresholds of basic image information and quality information, a high-resolution and high-quality image in the cognition of the evaluation model, namely, the third image data, can be obtained.
[0110] Step S320: Sampling the third image data to obtain fourth image data, and labeling the fourth image data as a positive sample or a negative sample.
[0111] The third image data is partially sampled to obtain the fourth image data, and then the fourth image data is annotated with high quality. The annotation method can use a pre-trained classification model or a manual annotation method to classify the data that the human eye considers to be of extremely high quality as positive samples, and vice versa as negative samples.
[0112] Step S330: According to the fourth image data marked as positive samples and the fourth image data marked as negative samples, the quality assessment model is adjusted to obtain a quality decision model.
[0113] The quality assessment model is fine-tuned according to the annotated fourth image data to obtain an ultra-high-quality quality decision model.
[0114] Step S340: Rescreen the second image data according to the quality decision model to obtain fifth image data, and construct a second image data set based on the fifth image data.
[0115] The second image data is screened again by the quality decision model to select fifth image data of super high quality, and a second image data set is constructed based on the fifth image data. The second image data set includes some images with the highest quality.
[0116] In the ultra-high-quality screening stage, data sets with extremely high quality and meeting the subjective standards of the human eye can be extracted to obtain higher quality image data sets.
[0117] Since the index data and semantic feature data of the image are retained during the screening process, efficient free screening and query can be performed later. For example, according to personal requirements, the data of quality information in a specific segment or the data of a specific semantic content can be extracted. At the same time, the attribute files and feature files of the image dataset can be flexibly expanded, which facilitates the subsequent expansion of other screening conditions.
[0118] like Figure 4 FIG. 1 is a flow chart of candidate image data collection in a specific implementation of the present disclosure, which is an example of step S110 in this example implementation. The specific steps of the flow chart are as follows:
[0119] Step S410: Download original data.
[0120] Download raw data from the Internet or existing datasets, including raw image data and raw video data.
[0121] Step S420: Extract frames from the original video data.
[0122] The original video data is randomly sampled, and the original video frames obtained by sampling are used together with the original image data as candidate image data to be screened.
[0123] like Figure 5 FIG. 1 is a flowchart of an image quality screening stage in a specific implementation manner of the present disclosure, which is an example of step S120 in this example implementation manner. The specific steps of the flowchart are as follows:
[0124] Step S510: Quality model evaluation.
[0125] The candidate image data is input into the quality assessment model to obtain the corresponding quality information.
[0126] Step S520: Determine the threshold screening condition.
[0127] According to prior knowledge, corresponding first threshold conditions are respectively formulated for each basic image information and quality information. When all the first threshold conditions are met, the candidate image data is judged as a high-quality image, otherwise, it is judged as a low-quality image.
[0128] like Figure 6 FIG. 1 is a flow chart of the category balancing stage in a specific implementation manner of the present disclosure, which is an example of step S130 in this example implementation manner. The specific steps of the flow chart are as follows:
[0129] Step S610: CLIP model extracts features.
[0130] The semantic features of the first image data are extracted through the multimodal model CLIP.
[0131] Step S620: Kmeans clustering.
[0132] The first image data is adaptively divided into multiple categories by using the kmeans unsupervised clustering method.
[0133] Step S630: Sorting within the category.
[0134] The first image data in each category are sorted respectively according to the weighted quality information, and the first N data with the highest weighted quality information are extracted, so as to obtain a high-quality image data set with balanced categories.
[0135] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.
[0136] It should be noted that the image data involved in this disclosure are all information authorized by the user or fully authorized by all parties.
[0137] Furthermore, the present disclosure also provides a device for constructing an image dataset. Figure 7 As shown, the image data set construction device may include a candidate data acquisition module 710, a first data screening module 720, a second data screening module 730 and a first data set construction module 740. Among them:
[0138] The candidate data acquisition module 710 is configured to acquire candidate image data from the original data;
[0139] The first data screening module 720 is configured to obtain basic image information and quality information of candidate image data, and screen the candidate image data according to the basic image information and the quality information to obtain first image data;
[0140] The second data screening module 730 is configured to divide the first image data into a plurality of categories, and screen the first image data in each category according to the quality information to obtain the second image data;
[0141] The first data set construction module 740 is configured to construct a first image data set based on the second image data.
[0142] In some exemplary embodiments of the present disclosure, the candidate data acquisition module 710 may include a video frame extraction unit and a basic data acquisition unit.
[0143] A video frame extraction unit is configured to extract original video frames from original video data;
[0144] The basic data acquisition unit is configured to obtain candidate image data according to the original image data and the original video frame.
[0145] In some exemplary embodiments of the present disclosure, the first data screening module 720 may include a basic image information determination unit and a quality information determination unit.
[0146] A basic image information determination unit is configured to obtain basic image information of the candidate image data according to the image size data and the average number of pixel bits of the candidate image data;
[0147] The quality information determining unit is configured to input the candidate image data into a plurality of pre-trained quality assessment models to obtain a plurality of quality information of the candidate image data.
[0148] In some exemplary embodiments of the present disclosure, the first data screening module 720 may further include a first threshold condition screening unit, which is configured to execute the first threshold conditions corresponding to each basic image information and quality information, and screen the candidate image data according to the first threshold conditions to obtain the first image data.
[0149] In some exemplary embodiments of the present disclosure, the second data screening module 730 may include a semantic feature extraction unit and an image category classification unit.
[0150] A semantic feature extraction unit, configured to extract semantic features of the first image data by using a pre-trained multimodal feature extraction model;
[0151] The image category classification unit is configured to perform an unsupervised clustering method to classify the first image data into multiple categories according to the semantic features of the first image data.
[0152] In some exemplary embodiments of the present disclosure, the second data screening module 730 may further include a weighted score determination unit, an image data sorting unit, and a category image screening unit.
[0153] A weighted score determination unit, configured to obtain weighted quality information of the first image data according to a plurality of quality information of the first image data;
[0154] an image data sorting unit, configured to sort the first image data in each category according to the weighted quality information, to obtain a sorting result of the first image data in each category;
[0155] The category image screening unit is configured to screen the first image data in each category according to the sorting result in each category and the preset number of category images to obtain the second image data.
[0156] In some exemplary embodiments of the present disclosure, the apparatus for constructing an image data set provided by the present disclosure may further include a second data set construction module, and the second data set construction module may include a third image data screening unit, a positive and negative sample labeling unit, a quality decision model determination unit, and a second image set construction unit. Wherein:
[0157] A third image data screening unit is configured to determine the second threshold conditions corresponding to each of the basic image information and the quality information, and screen the second image data according to the second threshold conditions to obtain third image data;
[0158] a positive and negative sample labeling unit, configured to perform sampling on the third image data to obtain fourth image data, and label the fourth image data as a positive sample or a negative sample;
[0159] A quality decision model determination unit is configured to adjust the quality assessment model according to the fourth image data marked as a positive sample and the fourth image data marked as a negative sample to obtain a quality decision model;
[0160] The second image set construction unit is configured to re-screen the second image data according to the quality decision model to obtain fifth image data, and construct a second image data set based on the fifth image data.
[0161] The specific details of each module / unit in the above-mentioned image data set construction device have been described in detail in the corresponding method embodiment part, and will not be repeated here.
[0162] Figure 8 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure is shown.
[0163] It should be noted that Figure 8 The computer system 800 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0164] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 802 or the program loaded from the storage part 808 to the random access memory (RAM) 803. In the RAM 803, various programs and data required for system operation are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.
[0165] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.
[0166] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 809, and / or installed from a removable medium 811. When the computer program is executed by a central processing unit (CPU) 801, various functions defined in the system of the present disclosure are executed.
[0167] The exemplary embodiments of the present disclosure further provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method for constructing the image data set is implemented.
[0168] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.
[0169] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.
[0170] The code of the computer program can be written in one or more programming languages. Programming languages such as C language, Java, C++, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).
[0171] The computer program may be carried or transmitted via electrical, magnetic, optical, electromagnetic, infrared, or other signals. The electronic device may convert the signal carrying the computer program into a digital signal, thereby running the computer program. When the computer program is run on an electronic device, its code is used to enable the electronic device to execute (more specifically, the processor of the electronic device may execute) the method steps of various exemplary embodiments of the present disclosure, such as the method for constructing the above-mentioned image data set.
[0172] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0173] It should be noted that, although several modules of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules described above can be concretized in one module. Conversely, the features and functions of one module described above can be further divided into multiple modules to be concretized.
[0174] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.
[0175] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A method for constructing an image dataset, characterized in that: include: Obtaining candidate image data from original data; Acquiring basic image information and quality information of the candidate image data, and screening the candidate image data according to the basic image information and the quality information to obtain first image data; dividing the first image data into a plurality of categories, and filtering the first image data in each category according to the quality information to obtain second image data; A first image data set is constructed based on the second image data.
2. The method for constructing an image dataset according to claim 1, characterized in that: The original data includes original image data and original video data, and obtaining candidate image data from the original data includes: Extracting original video frames from the original video data; The candidate image data is obtained according to the original image data and the original video frame.
3. The method for constructing an image dataset according to claim 1, characterized in that: The obtaining of basic image information and quality information of the candidate image data includes: Obtaining basic image information of the candidate image data according to image size data and average pixel bit number of the candidate image data; The candidate image data is input into a plurality of pre-trained quality assessment models to obtain a plurality of quality information of the candidate image data.
4. The method for constructing an image dataset according to claim 3, characterized in that: The step of screening the candidate image data according to the basic image information and the quality information to obtain the first image data includes: Determine first threshold conditions corresponding to each of the basic image information and the quality information, and screen the candidate image data according to the first threshold conditions to obtain first image data.
5. The method for constructing an image dataset according to claim 1, characterized in that: The dividing the first image data into a plurality of categories comprises: Extracting semantic features of the first image data by using a pre-trained multimodal feature extraction model; According to the semantic features of the first image data, the first image data is divided into multiple categories using an unsupervised clustering method.
6. The method for constructing an image dataset according to claim 1, characterized in that: The step of filtering the first image data in each category according to the quality information to obtain second image data includes: Obtaining weighted quality information of the first image data according to the plurality of quality information of the first image data; sorting the first image data in each category respectively according to the weighted quality information to obtain a sorting result of the first image data in each category; According to the sorting result in each category and the preset number of category images, the first image data in each category is screened respectively to obtain second image data.
7. The method for constructing an image dataset according to claim 4, characterized in that: The method further comprises: Determine second threshold conditions corresponding to each of the basic image information and the quality information, and filter the second image data according to the second threshold conditions to obtain third image data; Sampling the third image data to obtain fourth image data, and marking the fourth image data as a positive sample or a negative sample; According to the fourth image data marked as the positive sample and the fourth image data marked as the negative sample, adjusting the quality assessment model to obtain a quality decision model; The second image data is rescreened according to the quality decision model to obtain fifth image data, and a second image data set is constructed based on the fifth image data.
8. A device for constructing an image data set, characterized in that: include: A candidate data acquisition module is configured to acquire candidate image data from the original data; A first data screening module is configured to obtain basic image information and quality information of the candidate image data, and screen the candidate image data according to the basic image information and the quality information to obtain first image data; a second data screening module configured to divide the first image data into a plurality of categories, and screen the first image data in each category according to the quality information to obtain second image data; The first data set construction module is configured to construct a first image data set based on the second image data.
9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method for constructing an image dataset as described in any one of claims 1 to 7.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for constructing an image dataset according to any one of claims 1 to 7 is implemented.