Image data set construction method and system

Abstract

The invention relates to the technical field of image processing, and discloses an image data set construction method and system, and the method comprises the steps: employing a multi-mode large model CLIP to recognize the defects of an existing image data set, and carrying out the supplementation; annotating the image data set obtained after supplementation, and ensuring the annotation accuracy through multi-level quality control; expanding the marked image data set; and fusing the expanded image data set with other modal data to complete the construction of the final image data set. According to the method, the accuracy of data acquisition and the labeling efficiency are improved, the accuracy and the consistency of data labeling are improved, and human errors are reduced; according to the method, the data set can be dynamically expanded, so that the data set is more diversified, and the generalization ability of the model is improved; the method supports the fusion of the image and other modalities, and enhances the application scene and flexibility of an image data set by using the cross-modal alignment capability of a large model.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and system for constructing an image data set. Background Art

[0002] In existing technologies, the construction of large-scale image datasets is usually achieved through the following methods:

[0003] Manual data collection and annotation:

[0004] Traditionally, image data collection relies primarily on manual or semi-automated tools, and data labeling is typically performed by human annotators or assisted by simpler models. While this approach can achieve a certain degree of accuracy, it requires a high level of human resources and is time-consuming. Furthermore, it lacks intelligence, relies on manual or simple algorithms, and lacks the intelligent decision-making capabilities driven by large models. This results in low efficiency, high costs, and difficulty adapting quickly to the dynamic demands of datasets. Furthermore, due to inadequate quality control and a lack of intelligent quality control methods, existing technologies can lead to errors and inconsistencies in data labeling, which are difficult to automatically identify and correct.

[0005] Static data enhancement:

[0006] Data augmentation in existing technologies is mainly performed through fixed rules (such as rotation, flipping, scaling, etc.). These enhancement methods lack specificity and intelligence, easily generate redundant or irrelevant data, and cannot effectively improve the diversity of the dataset and the generalization ability of the model.

[0007] Single-modal data management:

[0008] Existing dataset management is mostly based on single-modal data, failing to fully utilize the complementarity of images and other modal data, limiting the application ability of datasets in multimodal tasks. Summary of the Invention

[0009] The present invention aims to at least solve the technical problems existing in the prior art, such as imperfect quality control, low processing efficiency, enhancement methods that easily generate useless data, and data management and application that mainly focus on a single modality, which limits the application capability of datasets in multimodal tasks.

[0010] To this end, an object of the present invention is to provide a method for constructing an image dataset, comprising:

[0011] Use the multimodal large model CLIP to identify deficiencies in existing image datasets and supplement them;

[0012] The supplemented image dataset is annotated using the multimodal large model CLIP, and the accuracy of the annotation is ensured through multi-level quality control;

[0013] Expand the labeled image dataset;

[0014] The expanded image dataset is fused with other modality data to complete the construction of the final image dataset.

[0015] Furthermore, we use the multimodal large model CLIP to identify and supplement the deficiencies of existing image datasets, including:

[0016] Use the multimodal large model CLIP to analyze existing image datasets and identify categories and scenes that are scarce or underrepresented in existing image datasets;

[0017] According to the categories and the scenarios, a collection strategy is designed to collect data under the categories and the scenarios, thereby supplementing the existing image dataset.

[0018] Furthermore, the labeled image dataset is expanded to include:

[0019] Perform data enhancement for the categories and / or scenarios;

[0020] Generate new images using generative adversarial networks;

[0021] The enhanced or generated images are screened using a large model to ensure that the features of the enhanced or generated images are consistent with the real data, and the screened images are added to the annotated image dataset.

[0022] Furthermore, the enhanced or generated images are screened using the large model to ensure that the features of the enhanced or generated images are consistent with the real data, and the screened images are added to the annotated image dataset, which also includes:

[0023] Use the multimodal large model CLIP to perform real-time analysis on the annotated image dataset, generate coverage reports and dataset performance evaluations, and the multimodal large model CLIP will mark poorly performing categories and prompt the need for further data enhancement or collection.

[0024] Furthermore, the multimodal large model CLIP is used to annotate the supplemented image dataset, and the accuracy of the annotation is ensured through multi-level quality control, including:

[0025] Use the multimodal large model CLIP for reasoning to classify and annotate unlabeled images;

[0026] Use image embedding vectors to perform label clustering analysis to identify and correct labeling errors and ensure label consistency in feature space;

[0027] The annotation results are cross-validated through different artificial intelligence models to ensure the accuracy of the annotation results.

[0028] Furthermore, the confidence scoring system of the multimodal large model CLIP is used to select images with unqualified confidence and submit them to the manual review platform.

[0029] Furthermore, the other modal data includes text data, and the image dataset obtained after expansion is fused with the other modal data to complete the construction of the final image dataset, including:

[0030] Generate a text description corresponding to the image dataset using a multimodal large model CLIP, and fuse it with the text data; or

[0031] The multimodal large model CLIP is used to extract semantic information related to the image dataset from the text data and fuse it with the image dataset.

[0032] Furthermore, it also includes:

[0033] Version control is performed for each updated image dataset, including:

[0034] Use the multimodal large model CLIP to record the log of each image dataset update and generate a dataset snapshot.

[0035] Furthermore, it also includes:

[0036] Automatic anomaly detection and dynamic optimization of image datasets with each update, including:

[0037] Detect potential abnormal samples in each updated image dataset and generate a report for processing;

[0038] Based on the report, the image data acquisition and annotation strategy for each update is adjusted to continuously optimize the quality of the image dataset.

[0039] Furthermore, it also includes:

[0040] Combining differential privacy technology and blockchain technology, data is desensitized to ensure the privacy and security of data during processing, transmission and storage.

[0041] The present invention provides an image dataset construction system for implementing the above-mentioned image dataset construction method, comprising:

[0042] An intelligent data acquisition module to identify deficiencies in existing image datasets and supplement them;

[0043] Automatic annotation and quality control module, used to annotate the supplemented image dataset and ensure the accuracy of the annotation through multi-level quality control;

[0044] Adaptive data extension module, used to expand the labeled image dataset;

[0045] The cross-modal dataset construction module is used to fuse the expanded image dataset with other modal data to complete the construction of the final image dataset.

[0046] The present invention provides an image dataset construction method and system, which has the following beneficial effects:

[0047] This invention, based on intelligent data collection and automatic annotation using a large multimodal model called CLIP, significantly reduces manual intervention and improves data collection accuracy and annotation efficiency. Particularly in scenarios or categories where data is scarce, this invention can automatically identify and address weak links in image datasets, ensuring their balance and representativeness, and ultimately improving their quality.

[0048] Multi-level quality control, such as label cluster analysis and cross-validation after automatic annotation, is performed through the multimodal large model CLIP, which greatly improves the accuracy and consistency of data annotation and reduces human errors.

[0049] By leveraging generative adversarial networks (GANs) and data augmentation techniques, the present invention can dynamically expand datasets, particularly for scarce data. This dynamic expansion capability diversifies datasets and improves the generalization capabilities of models.

[0050] This paper supports the fusion of images with other modalities (such as text and audio), leveraging the cross-modal alignment capabilities of large models to enhance the application scenarios and flexibility of image datasets and fully exploit the complementarity between images and other modal data. This is of great significance in multimodal tasks such as generating text descriptions from images and audio recognition.

[0051] Through CLIP-driven automated management of multimodal large models, this invention enables version control, anomaly detection, and dynamic optimization of image datasets, ensuring efficient management of datasets over the long term. Furthermore, it leverages differential privacy and blockchain technology to enhance data privacy and security, meeting current high demands for data privacy protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 This is a flowchart of a method for constructing an image dataset in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] Various aspects and features of the present invention are described herein with reference to the accompanying drawings.

[0055] It should be understood that various modifications may be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of an embodiment. Other modifications within the scope and spirit of the invention will occur to those skilled in the art.

[0056] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the general description of the invention given above and the detailed description of the embodiments given below, serve to explain the principles of the invention.

[0057] These and other characteristics of the invention will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0058] It should also be understood that although the invention has been described with reference to certain specific examples, those skilled in the art will be able to realize many other equivalent forms of the invention that have the characteristics recited in the claims and are therefore within the scope of protection defined thereby.

[0059] The above and other aspects, features and advantages of the present invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.

[0060] Specific embodiments of the present invention will be described hereinafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of the present invention, which may be implemented in a variety of ways. Well-known and / or repetitive functions and structures are not described in detail to avoid obscuring the present invention with unnecessary or superfluous detail. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but rather to serve as a basis and representative basis for teaching those skilled in the art to variously employ the present invention with virtually any suitable detailed structure.

[0061] Example

[0062] like Figure 1 As shown, this embodiment provides a method for constructing an image dataset, including:

[0063] Step S1: Use the multimodal large model CLIP to identify defects in the existing image dataset and supplement them;

[0064] Step S2: annotating the image dataset obtained in step S1 using the multimodal large model CLIP, and ensuring the accuracy of the annotation through multi-level quality control;

[0065] Step S3: Expand the labeled image dataset;

[0066] Step S4: Fuse the expanded image dataset with other modality data to complete the construction of the final image dataset.

[0067] According to another specific embodiment of the present invention, based on step S1, the multimodal large model CLIP is used to identify defects in the existing image dataset and supplement them, including:

[0068] Use the multimodal large model CLIP to analyze existing image datasets and identify categories and scenes that are scarce or underrepresented in existing image datasets;

[0069] According to the categories and the scenarios, a collection strategy is designed to collect data under the categories and the scenarios, thereby supplementing the existing image dataset.

[0070] According to another specific embodiment of the present invention, based on the above embodiment, a collection strategy is designed according to the categories and the scenarios to collect data under the categories and the scenarios, thereby supplementing the existing image dataset, including:

[0071] Data collection in specific geographic areas, environments, and conditions;

[0072] Use drones, self-driving vehicles, social media, and other methods to obtain image data of the categories or scenes mentioned above to supplement the weak parts of the image dataset; data can also be obtained from public social media platforms or satellite images.

[0073] According to another specific embodiment of the present invention, based on step S3, the annotated image dataset is expanded, including:

[0074] Perform data enhancement for the categories and / or scenarios;

[0075] Generate new images using generative adversarial networks;

[0076] The enhanced or generated images are screened using a large model to ensure that the features of the enhanced or generated images are consistent with the real data, and the screened images are added to the annotated image dataset to avoid generating samples that are inconsistent with reality.

[0077] Specifically, the multimodal large model CLIP will eliminate samples that differ too much from the real image features by comparing the embedding vectors, preventing the multimodal large model CLIP from learning incorrect features.

[0078] According to another specific embodiment of the present invention, based on the above embodiment, the enhanced or generated images are screened using a large model to ensure that the features of the enhanced or generated images are consistent with the real data, and the screened images are added to the annotated image dataset, further comprising:

[0079] Use the multimodal large model CLIP to perform real-time analysis on the labeled image dataset, and generate coverage reports and dataset performance evaluations in real time; the multimodal large model CLIP will also mark poorly performing categories and prompt the need for further data enhancement or collection.

[0080] The image dataset is updated regularly based on the coverage report and the dataset performance evaluation.

[0081] Specifically, new samples are introduced by re-collecting or generating new images to continuously optimize the coverage and category balance of the dataset.

[0082] According to another specific embodiment of the present invention, based on the above embodiment, data enhancement is performed for the categories and / or scenarios, including:

[0083] For categories or scenarios where the model performs poorly, data augmentation techniques such as image rotation, scaling, and color adjustment are used to generate diverse samples.

[0084] According to another specific embodiment of the present invention, based on step S2, the image dataset obtained in step S1 is annotated using the multimodal large model CLIP, and the accuracy of the annotation is ensured through multi-level quality control, including:

[0085] Use the multimodal large model CLIP for zero-shot reasoning to classify and annotate unlabeled images;

[0086] Use image embeddings to perform label clustering analysis, identify and correct labeling errors, and ensure label consistency in feature space. For example, if a group of images are labeled with the same category but their features are far apart in the embedding space, the model will flag these images for review.

[0087] The annotation results are cross-validated through different artificial intelligence models to ensure the accuracy of the annotation results.

[0088] According to another specific embodiment of the present invention, based on the above embodiment, cross-validation of the annotation results is performed using different artificial intelligence models to ensure the accuracy of the annotation results, including:

[0089] Cross-validation is performed using different large AI models, such as CLIP and DINO, to ensure the accuracy of the labeling results.

[0090] According to another specific embodiment of the present invention, based on the above embodiment, the confidence scoring system of the multimodal large model CLIP is used to select images with unqualified confidence and submit them to the manual review platform.

[0091] Specifically, the confidence scoring system of the multimodal large model CLIP is used to automatically select low-confidence images for submission to the manual review platform. Guided by the multimodal large model CLIP, manual reviewers can focus on reviewing high-risk annotations, optimizing manual review efficiency. This invention utilizes a simultaneous multimodal large model CLIP and manual review approach to improve the accuracy of annotation results.

[0092] According to another specific embodiment of the present invention, based on step S4, the other modality data includes text data, and the image dataset obtained after expansion is fused with the other modality data to complete the construction of the final image dataset, including:

[0093] Generate a text description corresponding to the image dataset using a multimodal large model CLIP, and fuse it with the text data; or

[0094] The multimodal large model CLIP is used to extract semantic information related to the image dataset from the text data and fuse it with the image dataset.

[0095] Specific implementation:

[0096] By using the image and text embedding vectors generated by the large model, the text description of the image is automatically generated to achieve cross-modal fusion between image and text.

[0097] Perform consistency checks on cross-modal data to ensure the consistency of labels and embedding vectors between different modalities in the semantic space.

[0098] A consistency check is performed on cross-modal datasets to ensure the consistency of labels and embedding vectors in the semantic space between different modalities.

[0099] According to another specific embodiment of the present invention, based on the above embodiment, the present invention further includes:

[0100] Version control is performed for each updated image dataset, including:

[0101] Use the multimodal large model CLIP to record the log of each image dataset update and generate a dataset snapshot.

[0102] According to another specific embodiment of the present invention, based on the above embodiment, the present invention further includes:

[0103] Automatic anomaly detection and dynamic optimization of image datasets with each update, including:

[0104] Detect potential abnormal samples in each updated image dataset and generate a report for processing;

[0105] Based on the report, the image data acquisition and annotation strategy for each update is adjusted to continuously optimize the quality of the image dataset.

[0106] According to another specific embodiment of the present invention, based on the above embodiment, the present invention further includes:

[0107] Combining differential privacy technology and blockchain technology, data is desensitized to ensure the privacy and security of data during processing, transmission and storage.

[0108] According to another specific embodiment of the present invention, based on step S1, the multimodal large model CLIP is used to identify defects in the existing image dataset and supplement it, including:

[0109] The multimodal large model CLIP is used to perform preliminary analysis on the unlabeled images in the supplemented image dataset, and to screen out potentially useful samples from the unlabeled images; based on the characteristics and contextual information of the samples, preliminary labels are classified and generated for the next step of processing, thereby improving image processing efficiency.

[0110] According to another specific embodiment of the present invention, based on step S2, the image dataset obtained in step S1 is annotated using the multimodal large model CLIP, and the accuracy of the annotation is ensured through multi-level quality control, including:

[0111] The samples are labeled based on the preliminary labels using the zero-shot reasoning capabilities of the large multimodal CLIP model. This approach is particularly suitable for new categories or rare scenarios, as the model can infer reasonable labels based on its semantic understanding capabilities during training.

[0112] The present invention provides an image dataset construction system for implementing the above-mentioned image dataset construction method, comprising:

[0113] An intelligent data acquisition module to identify deficiencies in existing image datasets and supplement them;

[0114] Automatic annotation and quality control module, used to annotate the supplemented image dataset and ensure the accuracy of the annotation through multi-level quality control;

[0115] Adaptive data extension module, used to expand the labeled image dataset;

[0116] The cross-modal dataset construction module is used to fuse the expanded image dataset with other modal data to complete the construction of the final image dataset.

[0117] The present invention provides an image dataset construction method and system, which has the following beneficial effects:

[0118] This invention, based on intelligent data collection and automatic annotation using a large multimodal model called CLIP, significantly reduces manual intervention and improves data collection accuracy and annotation efficiency. Particularly in scenarios or categories where data is scarce, this invention can automatically identify and address weak links in image datasets, ensuring their balance and representativeness, and ultimately improving their quality.

[0119] Multi-level quality control, such as label cluster analysis and cross-validation after automatic annotation, is performed through the multimodal large model CLIP, which greatly improves the accuracy and consistency of data annotation and reduces human errors.

[0120] By leveraging generative adversarial networks (GANs) and data augmentation techniques, the present invention can dynamically expand datasets, particularly for scarce data. This dynamic expansion capability diversifies datasets and improves the generalization capabilities of models.

[0121] This paper supports the fusion of images with other modalities (such as text and audio), leveraging the cross-modal alignment capabilities of large models to enhance the application scenarios and flexibility of image datasets and fully exploit the complementarity between images and other modal data. This is of great significance in multimodal tasks such as generating text descriptions from images and audio recognition.

[0122] Through CLIP-driven automated management of multimodal large models, this invention enables version control, anomaly detection, and dynamic optimization of image datasets, ensuring efficient management of datasets over the long term. Furthermore, it leverages differential privacy and blockchain technology to enhance data privacy and security, meeting current high demands for data privacy protection.

[0123] An embodiment of the present disclosure further provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned method for constructing an image dataset when executing the computer program in the memory, including:

[0124] Step S1: Use the multimodal large model CLIP to identify defects in the existing image dataset and supplement them;

[0125] Step S2: annotating the image dataset obtained in step S1 using the multimodal large model CLIP, and ensuring the accuracy of the annotation through multi-level quality control;

[0126] Step S3: Expand the labeled image dataset;

[0127] Step S4: Fuse the expanded image dataset with other modality data to complete the construction of the final image dataset.

[0128] In some embodiments, the processor that executes the computer program may be a processing device including one or more general-purpose processing devices, such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), etc. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor that runs other instruction sets, or a processor that runs a combination of instruction sets. The processor may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc.

[0129] The memory may be read-only memory (ROM), random access memory (RAM), phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), electrically erasable programmable read-only memory (EEPROM), other types of random access memory (RAM), flash disks or other forms of flash memory, cache, registers, static memory, compact disk read-only memory (CD-ROM), digital versatile disks (DVD) or other optical storage, magnetic tape cassettes or other magnetic storage devices, or any other possible non-transitory medium used to store information or instructions that can be accessed by a computer device.

[0130] The electronic devices of the embodiments of the present disclosure may include but are not limited to fixed terminal devices such as MCU controllers, servers, desktop computers, digital TVs, etc., as well as mobile terminal devices such as vehicle-mounted devices (such as head-up display devices), handheld devices (such as mobile phones, tablets, etc.), wearable devices (such as smart watches, smart bracelets, etc.), etc.

[0131] The present disclosure also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method for constructing an image dataset is implemented, including:

[0132] Step S1: Use the multimodal large model CLIP to identify defects in the existing image dataset and supplement them;

[0133] Step S2: annotating the image dataset obtained in step S1 using the multimodal large model CLIP, and ensuring the accuracy of the annotation through multi-level quality control;

[0134] Step S3: Expand the labeled image dataset;

[0135] Step S4: Fuse the expanded image dataset with other modality data to complete the construction of the final image dataset.

[0136] The computer-readable storage medium of the embodiments of the present disclosure may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. In the embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus or device, for example, the memory described above.

[0137] The computer programs of the embodiments of the present disclosure may be organized into one or more computer-executable components or modules. Any number and combination of such components or modules may be used to implement various aspects of the present disclosure. For example, various aspects of the present disclosure are not limited to the specific computer-executable instructions or specific components or modules shown in the accompanying drawings and described herein. Other embodiments may include different computer-executable instructions or components with more or less functionality than shown and described herein.

[0138] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

Claims

1. A method for constructing an image dataset, characterized in that: include: Use the multimodal large model CLIP to identify deficiencies in existing image datasets and supplement them; The multimodal large model CLIP is used to annotate the supplemented image dataset; Expand the labeled image dataset; The expanded image dataset is fused with other modality data to complete the construction of the final image dataset.

2. The method for constructing an image dataset according to claim 1, characterized in that: The multimodal large model CLIP is used to identify deficiencies in existing image datasets and to supplement them, including: Use the multimodal large model CLIP to analyze existing image datasets and identify categories and scenes that are scarce or under-represented in existing image datasets; According to the categories and the scenarios, a collection strategy is designed to collect data under the categories and the scenarios, thereby supplementing the existing image data set.

3. The method for constructing an image dataset according to claim 2, characterized in that: Expand the labeled image dataset to include: Perform data enhancement for the categories and / or scenarios; Generate new images using generative adversarial networks; The enhanced or generated images are screened using a large model to ensure that the features of the enhanced or generated images are consistent with the real data, and the screened images are added to the annotated image dataset.

4. The method for constructing an image dataset according to claim 3, characterized in that: Using the large model to screen the enhanced or generated images, ensuring that the features of the enhanced or generated images are consistent with the real data, and adding the screened images to the annotated image dataset, further comprising: Use the multimodal large model CLIP to perform real-time analysis on the annotated image dataset, generate coverage reports and dataset performance evaluations, and the multimodal large model CLIP will mark poorly performing categories and indicate the need for further data enhancement or acquisition. According to the coverage report and the data set performance evaluation, the coverage and category balance of the image data set are optimized by introducing new samples.

5. The method for constructing an image dataset according to claim 1, characterized in that: The multimodal large model CLIP is used to annotate the supplemented image dataset, and the accuracy of the annotation is ensured through multi-level quality control, including: Use the multimodal large model CLIP for reasoning to classify and annotate unlabeled images; Use image embedding vectors to perform label clustering analysis, identify and correct labeling errors, and ensure the consistency of labels in feature space; The annotation results are cross-validated through different artificial intelligence models to ensure the accuracy of the annotation results.

6. The method for constructing an image dataset according to claim 5, characterized in that: Using the confidence scoring system of the multimodal large model CLIP, select images with unqualified confidence and submit them to the manual review platform.

7. The method for constructing an image dataset according to claim 1, characterized in that: The other modality data includes text data, and the image dataset obtained after expansion is fused with the other modality data to complete the construction of the final image dataset, including: Generate a text description corresponding to the image data set using the multimodal large model CLIP, and fuse it with the text data; or The multimodal large model CLIP is used to extract semantic information related to the image dataset from the text data and fuse it with the image dataset.

8. The method for constructing an image dataset according to claim 1, characterized in that: Also includes: Version control, automatic anomaly detection, and dynamic optimization of image datasets with each update, including: Use the multimodal large model CLIP to record the log of each image dataset update and generate a dataset snapshot; Detect potential abnormal samples in each updated image dataset and generate reports for processing; Based on the report, adjust the image data acquisition and annotation strategy for each update to continuously optimize the image dataset quality.

9. The method for constructing an image dataset according to claim 1, characterized in that: Also includes: Combining differential privacy technology and blockchain technology, data is desensitized to ensure the privacy and security of data during processing, transmission and storage.

10. A system for constructing an image dataset, characterized in that: The method for constructing an image dataset according to any one of claims 1 to 9 comprises: Intelligent data acquisition module to identify deficiencies in existing image datasets and supplement them; The automatic annotation and quality control module is used to annotate the supplemented image dataset and ensure the accuracy of the annotation through multi-level quality control; Adaptive data extension module, used to expand the labeled image dataset; The cross-modal dataset construction module is used to fuse the expanded image dataset with other modality data to complete the construction of the final image dataset.

Images