Image multi-attribute automatic pre-labeling method based on multi-modal large language model

Abstract

The invention particularly relates to an image multi-attribute automatic pre-labeling method based on a multi-modal large language model, which relates to the technical field of computer vision and artificial intelligence, and comprises the following steps of: step 3, dynamically generating a complete cue word aiming at a current image according to Schema configuration of a task and a cue word template; sending the Base64 coded image and the cue word together to a multi-mode large language model API (Application Program Interface); and analyzing and extracting a natural language response output by the model into a structured JSON format labeling result through a system. According to the method, a full-process automation and batch processing mechanism is adopted, the task labeling cycle is compressed, and repeated labor is reduced; in the task initialization stage, the Schema configuration design enables the system to automatically generate a labeling form and a structured cue word, and a template does not need to be manually written or an interface does not need to be manually built; the format standardization of an image preprocessing link, Base64 code conversion and compression are automatically completed, and the complexity and errors of manual processing are avoided.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to an automatic pre-annotation method for multiple attributes of images based on a multimodal large language model. Background Technology

[0002] With the rapid development of artificial intelligence and machine learning technologies, image recognition and analysis have been widely applied in fields such as autonomous driving, intelligent monitoring, medical diagnosis, and industrial inspection. These applications all require large amounts of high-quality labeled data to train and optimize algorithm models. However, traditional image annotation methods suffer from the following technical problems: Manual annotation is inefficient: Traditional image annotation relies entirely on manual operation. Annotators need to examine each image one by one and manually mark various attributes, which is a huge workload and extremely inefficient. Taking vehicle image annotation as an example, it usually takes 2-5 minutes to annotate an image containing multiple attributes such as vehicle type, color, direction, and license plate type. The annotation of large-scale datasets may take months or even years. Unstable annotation quality: Manual annotation is easily affected by factors such as the annotator's subjective judgment, fatigue, and skill level, leading to inconsistent annotation results and a high error rate and subjectivity. Different annotators may have different judgments on the attributes of the same image, affecting the quality and consistency of the dataset; High annotation costs: Large-scale image annotation requires hiring a large number of professional annotators, resulting in huge labor costs. At the same time, the annotation process requires rigorous quality control and multiple rounds of verification, further increasing time and economic costs.

[0003] Lack of flexibility and scalability: Existing automatic annotation methods are usually designed for specific image types and attribute categories. When it is necessary to annotate new image types or add new attribute dimensions, it is often necessary to retrain the model or develop new algorithms, which lacks versatility and flexibility. Multi-attribute annotation is highly complex: most existing technologies focus on the identification of a single attribute, lacking effective solutions for complex scenarios requiring the simultaneous identification of multiple related attributes. Furthermore, multiple attributes may have interrelationships and constraints, which traditional methods struggle to handle effectively. Low standardization of annotation: Different projects and teams often use different annotation standards and formats, lacking a unified annotation specification, which makes it difficult for datasets to be compatible and reused.

[0004] Therefore, an automatic image multi-attribute pre-annotation method based on a multimodal large language model is proposed to address the aforementioned problems. Summary of the Invention

[0005] The purpose of this invention is to propose an automatic pre-annotation method for multiple attributes of images based on a multimodal large language model in order to solve the above-mentioned problems.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: Automatic image multi-attribute pre-annotation methods based on multimodal large language models include: Step 1: Define the attribute types, optional value ranges, and constraints between attributes in JSON format. The system parses the schema configuration to generate the corresponding annotation form structure and validation rules. Step 2: Perform format verification and standardization on the uploaded images, and establish the association between image files and tasks; Step 3: Dynamically generate complete prompts for the current image based on the task's schema configuration and prompt template; send the Base64 encoded image and prompts together to the multimodal large language model API; the natural language response output by the model is parsed and extracted by the system into structured JSON format annotation results; Step 4: Verify the legality of the AI ​​pre-labeling results based on the schema configuration; at the same time, the system evaluates the confidence level of the model output.

[0007] Preferably, step one further includes: Users then upload prompt template files for specific image types and attributes. The system dynamically generates structured AI prompts that include attribute definitions, optional value descriptions, and output format requirements based on the schema configuration; and assigns a unique identifier to each annotation task.

[0008] Preferably, in step three: The prompts include role definitions, task descriptions, attribute definitions, output format requirements, and example descriptions to ensure structure and standardization.

[0009] Preferably, in step four: In the confidence assessment, results marked as low confidence are manually verified, and the overall annotation quality is quantitatively managed through a scoring mechanism.

[0010] Preferably, the method further includes: Step 5: Provide a visual annotation interface to display the original image and AI pre-annotation results. The system records the content and reasons for manual modifications.

[0011] Preferably, the method further includes: Step 6: Support asynchronous batch pre-annotation of all images in the task, and run concurrent processing mechanisms.

[0012] Preferably, the system also provides real-time processing progress display, records the number of successful and failed images, and allows tasks to be paused and resumed midway through processing for flexible management.

[0013] Preferably, the method further includes: Step 7: Supports exporting results in multiple standard formats, allowing export of pre-annotated results and final annotated results separately, and supports custom field and format selection.

[0014] Preferably, the final system generates a standard dataset that meets the requirements for machine learning training, including image paths, annotation results, and metadata.

[0015] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention reduces the annotation task cycle and repetitive labor through a fully automated and batch processing mechanism. In the task initialization stage, the schema configuration design allows the system to automatically generate annotation forms and structured prompts, eliminating the need for manual template writing or interface building. The format standardization, Base64 encoding conversion, and compression in the image preprocessing stage are all completed automatically, avoiding the tedium and errors of manual processing.

[0016] 2. This invention allows manual labor to focus only on verifying and correcting low-confidence results through a visual editing interface, eliminating the need for complete annotation of all images one by one. The entire process frees manual labor from repetitive tasks, retaining only key quality control steps, thus improving annotation efficiency several times over. It is particularly suitable for large-scale image dataset annotation scenarios, significantly reducing labor costs. Attached Figure Description

[0017] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a system architecture diagram of the present invention. Detailed Implementation

[0018] Several embodiments of this application will now be described in more detail with reference to the accompanying drawings to enable those skilled in the art to implement this application. This application may be embodied in many different forms and for various purposes and should not be limited to the embodiments set forth herein. These embodiments are provided to make this application thorough and complete, and to fully convey the scope of this application to those skilled in the art. The embodiments described do not limit this application.

[0019] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having a meaning consistent with their meaning in the relevant field and / or the context of this specification, and shall not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0020] Example 1 Its specific implementation method is combined with the appendix Figure 1 and attached Figure 2 Please provide a detailed explanation.

[0021] Appendix Figure 1 The diagram below illustrates the structure of the automatic image multi-attribute pre-annotation method based on a multimodal large language model provided in this embodiment of the invention. It shows the relationship between defining the attribute types to be annotated in JSON format and the steps of the final system generating a standard dataset that meets the requirements for machine learning training, and annotates the process of each step.

[0022] In this embodiment, it includes: Step 1: Annotation Task Configuration and Initialization: In the annotation task configuration stage, users define the attribute types, optional value ranges, and constraints between attributes in JSON format. The system parses the schema configuration to generate the corresponding annotation form structure and validation rules, supporting multiple attribute data types such as enumeration, numeric, and text. Users then upload prompt template files for specific image types and attributes. The system dynamically generates structured AI prompts that include attribute definitions, optional value descriptions, and output format requirements based on the schema configuration. A unique identifier is assigned to each annotation task, an independent file storage directory structure is created, and a task status tracking and progress management mechanism is initialized.

[0023] Step 2: Image Data Preprocessing: In the image data preprocessing stage, the system supports multiple methods such as single file upload, batch upload, and local path loading. It performs format verification and standardization on uploaded images and establishes the association between image files and tasks. Subsequently, the system converts the images to Base64 encoding for API transmission, performs appropriate compression and resizing on the images to optimize processing efficiency, and maintains a balance between image quality and speed. Step 3: Multimodal AI Pre-annotation Processing: In the multimodal AI pre-annotation stage, the system dynamically generates complete prompts for the current image based on the task's schema configuration and prompt word template. Subsequently, the system sends the Base64 encoded image and prompts together to the multimodal large language model API, and achieves image content understanding through visual language models such as Qwen2.5-VL. The natural language response output by the model is parsed and extracted by the system into structured JSON format annotation results, and format verification, data type conversion, and anomaly handling are performed. The prompts include role definitions, task descriptions, attribute definitions, output format requirements, and example descriptions to ensure structure and standardization.

[0024] Step 4: Quality Control of Annotation Results: In the quality control stage of annotation results, the system verifies the legality of AI pre-annotation results based on the schema configuration, checks whether the attribute values ​​are within the predefined range and verifies the logical consistency between attributes; at the same time, the system evaluates the confidence level of the determinism of the model output. In the confidence assessment, results marked as low confidence are manually reviewed, and the overall annotation quality is quantitatively managed through a scoring mechanism.

[0025] Step 5: Human-Computer Collaborative Annotation: In the human-computer collaborative annotation stage, the system provides a visual annotation interface to display the original image and AI pre-annotation results, and supports quick correction and confirmation through an intuitive attribute editing interface; annotators can view and modify the AI ​​results, and the system records the content and reasons for manual modifications, thereby establishing a comparative analysis mechanism between AI pre-annotation and manual annotation to continuously optimize model performance.

[0026] Step Six: Batch Processing and Progress Management: In the batch processing and progress management stage, the system supports asynchronous batch pre-annotation of all images in the task, improving overall efficiency and avoiding long waiting times through concurrent processing mechanisms.

[0027] The system also provides real-time processing progress display, records the number of successful and failed images, and allows tasks to be paused and resumed midway for flexible management.

[0028] Step 7: Data Export and Formatting: During the data export and formatting stage, the system supports exporting results in multiple standard formats such as JSONL, CSV, or XML. It can export pre-annotated results and final annotated results separately, and supports custom field and format selection.

[0029] The system ultimately generates a standard dataset that meets the requirements for machine learning training, containing complete information such as image paths, annotation results, and metadata, and has the ability to manage dataset versions and perform incremental updates.

[0030] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0031] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0032] It should be noted that, in this document, the use of relational terms such as "first" and "second" is merely for distinguishing one entity or operation from another, and does not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0033] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0034] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0035] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0036] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0037] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0038] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0039] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. An automatic image multi-attribute pre-annotation method based on a multimodal large language model, characterized in that, include: Step 1: Define the attribute types, optional value ranges, and constraints between attributes in JSON format. The system parses the schema configuration to generate the corresponding annotation form structure and validation rules. Step 2: Perform format verification and standardization on the uploaded images, and establish the association between image files and tasks; Step 3: Dynamically generate complete prompts for the current image based on the task's schema configuration and prompt template; send the Base64 encoded image and prompts together to the multimodal large language model API; the natural language response output by the model is parsed and extracted by the system into structured JSON format annotation results; Step 4: Verify the legality of the AI ​​pre-labeling results based on the schema configuration; at the same time, the system evaluates the confidence level of the model output.

2. The automatic image multi-attribute pre-annotation method based on a multimodal large language model according to claim 1, characterized in that, Step one also includes: Users then upload prompt template files for specific image types and attributes. The system dynamically generates structured AI prompts that include attribute definitions, optional value descriptions, and output format requirements based on the schema configuration; and assigns a unique identifier to each annotation task.

3. The automatic image multi-attribute pre-annotation method based on a multimodal large language model according to claim 1, characterized in that, In step three: The prompts include role definitions, task descriptions, attribute definitions, output format requirements, and example descriptions to ensure structure and standardization.

4. The automatic image multi-attribute pre-annotation method based on a multimodal large language model according to claim 1, characterized in that, In step four: In the confidence assessment, results marked as low confidence are manually verified, and the overall annotation quality is quantitatively managed through a scoring mechanism.

5. The automatic image multi-attribute pre-annotation method based on a multimodal large language model according to claim 1, characterized in that, Also includes: Step 5: Provide a visual annotation interface to display the original image and AI pre-annotation results. The system records the content and reasons for manual modifications.

6. The automatic image multi-attribute pre-annotation method based on a multimodal large language model according to claim 1, characterized in that, Also includes: Step 6: Support asynchronous batch pre-annotation of all images in the task, and run concurrent processing mechanisms.

7. The automatic image multi-attribute pre-annotation method based on a multimodal large language model according to claim 6, characterized in that, The system also provides real-time processing progress display, records the number of successful and failed images, and allows tasks to be paused and resumed midway for flexible management.

8. The automatic image multi-attribute pre-annotation method based on a multimodal large language model according to claim 1, characterized in that, Also includes: Step 7: Supports exporting results in multiple standard formats, allowing export of pre-annotated results and final annotated results separately, and supports custom field and format selection.

9. The automatic image multi-attribute pre-annotation method based on a multimodal large language model according to claim 8, characterized in that, The system ultimately generates a standard dataset that meets the requirements for machine learning training, including image paths, annotation results, and metadata.

Images