Label data generation method and device and storage medium

Abstract

The embodiment of the invention provides a data annotation generation method and device and a storage medium. The method comprises the steps that after a to-be-labeled image is acquired, the image content of the to-be-labeled image is recognized based on a plurality of pre-trained machine learning models, and a plurality of pieces of initial labeling data of the to-be-labeled image are determined; and determining target annotation data of the to-be-annotated image based on the initial annotation data corresponding to each machine learning model.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence, and in particular to a method for generating labeled data, a device for generating labeled data, and a storage medium. Background Art

[0002] With the rapid development of artificial intelligence applications, the demand for labeled data is increasing year by year. High-quality labeled data not only effectively improves model accuracy but also accelerates algorithm iteration and optimization. Therefore, efficient and accurate data labeling has become key to promoting the widespread implementation of AI applications.

[0003] In related technical solutions, data annotation is performed on the training data of the machine learning model manually. For example, in the scenario of business type annotation, based on the business environment image of the business, the content in the image is manually identified, such as: the fixedness of the business facilities (such as fixed shelves, permanent signs), the spatial layout of the venue (such as the difference between mobile stalls and fixed stores), the completeness of the infrastructure and other key visual features, and then the business type of the business is annotated. However, in this technical solution, when the amount of data is large, the manual method is not only time-consuming, but also different annotators have different understandings of the annotation rules, resulting in low accuracy.

[0004] The content of the background technology section is merely information known to the inventor personally, and does not mean that the above information has entered the public domain before the application date of this disclosure, nor does it mean that it can become the prior art of the present disclosure. Summary of the Invention

[0005] This specification provides a method, device and storage medium for generating annotation data, which can combine multiple machine learning models to determine the target annotation data of an image based on the annotation data of the image, thereby not only improving the image annotation efficiency, but also improving the accuracy of the annotation data.

[0006] In a first aspect, the present specification provides a method for generating annotation data, comprising: obtaining an image to be annotated; identifying the image content of the image to be annotated based on multiple pre-trained machine learning models, and determining multiple initial annotation data of the image to be annotated; and determining target annotation data of the image to be annotated based on the initial annotation data corresponding to each of the machine learning models.

[0007] In some example embodiments, based on the above solution, the multiple machine learning models include a multimodal recognition model and an image recognition model. The multiple pre-trained machine learning models recognize the image content of the image to be annotated and determine the multiple pieces of initial annotation data for the image to be annotated, including:

[0008] Recognizing the image content of the image to be annotated based on the multimodal recognition model to generate first initial annotation data for the image to be annotated; and

[0009] The image content of the image to be annotated is recognized based on the image recognition model to generate second initial annotation data for the image to be annotated.

[0010] In some example embodiments, based on the above solution, the first initial annotation data includes an image description, the multiple machine learning models also include a language model, and the method further includes:

[0011] inputting the image description output by the multimodal recognition model into the language model;

[0012] The image description is recognized based on the language model to generate third initial annotation data of the image to be annotated.

[0013] In some example embodiments, based on the above scheme, the multimodal recognition model is a large multimodal model, and the prompt words of the large multimodal model include image classification definitions, classification instructions, and image description instructions; the language model is a large language model, and the prompt words of the large language model include the image classification definitions, the image descriptions, and the classification instructions.

[0014] In some example embodiments, based on the above solution, the prompt word of the large language model further includes an instruction for generating a confidence score.

[0015] In some example embodiments, based on the above solution, the third initial labeled data includes a confidence score corresponding to the classification result, and the method further includes:

[0016] The plurality of initially labeled data are screened based on the confidence scores, and the initially labeled data having confidence scores less than a predetermined threshold are removed.

[0017] In some example embodiments, based on the above solution, determining target annotation data for the image to be annotated based on the initial annotation data corresponding to each of the machine learning models includes:

[0018] Performing multi-model voting based on the initial labeled data corresponding to each of the machine learning models; and

[0019] The target annotation data of the image to be annotated is determined based on the voting result of the multi-model voting.

[0020] In some example embodiments, based on the above solution, the initial annotation data includes a classification result, and determining the target annotation data of the image to be annotated based on the voting result of the multi-model voting includes:

[0021] If the classification results of the multiple machine learning models are consistent, the consistent initial labeled data will be used as the target labeled data.

[0022] In some example embodiments, based on the above scheme, the image recognition model includes a business classification model, and the image to be labeled is an image of the business environment of the merchant.

[0023] In some example embodiments, based on the above solution, before identifying the image content of the image to be annotated based on the pre-trained multiple machine learning models, the method further includes:

[0024] In response to a model selection operation for a target task, determining a plurality of pre-trained machine learning models corresponding to the target task;

[0025] In response to a parameter configuration operation for a target machine learning model, model parameters corresponding to the target machine learning model are determined.

[0026] In a second aspect, the present specification also provides a device for generating annotation data, comprising: at least one storage medium storing at least one instruction set for generating annotation data; and at least one processor communicatively connected to the at least one storage medium, wherein when the device is running, the at least one processor reads the at least one instruction set, and according to the instructions of the at least one instruction set: obtains an image to be annotated; identifies the image content of the image to be annotated based on multiple pre-trained machine learning models, and determines multiple initial annotation data for the image to be annotated; and determines the target annotation data of the image to be annotated based on the initial annotation data corresponding to each of the machine learning models.

[0027] In a third aspect, this specification also provides a computer-readable non-volatile storage medium, wherein the computer-readable non-volatile storage medium stores at least one instruction set, and when the at least one instruction set is executed by at least one processor, the annotation data generation method as described in any one of the first aspects is implemented.

[0028] According to the annotation data generation method, device and storage medium provided in this specification, on the one hand, the initial annotation data of the image to be annotated is determined based on multiple pre-trained machine learning models. Compared with manual annotation, the annotation efficiency of the machine learning model is high, and the processing of a large amount of data can be completed in a shorter time, which significantly reduces the overall cost of data annotation; on the other hand, the annotation data of the image based on multiple machine learning models are combined to determine the target annotation data of the image. By combining the complementary capabilities of multiple machine learning models, the features of different dimensions of the image can be captured, thereby improving the accuracy of data annotation and avoiding the problem of low accuracy caused by deviations in the understanding of annotation rules by different annotators.

[0029] Other functions of the annotated data generation method, device, and storage medium provided in this specification are partially listed in the following description. The creative aspects of the annotated data generation method, device, and storage medium provided in this specification can be fully explained by practicing or using the methods, devices, and combinations described in the following detailed examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of this specification, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0031] Figure 1 A schematic diagram of an application scenario of a method for generating labeled data according to an embodiment of this specification is shown;

[0032] Figure 2 shows a hardware structure diagram of a device for generating annotation data according to an embodiment of this specification;

[0033] Figure 3 A flow chart of a method for generating annotation data according to an embodiment of this specification is shown;

[0034] Figure 4 A schematic diagram illustrating an interface of a model configuration interface provided according to an embodiment of this specification; and

[0035] Figure 5 A schematic flow chart of a method for generating annotation data according to other embodiments of this specification is shown. DETAILED DESCRIPTION

[0036] The following description provides specific application scenarios and requirements for this specification, with the goal of enabling those skilled in the art to make and use the contents of this specification. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but is intended to be accorded the broadest scope consistent with the claims.

[0037] The terms used herein are for the purpose of describing specific example embodiments only and are not intended to be limiting. For example, as used herein, the singular forms "a," "an," and "the" may also include the plural forms, unless the context clearly indicates otherwise. When used in this specification, the terms "comprise," "include," and / or "contain" are intended to refer to the presence of the associated integers, steps, operations, elements, and / or components, but do not preclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups or the addition of other features, integers, steps, operations, elements, components, and / or groups to the device / method.

[0038] In this specification, "X includes at least one of A, B, or C" means that X includes at least A, or X includes at least B, or X includes at least C. In other words, X may include only any one of A, B, and C, or any combination of A, B, and C, as well as other possible contents / elements. Any combination of A, B, and C may be A, B, C, AB, AC, BC, or ABC.

[0039] In this specification, unless otherwise specified, the association relationship between structures can be a direct association relationship or an indirect association relationship. For example, when describing "A is connected to B", unless it is clearly stated that A is directly connected to B, it should be understood that A can be directly connected to B or indirectly connected to B; for another example, when describing "A is above B", unless it is clearly stated that A is directly above B (AB are adjacent and A is above B), it should be understood that A can be directly above B or indirectly above B (AB is separated by other elements and A is above B). And so on.

[0040] These and other features of this specification, as well as the operation and function of the associated elements of the structure, and the economical assembly and manufacture of the components, can be significantly improved with consideration of the following description. Reference is made to the accompanying drawings, all of which form a part of this specification. However, it should be expressly understood that the drawings are for illustration and description purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0041] The flowcharts used in this specification illustrate operations implemented by devices according to some embodiments of this specification. It should be clearly understood that the operations of the flowcharts may not be implemented in order. Rather, the operations may be implemented in reverse order or simultaneously. In addition, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0042] For the convenience of description, the terms that will appear in the following text of this specification are first explained.

[0043] Multimodal large models: AI models capable of processing and understanding multiple modalities. Modality refers to the different forms of data, with common modalities including text, images, audio, and video. Multimodal large models can not only process text but also simultaneously understand and generate other forms of data, such as images, sound, and video. They can fuse and correlate information from these different modalities, enabling richer reasoning and task execution.

[0044] Data annotation: In machine learning, the process of adding labels, categories, or annotations to raw data in a dataset (such as text, images, video, or audio). Data annotation enables models to learn from large amounts of data and make predictions or decisions. Data annotation is the foundation of supervised learning and is widely used in fields such as natural language processing, computer vision, and speech recognition.

[0045] Prompts: In the fields of artificial intelligence (AI) and natural language processing (NLP), prompts are text or questions fed into a language model, guiding the model to generate relevant outputs. Prompts are commands or questions entered by users when interacting with the model, and the model generates corresponding responses based on these prompts. Prompts serve as a bridge between the model and the user, determining the direction, style, and content of the generated text.

[0046] In related technical solutions, data annotation is performed manually on training data for machine learning models. However, manual annotation is time-consuming when the data volume is large. Furthermore, different annotators may have different understandings of annotation rules when faced with the same task, resulting in low accuracy of the labeled data. However, machine learning model annotation results are relatively consistent, and the quality of annotation is not affected by factors such as emotion and fatigue, making model annotation more advantageous. Furthermore, if only a single machine learning model is used for annotation, accuracy may not be fully guaranteed due to the model's limitations. Because each model has inherent biases and errors, relying on a single model may be unable to handle certain complex situations. However, using multiple models in collaboration, using independent models to make judgments and then combining the annotation data from multiple models to determine the final annotation data, can effectively reduce the probability of misjudgment by a single model. The complementarity between multiple models can offset their respective shortcomings, thereby improving overall annotation accuracy and reducing the likelihood of errors.

[0047] Based on the above content, the embodiments of this specification provide a method for generating annotation data, an annotation data generating device and a storage medium. Among them, the client in the annotation data generating device receives the image to be annotated, identifies the image content of the image to be annotated based on multiple pre-trained machine learning models, and determines multiple initial annotation data of the image to be annotated; then, multi-model voting is performed based on the initial annotation data corresponding to each machine learning model. If the classification results of multiple machine learning models are consistent, the consistent initial annotation data is used as the target annotation data. According to the technical solution in the above example embodiment, on the one hand, the initial annotation data of the image to be annotated is determined based on multiple pre-trained machine learning models. Compared with manual annotation, the annotation efficiency of the machine learning model is high, and a large amount of data can be processed in a shorter time, significantly reducing the overall cost of data annotation; on the other hand, the target annotation data of the image is determined by combining the annotation data of the image with multiple machine learning models. By combining the complementary capabilities of multiple machine learning models, the features of different dimensions of the image can be captured, thereby improving the accuracy of data annotation and avoiding the problem of low accuracy caused by the deviation in the understanding of the annotation rules by different annotators.

[0048] The following is an introduction to the application scenarios of this manual.

[0049] The embodiments of this specification can be applied to various scenarios involving the generation of image annotation data, and are particularly suitable for image classification scenarios, such as those for business environments, and can be used to provide business type annotation services. The aforementioned businesses can be those that provide various goods or services, including but not limited to: restaurants, grocery stores, fruit and vegetable stores, florists, bookstores, hotels, beauty salons, dry cleaners, barber shops, etc., without any specific limitation in this specification.

[0050] It should be noted that the above-mentioned merchant transaction scenario is only one of the multiple usage scenarios provided in this specification. The annotation data generation method provided in this specification can be applied not only to the merchant business type annotation scenario, but also to various image classification scenarios, such as security scenarios, medical image analysis scenarios, autonomous driving scenarios, industrial inspection scenarios, food safety inspection scenarios, etc. Those skilled in the art should understand that when the data annotation generation method provided in this specification is applied to other usage scenarios, its implementation method and technical effects are similar.

[0051] Figure 1 FIG1 shows a schematic diagram of an application scenario of a method for generating annotated data according to an embodiment of this specification. Figure 1 As shown, the application scenario 100 may include a client 120 , a server 130 , and a network 140 .

[0052] User 110 may be the user who triggers the data annotation operation. User 110 may perform configuration operations for data annotation on client 120, such as configuring model parameters and input and output parameters of each machine learning model.

[0053] Client 120 can be a device that recognizes the image content of an image to be annotated based on multiple machine learning models configured by user 110. In some embodiments, the method for generating annotated data can be executed on client 120. In this case, client 120 can store data or instructions for executing the method for generating annotated data described in this specification and can execute or be used to execute such data or instructions. In some embodiments, client 120 can include hardware devices with data information processing capabilities and the necessary programs to operate the hardware devices.

[0054] The client 120 is a multifunctional terminal. In some embodiments, the client 120 may be a mobile device, such as a handheld device. The client 120 can be carried by the user to various locations for operation, providing greater convenience. In some embodiments, the client 120 may be a non-mobile device. In this case, the client 120 is typically placed in a fixed location.

[0055] In some embodiments, the client 120 may have a display screen, or the client 120 may be connected to an external display screen. The client 120 may provide interactive capabilities through the display screen. For example, the client 120 may display a store image of a merchant through the display screen.

[0056] In some embodiments, the client 120 may include a mobile device, a tablet computer, a laptop computer, or any combination thereof. In some embodiments, the client 120 may include an image acquisition device for acquiring video or image information to obtain images to be annotated. In some embodiments, the image acquisition device may be a two-dimensional image acquisition device (such as an RGB camera), or a two-dimensional image acquisition device (such as an RGB camera) and a depth image acquisition device (such as a 3D structured light camera, a laser detector, etc.).

[0057] In some embodiments, client 120 may have one or more application programs (APPs) installed. These APPs can provide user 110 with the ability and interface to interact with the outside world via network 140. In some embodiments, client 120 may have a target APP installed. These target APPs can collect video or image information for client 120, thereby obtaining images to be annotated.

[0058] The server 130 may be a server that provides various services, such as a background server that provides support for the images to be annotated collected on the client 120. In some embodiments, the annotation data generation method may be executed on the server 130. At this time, the server 130 may store data or instructions for executing the data annotation generation method described in this specification, and may execute or be used to execute the data or instructions. In some embodiments, the server 130 may include a hardware device with a data information processing function and the necessary programs required to drive the hardware device to work. The server 130 may be connected to multiple clients 120 for communication and receive data sent by the client 120.

[0059] The network 140 is used to provide a medium for communication connection between the client 120 and the server 130. The network 140 can facilitate the exchange of information or data. Figure 1 As shown, the client 120 and the server 130 can be connected to the network 140 and transmit information or data to each other through the network 140. In some embodiments, the network 140 can be any type of wired or wireless network, or a combination thereof. For example, the network 140 can include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, a near field communication (NFC) network, or the like. In some embodiments, the network 140 can include one or more network access points. For example, the network 140 can include a wired or wireless network access point, such as a base station or an Internet exchange point, through which one or more components of the client 120 and the server 130 can connect to the network 140 to exchange data or information.

[0060] It should be understood that Figure 1 The number of clients 120, servers 130 and networks 140 in FIG. 1 is merely illustrative. Any number of clients 120, servers 130 and networks 140 may be provided as needed.

[0061] It should be noted that the annotation data generating method may be completely executed on the client 120 , may be completely executed on the server 130 , or may be partially executed on the client 120 and partially executed on the server 130 .

[0062] It should be noted that the user data obtained in this manual has been authorized by the user and does not involve user privacy.

[0063] Figure 2 FIG1 shows a hardware structure diagram of a device 200 for generating annotated data according to an embodiment of the present specification. The device 200 can be used as Figure 1 The client 120 or server 130 executes the annotation data generation method described in this specification.

[0064] like Figure 2 As shown, the annotation data generating device 200 may include at least one storage medium 230 and at least one processor 220. In some embodiments, the annotation data generating device 200 may further include a communication port 250 and an internal communication bus 210. The annotation data generating device 200 may further include an I / O component 240.

[0065] The internal communication bus 210 can connect various system components, such as the storage medium 230 , the processor 220 , the communication port 250 , and the I / O component 240 .

[0066] The I / O component 240 supports input / output between the annotation data generating device 200 and other components.

[0067] The communication port 250 is used for data communication between the annotation data generating device 200 and the outside world. For example, the communication port 250 can be used for data communication between the annotation data generating device 200 and the wireless network 140. The communication port 250 can be a wired communication port or a wireless communication port.

[0068] The storage medium 230 may include a data storage device. The data storage device may be a computer-readable non-transitory storage medium or a computer-readable temporary storage medium. For example, the data storage device may include one or more of a disk 232, a read-only storage medium (ROM) 234, or a random access storage medium (RAM) 236. The storage medium 230 also includes at least one instruction set stored in the data storage device. The instruction set may include computer program code, and the computer program code may include programs, routines, objects, components, data structures, processes, modules, etc. At least one set of instruction sets is stored in the storage medium 230, and the instruction set is a computer program code. When these instruction sets are called and executed by the processor, they can instruct the processor to perform specific functions or steps, such as the various methods and steps to be introduced later in this application.

[0069] At least one processor 220 can be in communication with at least one storage medium 230. When the annotation data generating device 200 is running, the at least one processor 220 reads the at least one set of instruction sets and executes the methods and steps provided in this specification according to the instructions of the at least one instruction set. The processor 220 can be in the form of one or more processors. In some embodiments, the processor 220 can include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), any circuit or processor capable of performing one or more functions, etc., or any combination thereof.

[0070] For illustrative purposes only, the annotation data generating device 200 in the accompanying drawings shows only one processor 220. However, it should be noted that the annotation data generating device 200 in this specification may also include multiple processors. Therefore, the operations and / or method steps disclosed in this specification may be performed by one processor or jointly by multiple processors. For example, if the processor 220 of the annotation data generating device 200 is described in this specification as performing step A and step B, it should be understood that step A and step B may also be performed jointly or separately by two different processors 220 (for example, the first processor performs step A, the second processor performs step B, or the first and second processors perform steps A and B together).

[0071] The annotated data generating device 200 can execute the annotated data generating method of this specification. Specifically, the processor 220 in the annotated data generating device 200 can read the instruction set stored in its local storage medium, and then execute the annotated data generating method of this specification according to the provisions of the instruction set.

[0072] Figure 3 A flowchart of a method P300 for generating labeled data according to an embodiment of this specification is shown. As previously described, electronic device 200 can execute method P300 for generating labeled data according to this specification. Specifically, processor 220 in electronic device 200 can read an instruction set stored in its local storage medium and then execute method P300 according to the instructions set.

[0073] like Figure 3 As shown, the annotation data generation method P300 may include:

[0074] S310: Acquire an image to be annotated.

[0075] In an example embodiment, images to be labeled refer to images that have not yet been labeled or annotated in the field of computer vision and machine learning. These images are usually raw, unprocessed image data that contain rich visual information but lack additional annotations or labels. The process of annotating images is to attach labels to meaningful content in the image, which is usually used to train machine learning models, especially for tasks such as image classification, target detection, and semantic segmentation in supervised learning. The images to be labeled contain rich visual information, such as people, objects, scenes, backgrounds, etc. The goal of the labeling process is to identify and describe important elements in the image. For example, in a merchant transaction scenario, the image to be labeled can be an image of the merchant's transaction environment.

[0076] In some embodiments, the image to be labeled can be used for image classification. For example, given an image of a business environment to be labeled, the business environment image includes the merchant's cash register status, the merchant's store layout, the display of goods, the store's decoration style, the type and quantity of products, the location of the store, the cleanliness of the store, etc. According to the information in the image, the merchant type is judged as "mobile merchant" or "sedentary merchant", and a label is added to the merchant's business environment image so that the merchant's business environment image can be classified into the corresponding category.

[0077] In some embodiments, the image to be labeled can be used for target detection. For example, consider a traffic image to be labeled, which may contain multiple cars, pedestrians, and buildings. In the traffic image to be labeled, a rectangular box is drawn around each object (e.g., a car or pedestrian), and each object is assigned a category label (e.g., "car" or "pedestrian"). This now provides labeled information for use in training target detection models.

[0078] In some embodiments, the image to be annotated can be used for semantic segmentation. For example, given a city street scene image to be annotated, which contains elements such as roads, buildings, pedestrians, and cars, semantic segmentation can be achieved by labeling each pixel in the city street scene image with its category, such as labeling roads as "road", buildings as "building", and pedestrians as "pedestrian".

[0079] Data labeling of unlabeled images is a crucial part of machine learning. This process assigns meaningful labels to images, providing raw training data for model training. Using labeled images, machine learning models can learn to recognize, classify, and understand different objects within an image.

[0080] In this example embodiment, the annotated data generation device 200 obtains an image to be annotated from a database that stores a plurality of pre-collected images to be annotated. For example, using the image to be annotated for image classification, the image to be annotated may be an image of a business environment, and the database stores a plurality of pre-uploaded images of business environments by multiple businesses.

[0081] Depending on different needs and data sources, the method of obtaining images to be annotated may also vary. In some embodiments, if images of certain scenes or types are required, a method of photographing images may be used. Different scenes may be photographed manually to construct a dataset of images to be annotated related to the data annotation project, such as using a mobile phone, camera, or professional equipment to capture the required images. There are many ways to obtain images to be annotated, and the annotation data generation device can select an appropriate method based on the needs. When choosing an acquisition method, the diversity of the images, the legitimacy of the data, and whether it meets specific requirements should be considered.

[0082] S320: Identify the image content of the image to be annotated based on multiple pre-trained machine learning models, and determine multiple initial annotation data of the image to be annotated.

[0083] In an exemplary embodiment, a pre-trained machine learning model refers to a machine learning model that has been trained on a large-scale dataset to enable transfer learning or fine-tuning across different tasks. Pre-trained models typically demonstrate good generalization across a variety of tasks and can achieve good performance on new tasks. The model parameters of multiple pre-trained machine learning models are pre-configured, and the annotation data generation device 200 invokes the configured multiple machine learning models to identify the image content of the image to be annotated, and determines the initial annotation data for each machine learning model for the image to be annotated.

[0084] For example, the labeling data generating device 200 determines a plurality of pre-trained machine learning models corresponding to the target task in response to a model selection operation for the target task; and determines model parameters corresponding to the target machine learning model in response to a parameter configuration operation for the target machine learning model. Figure 4 As shown, the target task is an image classification task, and multiple pre-trained machine learning models are pre-configured on the model configuration page. The annotation data generation device 200 responds to the model selection operation for the image classification task, selects a multimodal large model and an image classification model for the image classification task, and responds to the parameter configuration operation for the above-mentioned machine learning models, and configures the parameters of the multimodal large model, such as prompt words and prompt content parameters, in the parameter configuration module.

[0085] It should be noted that the multiple machine learning models can be models of different model architectures. For example, the multiple machine learning models include at least two of a multimodal recognition model, a language model, an image classification model, an object detection model, a semantic segmentation model, or an instance segmentation model. Among them, the image classification model is used to predict the category to which the image to be labeled belongs; the object detection model is used to identify the object category in the image and locate the position of each object (usually a bounding box). The semantic segmentation model assigns a label to each pixel in the image to be labeled, distinguishing between regions of different categories or different individuals under the same category.

[0086] By combining the complementary capabilities of multiple pre-trained machine learning models, it is possible to capture features of different dimensions of images, thereby improving the accuracy of data annotation.

[0087] In some example embodiments, the multiple machine learning models include a multimodal recognition model and an image recognition model. The electronic device 200 recognizes the image content of the image to be annotated based on the multimodal recognition model to generate first initial annotated data for the image to be annotated. Furthermore, the electronic device 200 recognizes the image content of the image to be annotated based on the image recognition model to generate second initial annotated data for the image to be annotated. For example, in an image classification task, both the first initial annotated data and the second initial annotated data include classification results.

[0088] Multimodal recognition models combine image recognition models and language models to form artificial intelligence models that can process and understand information from different modalities (data sources). In practical applications, these modalities usually refer to different types of input data, such as text, images, sounds, videos, etc. Multimodal recognition models can process and fuse data from multiple modalities. For example, a typical multimodal recognition model may accept text (such as sentence descriptions) and images (such as images or images) as input at the same time. Multimodal recognition models can not only understand each modality separately, but also make more accurate predictions by fusing this information.

[0089] Image recognition models are core tools in the field of computer vision, designed to analyze the content in images (such as objects, scenes, faces, etc.) and extract information or classify them. In some embodiments, the image recognition model includes a mobile vendor classification model, which is a pre-trained machine learning model that classifies images of business environments. Mobile vendors are those who operate businesses in fixed locations, such as shops and restaurants; while mobile vendors are those who do not have fixed locations and sell goods through mobile stalls or on the move, such as hawkers and taxi drivers.

[0090] In other embodiments, the multiple machine learning models include three machine learning models, namely an image recognition model, a language model, and a multimodal recognition model, which respectively recognize input image data from the perspectives of image, language, and multimodality. Language models are core tools in natural language processing (NLP), designed to understand, generate, or predict text content and capture the statistical laws and semantic relationships of language.

[0091] For example, the electronic device 200 recognizes the image content of the image to be annotated based on a multimodal recognition model, and generates first initial annotation data for the image to be annotated, where the first initial annotation data includes an image description; recognizes the image content of the image to be annotated based on the image recognition model, and generates second initial annotation data for the image to be annotated; and inputs the image description output by the multimodal recognition model into a language model, recognizes the image description based on the language model, and generates third initial annotation data for the image to be annotated.

[0092] Using multiple machine learning models can reduce the probability of misjudgment by a single model. The complementarity between multiple models can make up for their respective shortcomings, thereby improving the accuracy of overall labeling and reducing the chance of errors.

[0093] Furthermore, in an exemplary embodiment, the multimodal recognition model is a multimodal large model, and the prompt words of the multimodal large model include image classification definitions, classification instructions, and image description instructions. The annotation data generating device 200 recognizes the image content of the image to be annotated based on the multimodal large model, inputs the image to be annotated and the prompt words into the multimodal recognition model, and generates the first initial annotation data of the image to be annotated. Prompt words refer to instructions or text input to the model to help the model understand the user's needs and generate a response. The prompt words of the multimodal large model include image classification definitions, classification instructions, instructions to explain the classification reasons, and image description instructions. The output of the multimodal large model is the first initial annotation data of the image to be annotated, and the first initial annotation data includes the first classification result and the image description.

[0094] For example, a device for generating labeled data is used in a merchant transaction scenario. It can determine the merchant's type, such as a traveling merchant or a sedentary merchant, based on the merchant's business environment image. An image of the merchant's business environment is input into the multimodal large model, along with a prompt word. The prompt word of the multimodal large model includes the definition of traveling and sedentary merchants, instructions for classifying traveling and sedentary merchants, an explanation of the classification reason, and a description of the merchant's business environment image content. For example, the prompt word may include "Sedentary merchants refer to those who run businesses in fixed stores, such as shops and restaurants; while traveling merchants refer to those who do not have fixed locations and sell goods through mobile stalls or on the move, such as hawkers and taxi drivers. Please classify the image according to the above definition (sedentary merchant / traveling merchant) and explain the classification reason based on the image description. The output format is "Classification label. Classification reason based on image description". Based on the prompt word and the input image to be labeled, the multimodal large model is called to make a judgment on the merchant's business type and output an image description of the merchant's business environment image and the merchant type. The output result is the first initial labeled data for the image to be labeled, for example, the classification label: traveling merchant. Reason: The store does not have a fixed location and sells goods through mobile stalls or by walking around.

[0095] Furthermore, the aforementioned language model is a large language model, and the prompt words of the large language model include an image classification definition, an image description, and a classification instruction. The image classification definition and classification instruction are consistent with the image classification definition and classification instruction in the prompt words of the multimodal large model, and the image description is the image description output by the multimodal large model. The annotation data generation device 200 inputs the image description, image classification definition, and classification instruction of the image to be annotated into the large language model, recognizes the image description of the image to be annotated based on the large language model, and generates third initial annotation data for the image to be annotated, wherein the third initial annotation data includes the classification result.

[0096] By setting image classification definitions and classification instructions in the prompt words of the large model, it is possible to ensure that the large model maintains consistent classification rules for each image, avoiding the problem of low accuracy caused by deviations in the understanding of labeling rules by different labelers during manual labeling.

[0097] S330: Determine target annotation data for the image to be annotated based on the initial annotation data corresponding to each machine learning model.

[0098] In an exemplary embodiment, after obtaining the initial annotation data corresponding to each machine learning model, the annotation data generating device 200 fuses the initial annotation data corresponding to each machine learning model according to a predetermined fusion strategy to obtain target annotation data for the image to be annotated. Fusion strategies may include averaging, voting, weighted averaging / voting, and other strategies. The averaging method determines the target annotation data based on the average value of the output of each machine learning model, such as the average value of the classification probability; the voting mechanism selects the category with the most occurrences as the target annotation data; and the weighted averaging / voting method determines the target annotation data by calculating a weighted average or performing weighted voting based on the weights of each model.

[0099] In some example embodiments, the labeled data generating device 200 performs multi-model voting based on the initial labeled data corresponding to each machine learning model; and determines target labeled data for the image to be labeled based on the voting results of the multi-model voting. Multi-model voting is an ensemble learning method in machine learning that combines the prediction results of multiple models to improve overall prediction accuracy and robustness. The basic idea of this method is that while the predictions of a single model may contain errors, multiple models working together can reduce errors and provide more reliable predictions.

[0100] Furthermore, the labeled data generation device 200 votes on the classification results of multiple machine learning models. If the classification results of the multiple machine learning models are consistent, the consistent initial labeled data is used as the target labeled data. For example, if the classification results of three machine learning models are voted on, and all three classification results are "trader", the final vote result is "trader". The "trader" is the target labeled data.

[0101] Multi-model voting improves prediction accuracy, robustness, and generalization by combining the predictions of multiple machine learning models. Compared to predictions from a single machine learning model, this integration reduces errors and improves the stability of the labeled data generation device.

[0102] according to Figure 3The technical solution in the example embodiment, on the one hand, determines the initial annotation data of the image to be annotated based on multiple pre-trained machine learning models. Compared with manual annotation, the annotation efficiency of the machine learning model is high, and the processing of a large amount of data can be completed in a shorter time, which significantly reduces the overall cost of data annotation; on the other hand, the annotation data of the image based on multiple machine learning models are combined to determine the target annotation data of the image. By combining the complementary capabilities of multiple machine learning models, the features of different dimensions of the image can be captured, thereby improving the accuracy of data annotation and avoiding the problem of low accuracy caused by deviations in the understanding of annotation rules by different annotators.

[0103] Furthermore, in some example embodiments, the prompt word of the large language model also includes an instruction for generating a confidence score, and the third initial annotated data includes a confidence score corresponding to the classification result. Before performing multi-model voting, the annotated data generating device 200 screens the multiple pieces of initial annotated data based on the confidence score, for example, removing initial annotated data with a confidence score below a predetermined threshold and removing data that the large language model determines to be "unable to determine."

[0104] In a large language model, a predetermined threshold can be set for the confidence score. In some embodiments, the threshold can be set to 100. When the confidence score of a piece of data is less than the predetermined threshold, it indicates that the model's prediction is not certain enough. For example, if the image to be labeled is an image of a business environment, the model's confidence score for predicting that the image is of a traveling merchant is 70 or 80, rather than 100. This indicates that the model has some doubts about the accuracy of the prediction.

[0105] In the large language model, data judged as "unable to judge" means that the model is uncertain or not confident enough in its answer to the question or input. In some embodiments, if the input to the large language model is ambiguous, the question is not clear enough or there is insufficient information, resulting in the model being unable to understand the intent, the model will not be able to provide an accurate or reliable answer, resulting in an "unable to judge" result. In some embodiments, for some particularly complex tasks or tasks that require professional knowledge, the model may not have enough training data to provide an answer, causing it to determine that it is unable to provide an answer. In some embodiments, if the input data is too vague and the model does not have enough information to infer or make a reasonable judgment, it may also return an "unable to judge" result.

[0106] According to the technical solution in the above-mentioned example embodiment, by removing the labeled data with a confidence score lower than a predetermined threshold, the model can be prevented from providing uncertain or erroneous information, which helps to improve the reliability and accuracy of the overall device, and retains those predictions with scores higher than the predetermined threshold that can provide valuable answers.

[0107] Figure 4A schematic diagram of the model configuration interface provided according to an embodiment of this specification is shown.

[0108] Reference Figure 4 As shown, the model configuration interface includes a target task module, a model selection module, and a parameter configuration module. The target task module is used to select training tasks for the images to be annotated. For example, target tasks include image classification, object detection, and semantic segmentation. The model selection module is used to provide multiple pre-trained machine learning models for performing the target tasks. The parameter configuration module is used to configure the model parameters of the selected machine learning model. For example, if a multimodal large model is selected, prompt words and prompt content parameters need to be configured. Prompt content parameters include sampling temperature, kernel sampling threshold, and the number of candidate words. Sampling temperature, kernel sampling threshold, and the number of candidate words are important parameters for controlling the diversity and quality of generated text. The sampling temperature is a parameter used to adjust the probability distribution of the model output; the kernel sampling threshold refers to the process of considering only all words with a cumulative probability reaching a certain threshold p as the candidate set during the sampling process; and the number of candidate words refers to the selection of the top k most likely words as the candidate set from all possible words predicted by the model.

[0109] In an exemplary embodiment, the annotation data generating device 200 determines a plurality of pre-trained machine learning models corresponding to the target task in response to a model selection operation for the target task; and determines model parameters corresponding to the target machine learning model in response to a parameter configuration operation for the target machine learning model. Figure 4 As shown, the target task is an image classification task, and multiple pre-trained machine learning models are pre-configured on the model configuration page. The annotation data generation device 200 responds to the model selection operation for the image classification task, selects a multimodal large model and an image classification model for the image classification task, and responds to the parameter configuration operation for the above-mentioned machine learning model, and configures the parameters of the multimodal large model, such as prompt words and prompt content parameters, in the parameter configuration module.

[0110] Taking the image to be labeled as an image of a business environment as an example, the prompt words configured for the multimodal large model include the definition of mobile and sedentary merchants, instructions for classifying mobile and sedentary merchants, explanation of the classification reasons, and description of the merchant's business environment image content. For example, the configured prompt words include "sedentary merchants refer to those who run businesses in fixed stores, such as shops and restaurants; while mobile merchants refer to those who do not have fixed locations and sell goods through mobile stalls or on the move, such as hawkers and taxi drivers. Please classify the image according to the above definition (sedentary merchants / mobile merchants), and explain the classification reasons based on the image description. The output format is "classification label. Classification reasons based on image description"

[0111] According to the technical solution in the above-mentioned example embodiment, the target task, multiple machine learning models corresponding to the task, and model parameters are configured through the model configuration interface, so that images to be labeled for multiple tasks can be labeled, thereby generating labeled data for images to be labeled suitable for different tasks, further improving data labeling efficiency.

[0112] Figure 5 A schematic flow chart of a method for generating annotation data according to other embodiments of this specification is shown.

[0113] like Figure 5 As shown, in step S510, an image to be annotated is obtained.

[0114] In this example embodiment, the annotated data generation device 200 obtains an image to be annotated from a database that stores a plurality of pre-collected images to be annotated. For example, using the image to be annotated for image classification, the image to be annotated may be an image of a business environment, and the database stores a plurality of pre-uploaded images of business environments by multiple businesses.

[0115] In step S515 , the image to be annotated is preprocessed.

[0116] In an exemplary embodiment, preprocessing includes image resizing. The annotation data generating device 200 resizes the image to be annotated so that the image size can meet the input requirements of the multimodal large model. Preprocessing may also include normalization and color space conversion. For example, normalization of the image to be annotated may be performed, such as normalizing pixel values to a specific range (e.g., [0, 1] or [-1, 1]) to facilitate model convergence.

[0117] In step S520, prompt words of the multimodal large model are configured.

[0118] In an exemplary embodiment, the multimodal macromodel's prompts include image classification definitions, classification instructions, instructions explaining the classification reasons, and image description instructions. The annotation data generation device 200 configures the prompts for the multimodal macromodel in response to prompt word configuration operations on the model configuration page. For example, if the image to be annotated is an image of a business environment, the prompts for the multimodal macromodel include the definition of the business, instructions for classifying the business, explanations for the classification reasons, and a description of the business environment image content.

[0119] For example, the prompt words include "sedentary merchants refer to those who run businesses in fixed stores, such as shops and restaurants; while traveling merchants refer to those who do not have a fixed place and sell goods by setting up stalls or walking around, such as hawkers, taxi drivers, etc. Please classify the image according to the above definition (sedentary merchants / traveling merchants), and explain the classification reasons in combination with the image description. The format is "classification label. Classification reason combined with image description". Based on the prompt words and the input image, the multimodal large model makes a judgment on the merchant type, and outputs the image description and merchant type of the merchant's business environment image. The output result is the first initial annotation data of the image to be annotated, for example, classification label: traveling merchant. Reason: The store does not have a fixed place and sells goods by setting up stalls or walking around.

[0120] In step S525 , the multimodal large model is called.

[0121] In an exemplary embodiment, the annotation data generating device 200 calls a multimodal large model to recognize the image content of the image to be annotated based on the prompt word and the input image to be annotated.

[0122] In step S530 , the classification result and image description are output.

[0123] In this example embodiment, the annotated data generating device 200 uses a multimodal large model to identify the image content of the image to be annotated and then outputs an image description of the merchant's business environment image and the merchant type. For example, the output may be the following: "Classification label: Showing a person selling goods on a moving vehicle, which meets the characteristics of a traveling merchant because the scene indicates that the person does not have a fixed sales location." "Traveling merchant" is the first classification result, and "Showing a person selling goods on a moving vehicle, which meets the characteristics of a traveling merchant because the scene indicates that the person does not have a fixed sales location" is the image description.

[0124] In step S535 , prompt words of the large language model are configured.

[0125] In an example embodiment, the prompt words of the large language model include an image classification definition, an image description, a classification instruction, and an instruction for generating a confidence score, wherein the image classification definition and the classification instruction are consistent with the image classification definition and the classification instruction in the prompt words of the multimodal large model, and the image description is an image description output by the multimodal large model.

[0126] In response to prompt word configuration on the model configuration page, the annotation data generation device 200 configures prompt words for the large language model. For example, in the case of an image to be annotated of a business environment, the prompt words for the multimodal large model include a definition of the business, instructions for classifying the business, a description of the business environment image, and instructions for generating a confidence score.

[0127] In step S540 , the large language model is called.

[0128] In an exemplary embodiment, the annotation data generating device 200 inputs the image description, image classification definition, classification instructions, and confidence score generation instructions of the image to be annotated into a large language model, recognizes the image description of the image to be annotated based on the large language model, and generates a classification result and a confidence score for the image to be annotated.

[0129] For example, the image description generated by the multimodal large model provides the definition of a business and requires the model to classify it and calculate a confidence score (0-100). The annotation data generation device 200 inputs prompt words to the large language model, including the definition of the business, the image description, instructions for classifying the business, and instructions for generating a confidence score. The large language model is then called to output the annotation data: business, confidence score 100.

[0130] In step S545 , the classification result and the confidence score are output.

[0131] In an exemplary embodiment, the labeled data generating device 200 outputs the classification results and confidence score of the large language model. The confidence score is a number that measures the credibility of the prediction result or model output, and is used to indicate the "degree of certainty" or "reliability" of the model for a certain prediction. In many cases, it is a number between 0 and 100, and the closer it is to 100, the more confident the model is that its prediction is accurate. This score can help users better understand the reliability of the model's predictions and make more reasonable decisions in practical applications. For example, the output example given by the large language model is: "Trade merchant. Confidence score: 100."

[0132] In step S550, an image to be annotated is obtained.

[0133] In step S555, the image classification model to be optimized is called.

[0134] In an exemplary embodiment, the annotation data generation device 200 uses an image classification model to identify the image content of the image to be annotated and generates a classification result for the image to be annotated. For example, the image classification model includes a business classification model, which is a machine learning model to be optimized for classifying business environment images. The annotation data generation device 200 inputs a business image to the business classification model, which outputs the business type, such as a business.

[0135] In step S560, the classification result of the image classification model is output.

[0136] In step S565 , low confidence data is removed.

[0137] In an exemplary embodiment, the labeled data generating device 200 screens the plurality of initially labeled data based on confidence scores, removing initially labeled data with confidence scores below a predetermined threshold and removing data that the large language model determines as "unable to determine." For example, data that the large language model determines as "unable to determine" or with a confidence score below 100 is removed.

[0138] In step S570, multi-model voting is performed.

[0139] In an exemplary embodiment, the annotation data generating device 200 performs multi-model voting based on the initial annotation data corresponding to each machine learning model; and determines the target annotation data of the image to be annotated based on the voting results of the multi-model voting.

[0140] In step S575 , inconsistent data is removed.

[0141] In an exemplary embodiment, if the classification results of multiple machine learning models are consistent, the annotation data generating device 200 uses the consistent initial annotation data as the target annotation data. Figure 5 As shown in the figure, the classification results of the three machine learning models are voted on. All three classification results are "traders", and the final vote result is "traders", which is the target labeled data. The consistency of the classification results of multiple models generally indicates a higher confidence level, so this data is used as the final labeled data.

[0142] In step S580 , the labeled data is output.

[0143] In an exemplary embodiment, the annotation data generating device 200 outputs the determined target annotation data and saves the target annotation data and the corresponding image into a database.

[0144] according to Figure 5According to the technical solution in the example embodiment, after acquiring the image to be annotated, the annotation data generation device identifies the image content of the image to be annotated based on multiple pre-trained machine learning models. The multiple machine learning models include a multimodal recognition model, a language model, and an image recognition model. They identify the image to be annotated from different angles, determine multiple initial annotation data of the image to be annotated, and then perform multi-model voting based on the initial annotation data corresponding to each machine learning model to determine the target annotation data of the image to be annotated. By using multi-model voting to determine the target annotation data and combining the prediction results of multiple models to form the final prediction result, the robustness of the annotation data generation device can be enhanced and the overfitting of the training data by a single model can be reduced. At the same time, multi-model voting can combine the advantages of multiple models to provide more comprehensive and powerful prediction capabilities. Compared with manual data annotation, data annotation using machine learning models is more efficient. For example, to obtain 3,000 valid data, manual annotation takes about 30 hours. However, by using the data annotation generation method provided in the embodiment of this specification, the time taken for this process can be reduced to about 6 hours, and the overall annotation efficiency is increased by 4 times. Moreover, manual labeling may be affected by factors such as the labeler's subjective judgment, fatigue, and negligence, resulting in mislabeling, omissions, or poor consistency. Machine learning models can perform labeling tasks in a unified manner and reduce the deviation caused by human factors.

[0145] On the other hand, this specification provides a computer-readable non-transitory storage medium storing at least one instruction set for generating annotation data. When the at least one instruction set is executed by a processor, the at least one instruction set instructs the processor to implement the steps of the annotation data generation method P300 described in this specification. In some possible implementations, various aspects of this specification can also be implemented in the form of a program product, which includes program code. When the program product is run on the annotation data generation device 200, the program code is used to enable the annotation data generation device 200 to perform the steps of the annotation data generation method P300 described in this specification. The program product for implementing the above method can use a portable compact disk read-only memory (CD-ROM) to include program code and can be run on the annotation data generation device 200. However, the program product of this specification is not limited to this. In this specification, a readable storage medium can be any tangible medium containing or storing a program that can be used by an instruction execution system or used in combination with it. The program product can use any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media include: a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. The computer-readable storage medium may include a data signal transmitted in baseband as part of a carrier wave, which carries readable program code. This transmitted data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable storage medium may also be any readable medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof. Program code for performing the operations of the present specification may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, and conventional procedural programming languages such as C or similar programming languages. The program code may be executed entirely on the annotated data generating device 200, partially on the annotated data generating device 200, as a stand-alone software package, partially on the annotated data generating device 200 and partially on a remote computing device, or entirely on a remote computing device.

[0146] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0147] In summary, after reading this detailed disclosure, those skilled in the art will appreciate that the foregoing detailed disclosure may be presented by way of example only and may not be limiting. Although not expressly stated herein, those skilled in the art will understand that this specification encompasses various reasonable changes, improvements, and modifications to the embodiments. Such changes, improvements, and modifications are intended to be suggested by this specification and are within the spirit and scope of the exemplary embodiments of this specification.

[0148] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, “one embodiment,” “an embodiment,” and / or “some embodiments” mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is emphasized and should be understood that two or more references to “an embodiment,” “one embodiment,” or “an alternative embodiment” in various parts of this specification do not necessarily refer to the same embodiment. Furthermore, particular features, structures, or characteristics may be appropriately combined in one or more embodiments of this specification.

[0149] It should be understood that in the foregoing descriptions of the embodiments of this specification, to facilitate understanding of a feature and to simplify this specification, various features are combined in a single embodiment, figure, or description thereof. However, this does not necessarily mean that these features are combined. When reading this specification, a person skilled in the art may label some of the devices as separate embodiments. In other words, the embodiments of this specification can also be understood as the integration of multiple sub-embodiments. The content of each sub-embodiment is also valid even when it includes fewer than all the features of a single previously disclosed embodiment.

[0150] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, articles, and the like, cited herein, except to the extent that it is inconsistent or conflicting with this document or that it has a limiting effect on the broadest scope of the claims, is hereby incorporated by reference for all purposes now or hereafter connected with this document. In addition, in the event of any inconsistency or conflict between the description, definition, and / or use of a term in any material and the description, definition, and / or use of a term in this document, the term in this document shall control.

[0151] Finally, it should be understood that the embodiments of the application disclosed herein are illustrative of the principles of the embodiments of this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can adopt alternative configurations based on the embodiments in this specification to implement the application in this specification. Therefore, the embodiments of this specification are not limited to the embodiments precisely described in the application.

Claims

1. A method for generating labeled data, comprising: Get the image to be annotated; Recognizing the image content of the image to be annotated based on multiple pre-trained machine learning models to determine multiple pieces of initial annotation data for the image to be annotated; as well as Based on the initial annotation data corresponding to each of the machine learning models, target annotation data of the image to be annotated is determined.

2. The method according to claim 1, wherein The multiple machine learning models include a multimodal recognition model and an image recognition model. The multiple pre-trained machine learning models are used to recognize the image content of the image to be annotated and determine multiple pieces of initial annotation data for the image to be annotated, including: Recognizing the image content of the image to be annotated based on the multimodal recognition model to generate first initial annotation data for the image to be annotated; and The image content of the image to be annotated is recognized based on the image recognition model to generate second initial annotation data for the image to be annotated.

3. The method according to claim 2, wherein: The first initial annotation data includes an image description, the multiple machine learning models also include a language model, and the method further includes: inputting the image description output by the multimodal recognition model into the language model; The image description is recognized based on the language model to generate third initial annotation data of the image to be annotated.

4. The method according to claim 3, wherein: The multimodal recognition model is a large multimodal model, and the prompt words of the large multimodal model include image classification definition, classification instructions and image description instructions; the language model is a large language model, and the prompt words of the large language model include the image classification definition, the image description and the classification instructions.

5. The method according to claim 4, wherein: The prompt word of the large language model also includes an instruction to generate a confidence score.

6. The method of claim 3, wherein: The third initial labeled data includes a confidence score corresponding to the classification result, and the method further includes: The plurality of initially labeled data are screened based on the confidence scores, and the initially labeled data having confidence scores less than a predetermined threshold are removed.

7. The method of claim 1, wherein: The determining, based on the initial annotation data corresponding to each of the machine learning models, target annotation data of the image to be annotated includes: Performing multi-model voting based on the initial labeled data corresponding to each of the machine learning models; and The target annotation data of the image to be annotated is determined based on the voting result of the multi-model voting.

8. The method of claim 7, wherein: The initial annotation data includes a classification result, and determining the target annotation data of the image to be annotated based on the voting result of the multi-model voting includes: If the classification results of the multiple machine learning models are consistent, the consistent initial labeled data will be used as the target labeled data.

9. The method according to any one of claims 2 to 8, wherein The image recognition model includes a business classification model, and the image to be labeled is an image of the business environment.

10. The method of claim 1, wherein: Before identifying the image content of the image to be labeled based on the pre-trained multiple machine learning models, the method further includes: In response to a model selection operation for a target task, determining a plurality of pre-trained machine learning models corresponding to the target task; In response to a parameter configuration operation for a target machine learning model, model parameters corresponding to the target machine learning model are determined.

11. A device for generating annotated data, comprising: At least one storage medium storing at least one instruction set for generating annotation data; as well as At least one processor is communicatively connected to the at least one storage medium, wherein when the device is running, the at least one processor reads the at least one instruction set and, according to an instruction of the at least one instruction set: Get the image to be annotated; Identify the image content of the image to be labeled based on multiple pre-trained machine learning models, Determining a plurality of initial annotation data for the image to be annotated; as well as Based on the initial annotation data corresponding to each of the machine learning models, target annotation data of the image to be annotated is determined.

12. A computer-readable non-volatile storage medium, wherein: The computer-readable non-volatile storage medium stores at least one instruction set, and when the at least one instruction set is executed by at least one processor, the annotation data generation method according to any one of claims 1 to 10 is implemented.

Images