A method, device and storage medium for automatic data labeling

Abstract

The present application discloses a method, device and storage medium for automatic data labeling, which relates to the field of data processing technology and is used for automatically labeling data, while improving labeling efficiency and labeling quality. The present application method: manually pre-label the data of each task to obtain pre-labeled data; automatically derive multiple different prompt word schemes for each task; according to the prompt word scheme, import all tasks into a multimodal large model to conduct independent questioning, and obtain the first output result corresponding to each prompt word scheme; use an unsupervised voting algorithm to extract the results and generate voting statistics for each task; if the number of positively correlated tags reaches or exceeds the election threshold, the task is determined to be a positively correlated sample; the pre-labeled data and the positively correlated samples are input into a single-modal small model for training iteration; use the single-modal small model to automatically label the target data to be labeled in batches, and output the labeling results.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device and storage medium for automatic data labeling. Background Art

[0002] A large model refers to a machine learning model with large-scale parameters and complex computing architecture, and has powerful natural language processing (NLP) and cross-modal data processing capabilities. Large models can not only generate fluent and coherent text and effectively answer natural language questions, but also have the ability to understand and generate various types of content such as images, audio and video. Large models usually rely on massive amounts of labeled data for training, and given that the data processed by large models often comes from different channels and is of various types, effective labeling and structured processing of data has become the key to improving model performance.

[0003] Although there are some tools and methods that support multimodal data annotation, their scope of application and effectiveness are still significantly limited. Specifically, most of these existing solutions are more suitable for small-scale data processing scenarios, and can provide a certain degree of convenience and efficiency when the amount of data is relatively small. However, when faced with large-scale data processing, the existing tools and methods have the problem of low annotation efficiency. Due to the surge in data volume, traditional annotation methods often find it difficult to complete the annotation of all data within a reasonable time. Data noise and subjective factors of annotation personnel will also affect the quality of annotation. In large-scale data sets, the quality of data is often uneven and contains a large amount of noisy data. These noisy data not only increase the difficulty of annotation, but also may lead to a decrease in the accuracy of the annotation results. And because annotation work usually requires manual participation, the expertise, experience level and personal preferences of the annotation personnel will affect the annotation quality to a certain extent. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a method, device and storage medium for automatic data labeling.

[0005] The technical solution provided in this application is described below:

[0006] The first aspect of the present application provides a method for automatic data labeling, the method comprising:

[0007] In the scenario of multi-task input, the data in each task is manually pre-labeled to obtain pre-labeled data;

[0008] Automatically derive multiple different prompt word schemes for each task, each prompt word scheme is specifically associated with a certain type of data;

[0009] According to the prompt word scheme, all tasks are imported into the multimodal large model to conduct independent questions respectively, and the first output result corresponding to each prompt word scheme is obtained;

[0010] For the first output result, an unsupervised voting algorithm is used to extract the result and generate the voting statistics result of each task;

[0011] If the voting statistics contain a positive correlation mark, the election mechanism is initiated;

[0012] Comparing the election threshold preset under the election mechanism with the number of positive correlation marks in the voting statistics result;

[0013] If the number of positive correlation marks reaches or exceeds the selection threshold, the task is determined to be a positive correlation sample;

[0014] Inputting the pre-labeled data and the positively correlated samples into a unimodal small model for training iteration to obtain an optimized unimodal small model;

[0015] Use the single-modal small model to automatically label the target data to be labeled in batches, and output the labeling results;

[0016] The labeling result is quality checked to obtain a final labeling result.

[0017] Optionally, according to the prompt word scheme, all tasks are imported into the multimodal large model for independent questioning, and the first output result corresponding to each prompt word scheme is obtained, including:

[0018] Based on the step-by-step prompting method, each prompt word scheme is broken down into multiple steps;

[0019] According to the order of steps, use the multimodal large model to ask questions about the tasks one by one and obtain the second output result corresponding to each step;

[0020] The second output results of each prompt word solution are counted and summarized to obtain the first output results corresponding to each prompt word solution.

[0021] Optionally, the counting and summarizing of the second output results of each prompt word scheme to obtain the first output results corresponding to each prompt word scheme includes:

[0022] Use a multimodal big model to set corresponding expected results for each step of the question;

[0023] Counting and summarizing the second output results of each prompt word scheme to obtain a summary result of each prompt word scheme;

[0024] Determine whether the second output result in the summary result is consistent with the corresponding expected result;

[0025] According to the judgment result, the summary result is attached with a relevant mark and a first output result corresponding to each prompt word scheme is generated, wherein the relevant mark includes a positive relevant mark and a negative relevant mark.

[0026] Optionally, for the first output result, an unsupervised voting algorithm is used to extract the result and generate voting statistics for each task, including:

[0027] Obtain the first output result of all prompt word solutions in each task;

[0028] Determine whether there is a positive correlation mark in each prompt word scheme, and calculate the voting statistics of the task;

[0029] The voting statistics are calculated using the following formula:

[0030]

[0031] Among them, FinalResult represents the voting statistics result of the task, and Y[i] represents the output result of the i-th prompt word solution.

[0032] Optionally, before comparing the election threshold preset under the election mechanism with the number of positive correlation marks in the voting statistics result, the method further includes:

[0033] Obtaining the number of positive correlation marks in the voting statistics result;

[0034] Initialize an election mechanism and set an election threshold in the election mechanism.

[0035] Optionally, the step of inputting the pre-labeled data and the positively correlated samples into a small unimodal model for training iteration to obtain an optimized small unimodal model includes:

[0036] Integrate the pre-labeled data and the positively correlated samples into a training data set, and preprocess the training data set;

[0037] Initializing the unimodal small model, wherein the initialization is to set the weight and bias items of the unimodal small model;

[0038] Inputting the preprocessed training data set into the single-modal small model, and generating prediction results of positively correlated samples by layer-by-layer calculation;

[0039] Compare the prediction result with the pre-labeled data using a loss function to obtain a loss function value;

[0040] According to the gradient information of the loss function, the gradient of each parameter in the single-modal small model is calculated by a back-propagation algorithm;

[0041] Based on the gradient, updating the weights and biases of the unimodal small model using an optimization algorithm;

[0042] Determine whether the loss function value converges;

[0043] When it is determined that the loss function value converges, the current parameters are saved to obtain an optimized single-mode small model.

[0044] Optionally, the step of automatically labeling the target data to be labeled in batches using the single-modal small model and outputting the labeling results includes:

[0045] Using the single-modal small model to extract features from the target data to be labeled, to obtain feature information;

[0046] According to the feature information, the single-modal small model automatically generates corresponding annotation information for the target data to be annotated, and outputs the annotation result.

[0047] A second aspect of the present application provides a device for automatic data labeling, the device comprising:

[0048] A pre-labeling unit is used to manually pre-label the data in each task in a multi-task input scenario to obtain pre-labeled data;

[0049] A processing unit, configured to automatically derive a plurality of different prompt word schemes for each task, each prompt word scheme being specifically associated with a certain type of data;

[0050] A first acquisition unit is used to import all tasks into the multimodal large model according to the prompt word scheme to perform independent questions respectively, and obtain a first output result corresponding to each prompt word scheme;

[0051] An extraction unit, configured to extract results using an unsupervised voting algorithm for the first output result, and generate voting statistics results for each task;

[0052] A judgment unit, configured to start an election mechanism if the voting statistics result contains a positive correlation mark;

[0053] A comparison unit, used to compare the election threshold preset under the election mechanism with the number of positive correlation marks in the voting statistics result;

[0054] A determination unit, configured to determine that the task is a positively correlated sample if the number of positively correlated marks reaches or exceeds the selection threshold;

[0055] A training unit, used for inputting the pre-labeled data and the positively correlated samples into a unimodal small model for training iteration to obtain an optimized unimodal small model;

[0056] A labeling unit, used to automatically label the target data to be labeled in batches using the single-modal small model, and output the labeling results;

[0057] The quality inspection unit is used to perform quality inspection on the labeling result to obtain the final labeling result.

[0058] Optionally, the device further comprises:

[0059] A second acquisition unit is used to acquire the number of positive correlation marks in the voting statistical result;

[0060] Initialize the unit, initialize the election mechanism, and set an election threshold in the election mechanism.

[0061] Specifically, the first acquiring unit includes:

[0062] Based on the step-by-step prompting method, each prompt word scheme is broken down into multiple steps;

[0063] According to the order of steps, use the multimodal large model to ask questions about the tasks one by one and obtain the second output result corresponding to each step;

[0064] The second output results of each prompt word solution are counted and summarized to obtain the first output results corresponding to each prompt word solution.

[0065] Specifically, the extraction unit includes:

[0066] Obtain the first output result of all prompt word solutions in each task;

[0067] Determine whether there is a positive correlation tag in each prompt word solution, and calculate the voting statistics of the task.

[0068] Specifically, the training unit includes:

[0069] Integrate the pre-labeled data and the positively correlated samples into a training data set, and preprocess the training data set;

[0070] Initializing the unimodal small model, wherein the initialization is to set the weight and bias items of the unimodal small model;

[0071] Inputting the preprocessed training data set into the single-modal small model, and generating prediction results of positively correlated samples by layer-by-layer calculation;

[0072] Compare the prediction result with the pre-labeled data using a loss function to obtain a loss function value;

[0073] According to the gradient information of the loss function, the gradient of each parameter in the single-modal small model is calculated by a back-propagation algorithm;

[0074] Based on the gradient, updating the weights and biases of the unimodal small model using an optimization algorithm;

[0075] Determine whether the loss function value converges;

[0076] When it is determined that the loss function value converges, the current parameters are saved to obtain an optimized single-mode small model.

[0077] Specifically, the marking unit includes:

[0078] Using the single-modal small model to extract features from the target data to be labeled, to obtain feature information;

[0079] According to the feature information, the single-modal small model automatically generates corresponding annotation information for the target data to be annotated, and outputs the annotation result.

[0080] The third aspect of the present application also provides a device for automatic data labeling, the device comprising:

[0081] Processor, memory, input-output unit, and bus;

[0082] The processor is connected to the memory, the input and output unit, and the bus;

[0083] The memory stores a program, and the processor calls the program to execute the first aspect and any optional method in the first aspect.

[0084] A fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the program executes the first aspect and any optional method in the first aspect.

[0085] It can be seen from the above technical solutions that this application has the following advantages:

[0086] By pre-performing manual annotation in multi-task input scenarios and then using a large multimodal model to automatically test prompt word schemes and extract results, the time required for direct manual annotation is reduced, thereby speeding up the annotation process and improving overall annotation efficiency.

[0087] Multiple prompt word solutions are automatically derived for each task, and questions are asked independently in the large model, which can cover more possible data types and scenarios. By setting the election mechanism and election threshold, highly consistent and reliable positive correlation samples can be screened out. Positive correlation samples are used to train the single-modal small model, ensuring the accuracy and representativeness of the model training data, thereby improving the accuracy of the final annotation results.

[0088] By continuously inputting new pre-labeled data and positively correlated samples into the unimodal small model for training iterations, it can gradually adapt to the processing requirements of specific data types, thereby optimizing the unimodal small model to improve the performance and generalization ability of the model, and then improve the accuracy of data labeling, so that the unimodal small model can label different types of data and tasks more accurately, and the accuracy and efficiency of labeling will also be further improved.

[0089] Quality inspection of the annotation results ensures that the final output annotation results are not only highly automated but also quality-controlled, meeting the high standards for data annotation in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] In order to more clearly illustrate the technical solution in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0091] Figure 1 A schematic diagram of an embodiment of the method for automatic data annotation in this application;

[0092] Figure 2 A schematic diagram of an embodiment of a method for obtaining a first output result in the present application;

[0093] Figure 3 A schematic diagram of another embodiment of the method for obtaining the first output result in the present application;

[0094] Figure 4 A schematic diagram of an embodiment of a method for generating voting statistics results in this application;

[0095] Figure 5 A schematic diagram of an embodiment of a method for pre-configuring an election mechanism in the present application;

[0096] Figure 6 A schematic diagram of an embodiment of a method for obtaining an optimized single-mode small model in the present application;

[0097] Figure 7 A schematic diagram of an embodiment of a method for labeling data in the present application;

[0098] Figure 8 A schematic diagram of the structure of the device for automatic data annotation in this application;

[0099] Fig. 9 Another structural schematic diagram of the device for automatic data annotation in this application. DETAILED DESCRIPTION

[0100] It should be noted that the method for automatic data annotation provided in this application can be applied to a terminal, a system, or a server. For example, the terminal can be a smart phone or a computer, a tablet computer, a smart TV, a smart watch, a portable computer terminal, or a fixed terminal such as a desktop computer. For the convenience of explanation, this application uses the terminal as an example for explanation.

[0101] See also Figure 1 The present application first provides an embodiment of a method for automatic data annotation, which includes:

[0102] S101. In a multi-task input scenario, manually pre-label the data in each task to obtain pre-labeled data;

[0103] First, the system will manually pre-label the data in the task. Pre-labeling means selecting two to three thousand representative images or text data for manual pre-labeling. The labeling content includes but is not limited to the category, location (such as bounding box), attributes, etc. of the object.

[0104] S102, automatically deriving a plurality of different prompt word schemes for each task, each prompt word scheme being specifically associated with a certain type of data;

[0105] The system specifies the specific definition and goal of each task. For example, the sentiment analysis task is to determine the sentiment tendency of the text, and the entity recognition task is to identify specific entities in the text. Next, the system analyzes the type of data involved in each task. Sentiment analysis may involve text data, while entity recognition may involve text and image data (such as extracting text from an image and identifying entities).

[0106] In order to guide the multimodal large model to understand the task requirements from different angles or ways, the system automatically generates multiple prompt word schemes. These prompt word schemes cover multiple aspects of the task. Each prompt word scheme is associated with a certain type of data or task requirements. For example, for sentiment analysis tasks, one prompt word scheme may focus on the recognition of sentiment words, while another scheme may focus on the judgment of the sentiment tendency of the overall context.

[0107] S103, according to each prompt word scheme, import all tasks into the multimodal large model to conduct independent questioning respectively, and obtain the first output result corresponding to each prompt word scheme;

[0108] The ‌Multimodal Big Model‌ is a machine learning model that can process multiple types of data, mainly including modalities such as text, image, audio and video. The ‌Multimodal Big Model‌ used in the present invention has been fully trained on large-scale text and image data, can fully understand the object information in the real world, and can accurately detect most common categories of objects in the real world. The ‌Multimodal Big Model includes three main parts: 1) Image encoder: The image encoder divides the input image into multiple small blocks and encodes these visual small blocks into high-dimensional features. 2) Prompt encoder: Encodes various forms of prompts (points, boxes, text, etc.) provided by the user into vectors. Points and boxes use position encoding plus embedding for each prompt type, and free-form text is encoded using a text encoder. 3) Detection decoder: Combines image features and prompt information to generate detection box information for the target object.

[0109] The system inputs each task and its corresponding prompt word scheme X1, X2, ..., Xn into the multimodal large model, which processes each task independently according to the prompt word scheme and outputs the corresponding first output result Y1, Y2, ..., Yn. The first output result includes a classification label, a score, or other forms of output, depending on the task type and the design of the prompt word scheme.

[0110] S104: for the first output result, an unsupervised voting algorithm is used to extract the result and generate the voting statistics result of each task;

[0111] An unsupervised voting algorithm is a method of processing data using a voting mechanism in an unsupervised learning framework. In unsupervised learning, the data itself has no labels, and the algorithm needs to perform clustering, classification, or other forms of pattern recognition based on the characteristics of the data itself. The voting mechanism can be used to make decisions or choices in these unlabeled data. Common unsupervised voting algorithms include majority voting, average voting, etc. These algorithms can generate voting statistics based on the relevance of the output results.

[0112] The system collects the first output results of each task under all prompt word schemes, and summarizes and counts the first output results to generate voting statistics for each task. The voting statistics include summary information of the first output results of all prompt word schemes under each task.

[0113] Specifically, the voting algorithm counts and votes on the first output results to determine whether the first output results of all prompt word solutions under each task contain a positive correlation mark. If at least one output result contains a positive correlation mark, the voting statistical result is determined to be 1.

[0114] S105. If the voting statistics result contains a positive correlation mark, the election mechanism is started;

[0115] The election mechanism is a decision-making process used to select the final task that best meets the requirements of training samples from multiple tasks.

[0116] According to the statistics and voting of the unsupervised voting algorithm in the aforementioned step S104, the system has determined which tasks have positive correlation marks in the voting statistics results.

[0117] If there is a positive correlation mark in the voting statistics, the system starts the election mechanism and inputs the voting statistics into the election mechanism.

[0118] S106, comparing the election threshold preset under the election mechanism with the number of positive correlation marks in the voting statistics result;

[0119] The system sets an election threshold K (K is greater than or equal to 1) under the election mechanism to determine whether the number of positive correlation marks is large enough to support the task to be judged as a positive correlation sample in the subsequent process.

[0120] According to the voting statistics, an election mechanism is used to compare the election threshold with the number N of positive correlation marks in the voting statistics.

[0121] S107, if the number of positive correlation marks reaches or exceeds the election threshold, the task is determined to be a positive correlation sample;

[0122] After comparing the preset election threshold under the election mechanism with the number of positive correlation marks in the voting statistics, if the number of positive correlation marks reaches or exceeds the election threshold, the system determines that the task is a positive correlation sample.

[0123] S108, inputting the pre-labeled data and the positively correlated samples into the unimodal small model for training iteration to obtain an optimized unimodal small model;

[0124] A unimodal model is a model that specializes in processing a single type of data. It is usually optimized for a certain data modality to achieve the best performance on a specific task. For example, a text unimodal model mainly processes text data, an image unimodal model mainly processes image data, and an audio unimodal model mainly processes audio data.

[0125] The pre-labeling module first inputs the unimodal small model for training so that the unimodal small model can better understand the scene information of the data that needs to be labeled. The system collects all tasks that are judged as positively correlated samples and inputs them into the unimodal small model for training iteration again, and uses the prompt word scheme in the positively correlated samples as an instruction for the unimodal small model to understand the requirements of the labeling task. After training iterations, the original unimodal small model labeling efficiency and accuracy are improved, and an optimized unimodal small model is obtained.

[0126] S109, using a single-modal small model to automatically label the target data to be labeled in batches, and outputting the labeling results;

[0127] The system uses the optimized unimodal model to automatically annotate the target data to be annotated. The unimodal model outputs the corresponding annotation results according to the characteristics of the target data. The output annotation results are sorted, including associating the annotation information (such as category, location, attributes, etc.) with the original data to form a complete annotation data set. Users can choose to convert the annotation results into the required format (such as JSON, XML, etc.) according to actual needs for subsequent processing and analysis.

[0128] S110: Perform quality inspection on the labeling result to obtain the final labeling result.

[0129] After obtaining the annotation results, the annotation results need to be quality checked. The quality check of the annotation results includes automatic quality check and manual review. Automatic quality check uses preset quality check rules (such as the overlap of the annotation box, the clarity of the image, the degree of occlusion, the size of the annotation box, etc.) to automatically check the annotation results and screen out possible errors or non-compliance. Manual review manually reviews the suspected errors found in the automatic quality check to ensure the accuracy of the annotation results. At the same time, some data can also be randomly selected for manual inspection to evaluate the overall annotation quality. After the quality inspection, the final annotation results are obtained.

[0130] In this embodiment, by pre-performing manual annotation in a multi-task input scenario and then using a multimodal large model to perform automated prompt word scheme testing and result extraction, the time for direct manual annotation participation is reduced, thereby speeding up the annotation process and improving the overall annotation efficiency.

[0131] Multiple prompt word solutions are automatically derived for each task, and questions are asked independently in the large model, which can cover more possible data types and scenarios. By setting the election mechanism and election threshold, highly consistent and reliable positive correlation samples can be screened out. Positive correlation samples are used to train the single-modal small model, ensuring the accuracy and representativeness of the model training data, thereby improving the accuracy of the final annotation results.

[0132] By continuously inputting new pre-labeled data and positively correlated samples into the unimodal small model for training iterations, it can gradually adapt to the processing requirements of specific data types, thereby optimizing the unimodal small model to improve the performance and generalization ability of the model, and then improve the accuracy of data labeling, so that the unimodal small model can label different types of data and tasks more accurately, and the accuracy and efficiency of labeling will also be further improved.

[0133] Quality inspection of the annotation results ensures that the final output annotation results are not only highly automated but also quality-controlled, meeting the high standards for data annotation in practical applications.

[0134] See also Figure 2 , the present application provides an embodiment of a method for obtaining a first output result, the embodiment comprising:

[0135] S201, based on the step-by-step prompting method, decomposing each prompting word scheme into multiple steps;

[0136] Based on the distributed prompting method, each prompt word scheme is further broken down into a series of specific and actionable steps. Specifically, according to the complexity of the prompt word scheme and the type of task, it is broken down into several logically interrelated sub-steps.

[0137] S202, according to the order of steps, using the multimodal large model to ask questions about the tasks one by one, and obtain the second output result corresponding to each step;

[0138] According to the order of steps decomposed in step S201, the tasks and their corresponding steps are input into the multimodal large model one by one. The multimodal large model will independently process the tasks according to the input steps and output the corresponding second output result.

[0139] Specifically, for each step, we need to input the task data, prompt words, and specific requirements of the step into the multimodal large model. The multimodal large model processes the task based on this information and generates a second output result corresponding to the step.

[0140] S203 , counting and summarizing the second output results of each prompt word solution to obtain the first output results corresponding to each prompt word solution.

[0141] The system counts and summarizes all the second output results generated in step S202 to obtain the first output result corresponding to each prompt word solution.

[0142] Specifically, the second output results of each step are summarized and organized according to the task type and the design requirements of the prompt word scheme. For example, if the task is to classify objects in an image, then we can summarize the classification results of each step.

[0143] In this embodiment, by breaking down the prompt word scheme into multiple steps, the processing flow and requirements of the task can be more clearly understood. The multimodal large model has powerful processing capabilities and generalization performance, can automatically adapt to the input requirements of different steps, and generate corresponding first output results, thereby improving the accuracy and efficiency of the prompt word scheme processing.

[0144] See also Figure 3 , the present application provides another embodiment of a method for obtaining a first output result, the embodiment comprising:

[0145] S301. Use the multimodal large model to set corresponding expected results for each step of the question;

[0146] According to the specific steps under each prompt word scheme, the multimodal model sets corresponding expected results for each step. These expected results are the predictions of the multimodal model on the output results of each step.

[0147] Specifically, the multimodal big model carefully analyzes the questions asked at each step to clarify the type of information or task objectives obtained at that step. Then, based on these information types or task objectives, reasonable expected results are set.

[0148] S302, counting and summarizing the second output results of each prompt word scheme to obtain a summary result of each prompt word scheme;

[0149] Specifically, the second output result of each step in each prompt word scheme is summarized in an independent summary table or data set to obtain the summary result of each prompt word scheme.

[0150] S303, determining whether there is a second output result in the summary result that is consistent with the corresponding expected result;

[0151] The second output result in the summary result of each prompt word solution is compared with the corresponding expected result one by one by using a comparison algorithm or manual review.

[0152] S304 , according to the judgment result, the summary result is attached with a relevant mark and a first output result corresponding to each prompt word scheme is generated, wherein the relevant mark includes a positive relevant mark and a negative relevant mark.

[0153] A new data set or report is created for the summary results of each cue word solution, i.e., the first output result, and the relevant labeling information is attached thereto.

[0154] If the second output result in a certain summary result is consistent with the corresponding expected result, a positive correlation mark is attached to the summary result, indicating that the step or the prompt word scheme has achieved the expected goal. If there is a difference or inconsistency, a negative correlation mark is attached to the summary result, indicating that the step or the prompt word scheme has not achieved the expected goal.

[0155] After the results are summarized and the relevant tags are attached, the first output results corresponding to each prompt word solution are generated.

[0156] In this embodiment, setting the expected result not only helps to evaluate the performance of the multimodal large model, but also serves as a standard for marking relevant tags. By judging whether the second output result is consistent with the corresponding expected result, the correlation between the two can be determined, and relevant tags can be attached. The relevant tags provide important data support for subsequent tags.

[0157] See also Figure 4 The present application provides an embodiment of a method for generating voting statistics results, the embodiment comprising:

[0158] S401, obtaining the first output result of all prompt word solutions in each task;

[0159] The system will first traverse all tasks and collect the first output results under all prompt word solutions for each task.

[0160] S402, determining whether there is a positive correlation mark in each prompt word solution, and calculating the voting statistics of the task;

[0161] Use the following formula 1 to calculate the voting statistics:

[0162]

[0163] Among them, FinalResult represents the voting statistics result of the task, and Y[i] represents the output result of the i-th prompt word solution.

[0164] The system further analyzes the first output results of all prompt word solutions for each task. First, the system determines whether there is a positive correlation marker in the first output result of each prompt word solution. The positive correlation marker may be a specific label, numerical range, or text pattern, depending on the definition of the task and the output requirements.

[0165] Once the existence of positively correlated tags is determined, the system can calculate the voting statistics for each task using the given formula 1.

[0166] In this embodiment, by collecting the first output results of each task under all prompt word schemes, the integrity and comprehensiveness of the data are ensured, providing a reliable basis for subsequent voting statistics. By determining the positive correlation mark and calculating the voting statistics, the system can more accurately reflect the comprehensive performance of each task under all prompt word schemes, improving the accuracy and reliability of the results. By clarifying the process of determining the positive correlation mark and calculating the voting statistics, the system can provide clearer voting statistics.

[0167] See also Figure 5 , the present application provides an embodiment of a method for pre-configuring an election mechanism, the embodiment comprising:

[0168] S501, obtaining the number of positive correlation marks in the voting statistics result;

[0169] The voting statistics are usually a summary of the comprehensive results of multiple prompt word schemes under the same task, which includes the first output result of each prompt word scheme and its related mark with the expected result (such as positive correlation mark or negative correlation mark).

[0170] The system needs to traverse each first output result in the voting statistics, identify and count the number of results with positive correlation marks.

[0171] S502: Initialize the election mechanism and set an election threshold in the election mechanism.

[0172] The system needs to initialize an election mechanism, which is used to select the best solution or the solution that meets specific conditions from multiple prompt word solutions. While initializing the election mechanism, the system also needs to set an election threshold in the election mechanism. The election threshold is a threshold value used to screen candidate solutions. Only solutions that meet or exceed the election threshold are eligible to be selected as the final solution. The setting of the election threshold needs to be comprehensively considered based on factors such as the requirements of the specific task, the performance of the model, and the number and quality of candidate solutions.

[0173] In this embodiment, the number of positively correlated tags is counted to obtain the number of positively correlated tags in each prompt word solution, and the number of positively correlated tags can be used as a decision basis for the final selected task.

[0174] By initializing the election mechanism and setting the election threshold, the system can select the best solution or the solution that meets specific conditions from one or more prompt word solutions, thereby improving the performance of the model. The election mechanism can automatically handle the screening and sorting process of candidate solutions, thereby saving manpower and time costs and improving efficiency. The election mechanism and election threshold can be flexibly adjusted and optimized according to the requirements of specific tasks to adapt to different application scenarios and needs.

[0175] See also Figure 6 The present application provides an embodiment of a method for obtaining an optimized single-mode small model, the embodiment comprising:

[0176] S601, integrating the pre-labeled data and the positively correlated samples into a training data set, and preprocessing the training data set;

[0177] The system first collects all tasks that are judged to be positively correlated samples, and integrates the pre-labeled data and positively correlated samples into a training data set. The training data set is then preprocessed. The preprocessing stage may include steps such as data cleaning, normalization, and feature extraction.

[0178] S602, initializing the single-modal small model, where the initialization is to set the weight and bias item of the single-modal small model;

[0179] Initialize the unimodal small model. This usually includes setting the model's weights and biases to random values ​​or predefined initial values. Initialization is an important step in the training of unimodal small models, which determines the state of the unimodal small model at the beginning of training.

[0180] S603, inputting the preprocessed training data set into the single-modal small model, and generating the prediction results of the positively correlated samples by layer-by-layer calculation;

[0181] After initialization, the unimodal small model uses its weights and biases to calculate the preprocessed training data set. Specifically, according to the forward propagation algorithm, the unimodal small model generates the final prediction results of the positively correlated samples by layer-by-layer calculation.

[0182] S604, using a loss function to compare the prediction result with the pre-labeled data to obtain a loss function value;

[0183] After generating the prediction result, the unimodal model uses the loss function to evaluate the difference between the prediction result and the pre-labeled data, and digitizes the difference into the loss function value. The smaller the loss function value, the closer the prediction result of the unimodal model is to the true label; conversely, the larger the loss function value, the more the prediction result of the unimodal model deviates from the true label.

[0184] S605, calculating the gradient of each parameter in the single-mode small model through a back propagation algorithm according to the gradient information of the loss function;

[0185] In order to minimize the loss function value, the model needs to calculate the gradient of the loss function with respect to the model parameters. That is, according to the back-propagation algorithm, the chain rule is used to calculate from the output layer to the input layer layer by layer to obtain the gradient of each parameter.

[0186] S606, according to the gradient, using the optimization algorithm to update the weight and bias of the single-modal small model;

[0187] After obtaining the gradient, the model uses an optimization algorithm (such as stochastic gradient descent, Adam, etc.) to update the weights and bias terms of the single-modal small model. The optimization algorithm adjusts the parameter value based on the gradient information to minimize the loss function value.

[0188] S607, judging whether the loss function value converges;

[0189] After each parameter update, the single-mode small model will recalculate the loss function value and determine whether it has converged. Convergence usually means that the loss function value has stabilized in a smaller range and no longer has a clear downward trend.

[0190] S608: If the loss function value converges, save the current parameters to obtain an optimized single-mode small model.

[0191] When the loss function value converges, the unimodal model will save the current parameter values ​​and generate an optimized unimodal model. The unimodal model has been fully trained and has high classification accuracy and generalization ability.

[0192] In this embodiment, the training data set is preprocessed to ensure the quality and consistency of the input tasks. According to the gradient information, the gradient of the parameters of the unimodal small model is calculated and updated, and the loss function value can be minimized by updating the parameters (i.e., weights and biases) in the direction of gradient descent. By continuously updating the weights and bias terms, the unimodal small model can gradually improve its performance and improve the classification accuracy. Judging whether the loss function value converges can determine whether the training of the unimodal small model is completed; only when the loss function value converges can it be confirmed that the unimodal small model has been trained. The optimized unimodal small model has high classification accuracy and generalization ability, and can be applied to the annotation of various practical tasks.

[0193] See also Figure 7 , the present application provides an embodiment of a method for labeling data, the embodiment comprising:

[0194] S701, using a single-modal small model to extract features from the target data to be labeled, to obtain feature information;

[0195] Feature information refers to information that can represent the essence or key attributes of data. During the feature extraction process, the unimodal small model analyzes various attributes of the target data, such as vocabulary and grammatical structure in text, color, shape, texture in images, etc., and then extracts the most representative features from them.

[0196] S702: Based on the feature information, the single-modal small model automatically generates corresponding annotation information for the target data to be annotated, and outputs the annotation result.

[0197] After extracting the feature information, the unimodal model will use this feature information to automatically generate corresponding annotation information for the target data to be annotated. In the process of generating annotation information, the unimodal model will use machine learning or deep learning algorithms based on the feature information to predict the label of the target data. This prediction process may be based on classification algorithms, such as decision trees, support vector machines, etc.; it may also be based on regression algorithms, such as linear regression, neural networks, etc.

[0198] The single-modal small model organizes the output annotation results, including associating the annotation information (such as category, location, attributes, etc.) with the original image to form a complete annotation data set. Users can choose to convert the annotation results into the required format (such as JSON, XML, etc.) according to actual needs for subsequent processing and analysis.

[0199] In this embodiment, through feature extraction, complex target data can be converted into concise and clear feature information. By using feature information, the single-modal small model can more accurately predict the labeling of target data, thereby improving the accuracy of labeling. Automatic labeling can greatly reduce the workload of manual labeling, saving manpower and time costs.

[0200] See also Figure 8 , the present application provides a device for automatic data labeling, comprising:

[0201] A pre-labeling unit 801 is used to manually pre-label the data in each task in a multi-task input scenario to obtain pre-labeled data;

[0202] A processing unit 802, configured to automatically derive a plurality of different prompt word schemes for each task, each prompt word scheme being specifically associated with a certain type of data;

[0203] The first acquisition unit 803 is used to import all tasks into the multimodal large model according to the prompt word scheme to perform independent questions respectively, and obtain the first output result corresponding to each prompt word scheme;

[0204] An extraction unit 804 is used to extract the result using an unsupervised voting algorithm for the first output result, and generate a voting statistical result for each task;

[0205] The judgment unit 805 is used to start the election mechanism if the voting statistics result contains a positive correlation mark;

[0206] A comparison unit 806, used to compare the election threshold preset under the election mechanism with the number of positive correlation marks in the voting statistics result;

[0207] A determination unit 807 is used to determine that the task is a positive correlation sample if the number of positive correlation marks reaches or exceeds the selection threshold;

[0208] A training unit 808 is used to input the pre-labeled data and the positively correlated samples into the unimodal small model for training iteration to obtain an optimized unimodal small model;

[0209] Annotation unit 809, used to automatically annotate the target data to be annotated in batches using a single-modal small model, and output the annotation results;

[0210] The quality inspection unit 810 is used to perform quality inspection on the labeling result to obtain the final labeling result.

[0211] Optionally, the device further comprises:

[0212] The second acquisition unit 811 acquires the number of positive correlation marks in the voting statistics result;

[0213] The initialization unit 812 initializes the election mechanism and sets an election threshold in the election mechanism.

[0214] Specifically, the first acquisition unit 803 includes:

[0215] Based on the step-by-step prompting method, each prompt word scheme is broken down into multiple steps;

[0216] According to the sequence of steps, the multimodal large model asks questions about the tasks one by one and obtains the second output result corresponding to each step;

[0217] The second output results of each prompt word solution are counted and summarized to obtain the first output results corresponding to each prompt word solution.

[0218] Specifically, the extraction unit 804 includes:

[0219] Obtain the first output result of all prompt word solutions in each task;

[0220] Determine whether there are positively correlated tags in each cue word solution and calculate the voting statistics of the task.

[0221] Specifically, the training unit 808 includes:

[0222] Integrate the pre-labeled data and positively correlated samples into a training dataset and preprocess the training dataset;

[0223] Initialize the single-modal small model, and the initialization is to set the weight and bias items of the single-modal small model;

[0224] The preprocessed training data set is input into the single-modal small model, and the prediction results of positively correlated samples are generated by layer-by-layer calculation;

[0225] Use the loss function to compare the predicted results with the pre-labeled data to obtain the loss function value;

[0226] According to the gradient information of the loss function, the gradient of each parameter in the single-modal small model is calculated through the back-propagation algorithm;

[0227] Based on the gradient, use the optimization algorithm to update the weights and biases of the unimodal small model;

[0228] Determine whether the loss function value converges;

[0229] When the loss function value converges, the current parameters are saved to obtain the optimized single-mode small model.

[0230] Specifically, the marking unit 809 includes:

[0231] Use a small single-modal model to extract features from the target data to be labeled to obtain feature information;

[0232] According to the feature information, the single-modal small model automatically generates corresponding annotation information for the target data to be annotated and outputs the annotation results.

[0233] See also Fig. 9 The present application also provides a device for automatic data labeling, including:

[0234] Processor 901, memory 902, input and output unit 903, bus 904;

[0235] The processor 901 is connected to the memory 902, the input and output unit 903 and the bus 904;

[0236] The memory 902 stores a program, and the processor 901 calls the program to execute any of the above methods.

[0237] The present application also relates to a computer-readable storage medium, on which a program is stored. When the program is run on a computer, the computer executes any of the above methods.

[0238] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0239] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0240] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0241] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0242] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, etc. Various media that can store program codes.

Claims

1. A method for automatic data labeling, characterized in that: The method comprises: In the scenario of multi-task input, the data in each task is manually pre-labeled to obtain pre-labeled data; Automatically derive multiple different prompt word schemes for each task, each prompt word scheme is specifically associated with a certain type of data; According to the prompt word scheme, all tasks are imported into the multimodal large model to conduct independent questions respectively, and the first output result corresponding to each prompt word scheme is obtained; For the first output result, an unsupervised voting algorithm is used to extract the result and generate the voting statistics result of each task; If the voting statistics contain a positive correlation mark, the election mechanism is initiated; Comparing the election threshold preset under the election mechanism with the number of positive correlation marks in the voting statistics result; If the number of positive correlation marks reaches or exceeds the selection threshold, the task is determined to be a positive correlation sample; Inputting the pre-labeled data and the positively correlated samples into a unimodal small model for training iteration to obtain an optimized unimodal small model; Use the single-modal small model to automatically label the target data to be labeled in batches, and output the labeling results; The labeling result is quality checked to obtain a final labeling result.

2. The method according to claim 1, characterized in that According to the prompt word scheme, all tasks are imported into the multimodal large model for independent questioning, and the first output result corresponding to each prompt word scheme is obtained, including: Based on the step-by-step prompting method, each prompt word scheme is broken down into multiple steps; According to the order of steps, use the multimodal large model to ask questions about the tasks one by one and obtain the second output result corresponding to each step; The second output results of each prompt word solution are counted and summarized to obtain the first output results corresponding to each prompt word solution.

3. The method according to claim 2, characterized in that The counting and summarizing of the second output results of each prompt word scheme to obtain the first output results corresponding to each prompt word scheme includes: Use a multimodal big model to set corresponding expected results for each step of the question; Counting and summarizing the second output results of each prompt word scheme to obtain a summary result of each prompt word scheme; Determine whether the second output result in the summary result is consistent with the corresponding expected result; According to the judgment result, the summary result is attached with a relevant mark and a first output result corresponding to each prompt word scheme is generated, wherein the relevant mark includes a positive relevant mark and a negative relevant mark.

4. The method according to claim 3, characterized in that For the first output result, an unsupervised voting algorithm is used to extract the result and generate the voting statistics of each task, including: Obtain the first output result of all prompt word solutions in each task; Determine whether there is a positive correlation mark in each prompt word scheme, and calculate the voting statistics of the task; The voting statistics are calculated using the following formula: Among them, FinalResult represents the voting statistics result of the task, and Y[i] represents the output result of the i-th prompt word solution.

5. The method according to claim 1, characterized in that Before comparing the election threshold preset under the election mechanism with the number of positive correlation marks in the voting statistics, the method further includes: Obtaining the number of positive correlation marks in the voting statistics result; Initialize an election mechanism and set an election threshold in the election mechanism.

6. The method according to any one of claims 1 to 5, characterized in that: The step of inputting the pre-labeled data and the positively correlated samples into the unimodal small model for training iteration to obtain an optimized unimodal small model includes: Integrate the pre-labeled data and the positively correlated samples into a training data set, and preprocess the training data set; Initializing the unimodal small model, wherein the initialization is to set the weight and bias items of the unimodal small model; Inputting the preprocessed training data set into the single-modal small model, and generating prediction results of positively correlated samples by layer-by-layer calculation; Compare the prediction result with the pre-labeled data using a loss function to obtain a loss function value; According to the gradient information of the loss function, the gradient of each parameter in the single-modal small model is calculated by a back-propagation algorithm; Based on the gradient, updating the weights and biases of the unimodal small model using an optimization algorithm; Determine whether the loss function value converges; When it is determined that the loss function value converges, the current parameters are saved to obtain an optimized single-mode small model.

7. According to the method of any one of claims 1 to 5, the step of automatically labeling the target data to be labeled in batches using the single-modal small model and outputting the labeling results comprises: Using the single-modal small model to extract features from the target data to be labeled, to obtain feature information; According to the feature information, the single-modal small model automatically generates corresponding annotation information for the target data to be annotated, and outputs the annotation result.

8. A device for automatic batch labeling of data, characterized in that: The device comprises: A pre-labeling unit is used to manually pre-label the data in each task in a multi-task input scenario to obtain pre-labeled data; A processing unit, configured to automatically derive a plurality of different prompt word schemes for each task, each prompt word scheme being specifically associated with a certain type of data; A first acquisition unit is used to import all tasks into the multimodal large model according to the prompt word scheme to perform independent questions respectively, and obtain a first output result corresponding to each prompt word scheme; An extraction unit, configured to extract results using an unsupervised voting algorithm for the first output result, and generate voting statistics results for each task; A judgment unit, configured to start an election mechanism if the voting statistics result contains a positive correlation mark; A comparison unit, used to compare the election threshold preset under the election mechanism with the number of positive correlation marks in the voting statistics result; A determination unit, configured to determine that the task is a positively correlated sample if the number of positively correlated marks reaches or exceeds the selection threshold; A training unit, used for inputting the pre-labeled data and the positively correlated samples into a unimodal small model for training iteration to obtain an optimized unimodal small model; A labeling unit, used to automatically label the target data to be labeled in batches using the single-modal small model, and output the labeling results; The quality inspection unit is used to perform quality inspection on the labeling result to obtain the final labeling result.

9. A device for automatic batch labeling of data, characterized in that: The device comprises: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a program stored thereon, wherein the program, when executed on a computer, performs the method according to any one of claims 1 to 7.

Images