Data marking method, device and equipment

Abstract

The embodiment of the invention discloses a data marking method, device and equipment, and the method comprises the steps: obtaining target data to be marked, and the target data comprises one or more of image data, text data, video data and voice data; based on the target data and a pre-trained large model, determining a label corresponding to the target data, the large model being a first loss function constructed by generating comparative learning based on constructed data pairs associated with each other, and a model obtained by performing model training on a second loss function constructed by performing fine tuning on the large model.

Description

TECHNICAL FIELD

[0001] The present document relates to the technical field of computers, and particularly relates to a data labeling method, device and equipment. BACKGROUND

[0002] Label understanding is an important part of modern Internet platform content management. Through in-depth analysis and understanding of labels, relevant labels can be more accurately recommended, thereby improving the accuracy and efficiency of content classification, which not only greatly improves the content management background, but also plays a key role in the content distribution process. In community content management, it is often necessary to understand and label a large amount of data content in order to classify, search and recommend data content while protecting user privacy. Generally, specific requirements can be proposed by an operating agency, and then artificial labeling and identification of labels are performed. However, the above-mentioned label production method needs to be completed by artificial, which consumes a large amount of time and human resources. The labeling process for a large amount of data content can be very slow, and inconsistencies in labels can easily occur. Therefore, a more optimal data labeling method is needed to improve labeling efficiency and save resources. SUMMARY

[0003] The purpose of the embodiments of the present specification is to provide a more optimal data labeling method to improve labeling efficiency and save resources.

[0004] In order to achieve the above technical solutions, the embodiments of the present specification are implemented as follows: The data labeling method provided by the embodiments of the present specification comprises: obtaining target data to be labeled, the target data comprising one or more of image data, text data, video data and voice data; determining labels corresponding to the target data based on the target data and a pre-trained large model, the large model being a model obtained by model training based on a first loss function constructed by a way of generative contrast learning for a pair of mutually related data and a second loss function constructed by a way of fine-tuning the large model.

[0005] The data labeling device provided by the embodiments of the present specification comprises: a data acquisition module, which acquires target data to be labeled, the target data comprising one or more of image data, text data, video data and voice data; and a labeling module, which determines labels corresponding to the target data based on the target data and a pre-trained large model, the large model being a model obtained by model training based on a first loss function constructed by a way of generative contrast learning for a pair of mutually related data and a second loss function constructed by a way of fine-tuning the large model.

[0006] The data marking device provided by the embodiments of the present specification comprises a processor and a memory arranged to store computer executable instructions, which, when executed, cause the processor to: obtain target data to be marked, the target data comprising one or more of image data, text data, video data, and voice data; determine a label corresponding to the target data based on the target data and a pre-trained large model, the large model being a model obtained through model training based on a first loss function constructed in a manner of generative contrast learning for a pair of interrelated data and a second loss function constructed in a manner of fine-tuning the large model.

[0007] The embodiments of the present specification also provide a storage medium for storing computer executable instructions, which, when executed by a processor, implement the following processes: obtaining target data to be marked, the target data comprising one or more of image data, text data, video data, and voice data; determining a label corresponding to the target data based on the target data and a pre-trained large model, the large model being a model obtained through model training based on a first loss function constructed in a manner of generative contrast learning for a pair of interrelated data and a second loss function constructed in a manner of fine-tuning the large model.

[0008] The embodiments of the present specification also provide a computer program product comprising a computer program, which, when executed by a processor, implements the following processes: obtaining target data to be marked, the target data comprising one or more of image data, text data, video data, and voice data; determining a label corresponding to the target data based on the target data and a pre-trained large model, the large model being a model obtained through model training based on a first loss function constructed in a manner of generative contrast learning for a pair of interrelated data and a second loss function constructed in a manner of fine-tuning the large model. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present specification or the prior art, brief introductions to the drawings needed in the embodiments or prior art descriptions will be given below. Obviously, the drawings in the following description are only some embodiments described in the present specification, and those skilled in the art can also obtain other drawings according to these drawings without paying creative labor; Figure 1 A schematic diagram of a data marking system of the present specification; Figure 2 A schematic diagram of a data marking process of the present specification; Figure 3A schematic diagram of a data tagging process based on large model data of the present specification; Figure 4 A schematic diagram of a data tagging process based on large model data representation processing of the present specification; Figure 5 A schematic diagram of a data tagging process based on management user data of the present specification; Figure 6 A schematic diagram of each module of a data tagging system of the present specification; Figure 7 A schematic diagram of a data tagging device of the present specification; Figure 8 A schematic diagram of a data tagging device of the present specification. DETAILED DESCRIPTION

[0010] The embodiments of the present specification provide a data tagging method, device and equipment.

[0011] In order to enable those skilled in the art to better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be described clearly and completely in conjunction with the drawings in the embodiments of the present specification. Obviously, the described embodiments are only part of the embodiments of the present specification, not all. Based on the embodiments in the present specification, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present specification.

[0012] The embodiment of the specification provides a community content understanding tagging mechanism based on a large model. Tag understanding is an important part of modern Internet platform content management. Through in-depth analysis and understanding of tags, relevant tags can be more accurately recommended, thereby improving the accuracy and efficiency of content classification. This not only greatly improves the content management background, but also plays a key role in the content distribution process. When users browse data content, they will more easily find posts or topics that match their interests, thereby improving the user experience and stickiness of the platform. At the same time, accurate tag recommendation can also help content creators better reach target audiences, allowing high-quality content to be more widely disseminated. In short, tag understanding plays a crucial role in the efficient operation of information flow. In community content management, it is often necessary to understand and tag a large amount of data content in order to classify, search and recommend data content. Usually, specific requirements can be proposed by operating agencies, and then artificial identification and tagging of tags can be performed. The production link of the tags is as follows: for algorithms, tags can be predicted by models; for data, tag assets can be managed; for development, data sedimentation links and usage links can be used; for operation, cpv (i.e., pay-per-display) can be used; for tagging, artificial tagging can be used, and judges can guide tagging. However, the above-mentioned production method of tags needs to be completed by humans, which requires a large amount of time and human resources. The process of tagging a large amount of data content can be very slow, and the data asset is heavy, with heavy production, maintenance, update and consumption links. In addition, different personnel may give different tags to the same data content, resulting in inconsistency of the tags. Moreover, the scalability of this method is poor, and personal preferences and judgments may sometimes have an unnecessary impact on the results.

[0013] In addition, a small model in machine learning can be used for prediction, that is, a model is trained using samples that have been labeled so that it can automatically assign appropriate labels to new content. This approach generally includes the following steps: data collection and labeling, a large amount of community data content needs to be collected first, and a part of it is manually labeled to establish a training set; feature extraction, such as using TF-IDF, bag-of-words model, or current commonly used word vector representation (such as Word2Vec, BERT, etc.) to convert text content into machine-understandable features; model training, using labeled data to train a machine learning model, such as a small model of logistic regression, naive Bayes support vector machine, or even a simple neural network; prediction and evaluation, using the trained model to predict new data content, and evaluating it through preset indicators (such as accuracy, recall, F1 value, etc.). However, in the above approach, the model is usually trained for existing labels. When new labels need to be introduced or the label structure needs to be modified, it may be necessary to collect data again and retrain the model, which will consume a lot of time and resources. In addition, due to the limited capacity and complexity of small models, it is difficult to handle classification tasks with too many labels or complex hierarchical structures. Furthermore, if the quality of the labeled data is not high, the performance of the model will be greatly affected. In addition, for dynamically changing community content, the model may become outdated. Commonly used small models have limited ability to understand the deep meaning and context of the text, which may result in inaccurate label assignment. Therefore, the embodiments of the present specification provide a hot topic soft label mining scheme coupled to realize the rapid and effective mining of target understanding labels for all existing and new data content, and improve the production efficiency of labels. For specific processing, please refer to the specific content in the following embodiments.

[0014] The data labeling method provided by one or more embodiments of the present specification can be applied to the implementation environment of data labeling, with reference to Figure 1 The implementation environment at least includes: A client 100 and a server 200, wherein: The client 100 can run on a terminal device, which can be a mobile phone, a personal computer, a tablet computer, an e-book reader, a wearable device, a device for information interaction based on AR (Augmented Reality), VR (Virtual Reality), and a laptop computer, etc. The terminal device can install the client 100, which can be an application program, a browser, or a subprogram loaded in the application program, etc.

[0015] The service end 200 can run on a server, which can be one or more servers, a server cluster composed of several servers, a cloud server of a cloud computing platform, or the like. The server can install the service end 200, which can be an application program or a subprogram loaded in the application program, and the large model can be integrated in the service end 200, or the service end 200 can call the large model to perform corresponding operations.

[0016] In addition, a database 300 can also be included, which can be arranged in the server running the service end 200 or outside the server running the service end 200, and the database 300 can store data and / or related label information, and the like.

[0017] In this implementation environment, the service end 200 can obtain target data to be labeled through the client end 100, the target data including one or more of image data, text data, video data, and voice data, and the service end 200 determines the label corresponding to the target data based on the target data and a pre-trained large model. The large model is a model obtained by model training based on a first loss function constructed by a way of generating contrastive learning of mutually related data pairs and a second loss function constructed by a way of fine-tuning the large model.

[0018] As shown in Figure 2 The method provided by the embodiments of the present specification can be executed by a terminal device or a server, and the terminal device can be a mobile terminal device such as a mobile phone or a tablet computer, or a computer device such as a notebook computer or a desktop computer, or an IoT device (such as a smart watch or a vehicle-mounted device), and the server can be a single server or a server cluster composed of multiple servers, and the server can be a background server of a financial service or an online shopping service, or a background server of an application program. In the embodiments, the execution subject is taken as the server for detailed description. For the case that the execution subject is the terminal device, the case of the server can be referred to, and the description is not repeated here. The method can specifically include the following steps: In step S202, target data to be labeled is obtained, and the target data includes one or more of image data, text data, video data, and voice data.

[0019] The image data can be data of any image, where the image can be, for example, an image of a building, an image of a person, a face image, a fingerprint image, etc. The text data can be data containing any text content, for example, data of a webpage, data of a document, etc. The video data and the voice data can be data containing any content, which can be set according to actual conditions. It should be noted that the target data in the embodiment can be data from a community, where the community can be a data platform, for example, a short video platform, a blog platform, a microblog platform, an online shopping platform, etc., which can be set according to actual conditions.

[0020] In implementation, when it is necessary to tag certain data (i.e., target data), the target data can be uploaded through a specified page, or when a certain business is performed and a certain data (i.e., target data) is received, the target data can be tagged first, at which time the target data to be tagged can be obtained. The above is only a realizable processing manner, and in actual application, the target data to be tagged can also be obtained in various different ways, which can be set according to actual conditions.

[0021] In step S204, based on the target data and the pre-trained large model, the label corresponding to the target data is determined. The large model is obtained by model training based on a first loss function constructed by a mutual correlation data pair through a generative contrast learning manner, and a second loss function constructed by fine-tuning the large model.

[0022] The large model can be any large model that needs to tag data, where the large model can be a machine learning model with a large number of model parameters (which can usually reach hundreds of millions of model parameters) and a complex model structure. The large model can include generative large models and discriminative large models, etc. The specific type of large model can be selected according to actual conditions, which is not limited in the embodiments of the present specification. The label can include various labels, for example, risk, no risk, role-playing game, game, etc., which can be set according to actual conditions, which is not limited in the embodiments of the present specification.

[0023] In implementation, the large model can be jointly trained in two ways, i.e., a way combining a generative model and contrastive learning (i.e., a generative contrastive learning way), and a large model fine-tuning way. The generative contrastive learning way can use a generative model such as a generative adversarial network (GAN) to generate data samples. The goal of the generative model is to generate data samples similar to real data, which helps to enhance the diversity of data. Then, through a contrastive learning mechanism, the large model learns how to distinguish between positive data samples and negative data samples. The key of contrastive learning lies in constructing positive and negative data sample pairs, and training the large model by maximizing the similarity between positive data samples and minimizing the similarity between negative data samples. Finally, the generative model and the contrastive learning mechanism can be jointly trained. The data samples generated by the generative model can be used for contrastive learning, and the feedback provided by the contrastive learning can be used to improve the quality of the generative model. This mutually promoting process improves the overall learning effect.

[0024] The data pairs can be constructed to be associated with each other. Based on the above-mentioned generative contrastive learning way, a corresponding loss function (i.e., a first loss function) can be constructed. Then, based on the large model fine-tuning way, a second loss function can be constructed. The overall loss function of the large model can be generated based on the first loss function and the second loss function. The overall loss function of the large model generated above can be used to train the large model based on the pre-obtained training samples, so as to obtain a trained large model. The trained large model can perform labeling processing on the input data, so as to obtain the label corresponding to the input data. After obtaining the target data through the above-mentioned way, the target data can be input into the large model. The large model can analyze the content and context of the target data, determine the semantic information of the target data, and then generate the label of the target data. In actual application, the above-mentioned data labeling processing by the large model is only an optional processing way. In actual application, various different processing ways can be included, which can be set according to actual conditions, and the embodiments of the present specification do not limit this.

[0025] The embodiments of the present specification provide a data labeling method. The target data to be labeled is obtained, the target data including one or more of image data, text data, video data, and voice data. Then, the label corresponding to the target data can be determined based on the target data and a pre-trained large model. The large model is obtained by model training based on a first loss function constructed by a generative contrastive learning way on the basis of associated data pairs, and a second loss function constructed by a large model fine-tuning way. In this way, the label of the data is generated by the large model, so as to improve the production efficiency of the label, save resources, and realize the rapid and effective mining of the target understanding label for all stock and new data content.

[0026] In practical applications, the specific processing mode of step S204 can be various, and the following provides an optional processing mode, which can include the processing of steps S20402 and S20404. Based on the above Figure 2 , the method specifically includes the steps as shown in Figure 3 .

[0027] In step S20402, the first data representation for guiding the large model to generate the target data is generated based on the target data, and the first prompt information of the label corresponding to the target data is determined based on the first data representation.

[0028] The large model can be used to extract features of the input data (i.e., target data) to generate the first data representation of the target data, and the large model can also generate the label corresponding to the target data based on the generated first data representation. The first prompt information can be a prompt information Prompt applied to the large model, and the first prompt information is a text input into the large model, which is used to guide the large model to generate an output result meeting the requirements. The first prompt information can be a question, an instruction or a context, which helps the large model better understand and respond to various queries. The first prompt information can include explicit instruction type prompt information, context supplement type prompt information and example guidance type prompt information, etc. The first prompt information helps the large model generate more relevant and accurate output results by providing detailed background information and examples.

[0029] In step S20404, the target data and the first prompt information are input into the large model to obtain the label corresponding to the target data.

[0030] In practical applications, the specific processing mode of step S204 can be various, and the following provides an optional processing mode, which can include the processing of steps S20406~S20412. Based on the above Figure 2 , the method specifically includes the steps as shown in Figure 4 .

[0031] In step S20406, the second prompt information for guiding the large model to generate the second data representation of the target data is generated based on the target data.

[0032] The second data representation can be the same as or different from the first data representation. Based on this, one or more different feature extraction algorithms can be set in the large model to obtain corresponding data representations, which can be set according to actual conditions. The second prompt information is similar to the first prompt information, and the second prompt information and the first prompt information are both prompt information Prompt applied to the large model.

[0033] In step S20408, the target data and the second prompt information are input into the large model to obtain the second data representation corresponding to the target data.

[0034] In step S20410, based on the second data representation, one or more similar data representations with a similarity greater than a preset first similarity threshold are obtained from a preset first database.

[0035] The first similarity threshold can be set according to the actual situation, such as 90% or 95%.

[0036] In implementation, the similarity between the second data representation and each data representation in the first database can be calculated, and one or more data representations with a calculated similarity greater than a preset first similarity threshold can be obtained. The obtained data representations can be used as similar data representations.

[0037] In step S20412, the first label of the data corresponding to each similar data representation is obtained, and the label corresponding to the target data is determined based on the obtained first label.

[0038] In implementation, the first label of each similar data representation can be obtained. At this time, the first label can include one or more. Then, the obtained first label can be used as the label corresponding to the target data. Alternatively, the obtained first label can be evaluated (such as credibility score or selection probability). Based on the final evaluation result, the better first label (such as the first label with the highest score or the first label with the highest selection probability) can be selected as the label corresponding to the target data. Alternatively, one or more first labels can be randomly selected from the obtained first labels as the label corresponding to the target data. The specific settings can be set according to the actual situation.

[0039] In practical applications, the specific processing method for determining the label corresponding to the target data based on the acquired first label in step S20412 can vary. The following provides another optional processing method, which specifically includes the processing steps S204122 to S204124. Based on this, in the above... Figure 4 Based on this, the specific steps included in this method can be as follows: Figure 5 As shown.

[0040] In step S204122, the obtained first tag is recommended to the management user.

[0041] In step S204124, the first tag selected by the management user from the acquired first tags is received as the tag corresponding to the target data, and the selected first tag is used as the tag corresponding to the target data.

[0042] In practical applications, the system formed by the above large model can also be used to recommend data content to users. The specific processing can refer to steps A02-A10.

[0043] In step A02, the second label selected by the user is obtained.

[0044] The second label can be any label, such as a life label, a news label, etc. The specific setting can be based on actual conditions.

[0045] In implementation, the second label selected by the user can be obtained in various ways. For example, the labels clicked by the user in a certain historical time period can be obtained, the obtained labels can be used as the second label selected by the user, or the operation behavior of the user can be detected in real time, and when it is detected that the user clicks a label, the label is used as the second label selected by the user, or the label can be indirectly selected by the user, for example, the user views a video in an application, the label corresponding to the video can be obtained, and the label corresponding to the video can be used as the second label selected by the user, etc. The specific setting can be based on actual conditions.

[0046] In step A04, the third prompt information for guiding the large model to generate the label representation of the second label is generated based on the second label.

[0047] The third prompt information can be a prompt information Prompt applied to the large model. For details, refer to the related content described above, which will not be repeated here.

[0048] In step A06, the second label and the third prompt information are input into the large model to obtain the label representation corresponding to the second label.

[0049] In step A08, based on the label representation corresponding to the second label, the label representation corresponding to one or more similar labels having a similarity greater than a second similarity threshold with the label representation corresponding to the second label is retrieved from the second database.

[0050] The second similarity threshold can be set according to actual conditions, such as 90% or 97%, etc.

[0051] In step A10, based on the retrieved label representation corresponding to one or more similar labels, the corresponding one or more similar labels are obtained, and the corresponding first data is determined based on the obtained similar labels, and the first data is recommended to the user.

[0052] In implementation, based on the retrieved tag representations, the tags corresponding to each tag representation are obtained, and these obtained tags can be used as similar tags. Then, data with the tags of the above similar tags can be obtained, and this obtained data can be used as the first data. The first data can be sent to the user to recommend the first data to the user.

[0053] In practical applications, the label representations contained in the second database can also be used for updates, as detailed in steps B2 to B8 below.

[0054] In step B2, multiple sample data to be labeled are obtained, including one or more of image data, text data, video data, and voice data.

[0055] In step B4, a sample data representation is generated based on the sample data to guide the large model in generating sample data, and a fourth prompt message for the label corresponding to the sample data is determined based on the sample data representation.

[0056] In step B6, the sample data and the fourth prompt information are input into the large model to obtain the labels corresponding to the sample data.

[0057] In step B8, the label representations contained in the second database are updated based on the labels corresponding to the sample data.

[0058] In implementation, the labels corresponding to the label representations contained in the second database can be updated based on the labels corresponding to the sample data, or the labels corresponding to the label representations contained in the second database can be updated based on the label representations corresponding to the labels corresponding to the sample data, thereby updating the labels corresponding to the label representations contained in the second database, etc. The specific settings can be configured according to the actual situation.

[0059] In practical applications, the system constructed by the above-mentioned large model can also be used to recommend the following data content to users, as detailed in steps C02 to C10 below.

[0060] In step C02, the third tag selected by the user is obtained.

[0061] In step C04, a fourth cue message is generated based on the third label to guide the large model in generating the label representation of the third label.

[0062] In step C06, the third label and the fourth prompt information are input into the large model to obtain the label representation corresponding to the third label.

[0063] In step C08, based on the tag representation corresponding to the third tag, one or more data representations of the second data that match the tag representation corresponding to the third tag are determined from the third database.

[0064] In implementation, the third database can store the correspondence between the label representation and the data representation. The data representation corresponding to the label representation of the third label can be queried in the third database to obtain one or more data representations of the second data matched with the label representation corresponding to the third label. Alternatively, the data representations contained in the third database can be obtained, the label of the data corresponding to each data representation can be determined, the label representation of the determined label can be obtained, the similarity between the label representation corresponding to the third label and the label representation of the determined label can be calculated, and the data representation corresponding to the maximum similarity can be taken as the data representation matched with the label representation corresponding to the third label. In actual application, the data representation of one or more second data matched with the label representation corresponding to the third label can also be determined from the third database in other various manners, which can be set according to actual conditions.

[0065] In step C10, the corresponding second data is obtained based on the data representation of one or more matched second data, and the second data is recommended to the user.

[0066] In actual application, the above large model can also be fine-tuned in the following manner, which can be specifically referred to the processing of steps D2 and D4.

[0067] In step D2, the pre-trained large model and the training data for visual question answering are obtained.

[0068] The visual question answering (VQA) is a subfield of computer vision, which aims to enable computers to understand image content and provide accurate answers according to human questions. In order to train and test the large model, appropriate data sets need to be prepared or obtained, based on which the training data for visual question answering can be obtained.

[0069] In step D4, the pre-trained large model is fine-tuned by the training data for visual question answering to obtain the fine-tuned large model.

[0070] The embodiment of the present specification provides a data labeling method, which comprises the following steps: obtaining target data to be labeled, the target data comprising one or more of image data, text data, video data and voice data; and determining a label corresponding to the target data based on the target data and a pre-trained large model. The large model is obtained by training a first loss function based on a set of mutually related data pairs through generative contrast learning, and a second loss function through fine-tuning of the large model. In this way, the label of the data is generated through the large model, thereby improving the production efficiency of the label, saving resources, and achieving rapid and effective mining of the target understanding label for all stock and new data content. In addition, through the technology represented by the large model, the production efficiency of the label is further improved after aligning the image-text representation and the label representation.

[0071] Based on the data labeling method provided by the above embodiment of the present specification, the embodiment of the present specification further provides a data labeling system, as shown in Figure 6

[0072] The data labeling system comprises an input interface, a content representation module and a label L2 module. The input interface is used to receive input data content. The content representation module can manage one or more data representations for each data. The data can be image data, text data, video data, voice data and other multi-modal data. For multi-modal data, there can also be alignment tasks and VQA tasks. The label L2 module is used to obtain and manage the label L2. The label L2 can be a soft label, such as a user group and a scene. The label L2 can not need to be set with a number limit and does not have an accuracy requirement, but has a related probability. In addition, the label L2 is easy to maintain (usually involving adding new labels and representation iteration). The label involved in the above data labeling method in the embodiment of the present specification is the label L2.

[0073] Based on the above system architecture, the data labeling system can have one or more of the following functions: training data representation: extracting image-text labels, including factual labels (such as IP address, geographic location, content theme, etc.) and understanding labels (such as applicable scenarios, etc.), extracting multi-modal information of image-text and video, including text description ability, data source, multi-modal data alignment training, etc.; dynamic label operation: divided into stock labels and new labels, the stock labels maintain related scores, the new labels build label mining links, do not do label normalization processing, and the update efficiency can be T+1 (where T can represent days); new label discovery: if a new dynamic label is found, the hot commodity-label re-mining-label verification, the market monitoring-hot label-label semantic extension-label verification; soft label into operation label library: the current label system is insufficient, when the data content is supplemented and the operation label system needs to be supplemented, it is necessary to relabel.​

[0074] When the input interface of the data tagging system receives the input data, the content representation module can generate corresponding prompt information based on the input data, and then call the large model to input the input data and the prompt information into the large model to obtain the data representation 1 corresponding to the input data. The label L2 module can also generate labels 1, labels 2, labels 3… corresponding to the input data based on the data representation 1 corresponding to the input data through the large model. For specific processing, please refer to the steps in the foregoing data tagging method, which will not be repeated here. When the set update period is reached, another batch of input data can be obtained through the input interface, and the above processing process can also be repeated. The content representation module can obtain the data representation 2, and the label L2 module can obtain the labels f1, labels f2, labels f3…, which can be used to update the labels 1, labels 2, labels 3…, and finally obtain the updated labels.

[0075] In addition, the data tagging system can also include a label L1 module, which is used to obtain and manage the label L1. The label L1 can be an operation level label, such as a category, a theme, a group, etc. The label L1 has a preparation rate requirement and can meet the operation level requirement.

[0076] The label L1 module can also obtain the labels L1 such as the theme, the category and the group of the input data when the input interface receives the input data. In addition, for the label L2 determined by the label L2 module, in some cases (some label L2 is a label possessed by some data or a label possessed by most data in some cases, etc.), it can also be adjusted to the label L1.

[0077] The embodiment of the present specification provides a data tagging system. By obtaining target data to be tagged, the target data including one or more of image data, text data, video data, and voice data, the label corresponding to the target data can be determined based on the target data and a pre-trained large model. The large model is a model obtained by model training based on a first loss function constructed by generating contrast learning based on the interrelated data pairs and a second loss function constructed by fine-tuning the large model. In this way, the label of the data is generated through the large model, thereby improving the production efficiency of the label, saving resources, and achieving rapid and effective mining of the target understanding label for all stock and new quantity data content. In addition, through the large model representation technology, the production efficiency of the label is further improved after aligning the image-text representation and the label representation. Moreover, by defining the label L1 and the label L2, the hard label managed by the background and the soft label understood are distinguished, and the management efficiency of the label is improved.

[0078] The data marking method provided by the embodiments of the present specification is based on the same idea. The embodiments of the present specification also provide a data marking device, as shown in Figure 7

[0079] The data marking device comprises a data acquisition module 701 and a marking module 702, wherein: The data acquisition module 701 acquires target data to be marked, wherein the target data comprises one or more of image data, text data, video data, and voice data. The marking module 702 determines a label corresponding to the target data based on the target data and a pre-trained large model. The large model is obtained by model training based on a first loss function constructed by a mutual correlation data pair through a generative contrast learning manner and a second loss function constructed by a fine-tuning manner of the large model.

[0080] In the embodiments of the present specification, the marking module 702 comprises: A first prompt determination unit generates first data representation for guiding the large model to generate the target data based on the target data, and determines first prompt information of the label corresponding to the target data based on the first data representation. A first marking unit inputs the target data and the first prompt information into the large model to obtain the label corresponding to the target data.

[0081] In the embodiments of the present specification, the marking module 702 comprises: A second prompt determination unit generates second prompt information of second data representation for guiding the large model to generate the target data based on the target data. A data representation determination unit inputs the target data and the second prompt information into the large model to obtain the second data representation corresponding to the target data. A similarity calculation unit acquires one or more similar data representations with a similarity greater than a preset first similarity threshold from a preset first database based on the second data representation. A label determination unit acquires a first label of data corresponding to each similar data representation, and determines the label corresponding to the target data based on the acquired first label.

[0082] In the embodiments of the present specification, the label determination unit recommends the acquired first label to a management user, receives a first label selected by the management user from the acquired first label as the label corresponding to the target data, and takes the selected first label as the label corresponding to the target data.

[0083] ​In the embodiments of the present specification, the device further comprises: The second label acquisition module acquires the second label selected by the user. The third prompt determination module generates third prompt information for guiding the large model to generate the label representation of the second label based on the second label. The label representation determination module inputs the second label and the third prompt information into the large model to obtain the label representation corresponding to the second label. The retrieval module retrieves, based on the label representation corresponding to the second label, the label representation corresponding to one or more similar labels from the second database, wherein the similarity between the label representation corresponding to the second label and the label representation corresponding to the one or more similar labels is greater than a second similarity threshold. The first recommendation module acquires the corresponding one or more similar labels based on the label representation corresponding to the one or more similar labels, and determines corresponding first data based on the acquired similar labels, and recommends the first data to the user.

[0084] In the embodiments of the present specification, the device further comprises: The sample acquisition module acquires a plurality of sample data to be labeled, wherein the sample data comprises one or more of image data, text data, video data, and voice data. The large model processing module generates sample data representation for guiding the large model to generate the sample data based on the sample data, and determines fourth prompt information of the label corresponding to the sample data based on the sample data representation. The label determination module inputs the sample data and the fourth prompt information into the large model to obtain the label corresponding to the sample data. The finer module updates the label representation contained in the second database based on the label corresponding to the sample data.

[0085] In the embodiments of the present specification, the device further comprises: The third label acquisition module acquires the third label selected by the user. The fourth prompt determination module generates fourth prompt information for guiding the large model to generate the label representation of the third label based on the third label. The label representation module inputs the third label and the fourth prompt information into the large model to obtain the label representation corresponding to the third label. The matching module determines, based on the label representation corresponding to the third label, the data representation of one or more second data matched with the label representation corresponding to the third label from the third database. The second recommendation module obtains the corresponding second data based on the data representation of the matched one or more second data, and recommends the second data to the user.

[0086] In the embodiments of the present specification, the device further comprises: The data acquisition module acquires a pre-trained large model and training data for visual question answering. The fine-tuning module fine-tunes the pre-trained large model through the training data for visual question answering to obtain a fine-tuned large model.

[0087] The embodiments of the present specification provide a data labeling device. The target data to be labeled is acquired, the target data including one or more of image data, text data, video data, and voice data. Then, a label corresponding to the target data can be determined based on the target data and a pre-trained large model. The large model is obtained by model training based on a first loss function constructed by a mutual correlation data pair through a contrastive learning manner and a second loss function constructed by fine-tuning the large model. In this way, the label of the data is generated through the large model, so that the production efficiency of the label can be improved, resources are saved, and rapid and effective mining of the target understanding label for all stock and new data content is realized.

[0088] The above is the data labeling device provided by the embodiments of the present specification. Based on the same idea, the embodiments of the present specification also provide a data labeling device, as shown in Figure 8

[0089] The data labeling device can be a terminal device or a server, etc. provided by the above embodiments.

[0090] The data labeling device can have great differences due to different configurations or performances, and can include one or more processors 801 and memories 802. The memories 802 can store one or more storage application programs or data. The memory 802 can be temporary storage or persistent storage. The application programs stored in the memory 802 can include one or more modules (not shown in the figure), each of which can include a series of computer executable instructions in the data labeling device. Further, the processor 801 can be configured to communicate with the memory 802 and execute a series of computer executable instructions in the memory 802 on the data labeling device. The data labeling device can also include one or more power supplies 803, one or more wired or wireless network interfaces 804, one or more input / output interfaces 805, and one or more keyboards 806.

[0091] ​In particular, in the present embodiment, the data tagging device comprises a memory and one or more programs, wherein one or more programs are stored in the memory, and the one or more programs can comprise one or more modules, and each module can comprise a series of computer executable instructions in the data tagging device, and the one or more programs configured to be executed by one or more processors include the following computer executable instructions: obtaining target data to be tagged, the target data comprising one or more of image data, text data, video data, and voice data; determining a label corresponding to the target data based on the target data and a pre-trained large model, the large model being a model obtained by model training based on a first loss function constructed by a way of generative contrast learning on a constructed pair of interrelated data, and a second loss function constructed by a way of fine-tuning the large model.

[0092] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the data tagging device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0093] The embodiment of the specification provides a data tagging device. By obtaining target data to be tagged, the target data comprising one or more of image data, text data, video data, and voice data, a label corresponding to the target data can be determined based on the target data and a pre-trained large model. The large model is a model obtained by model training based on a first loss function constructed by a way of generative contrast learning on a constructed pair of interrelated data, and a second loss function constructed by a way of fine-tuning the large model. In this way, the label of the data is generated by the large model, so as to improve the production efficiency of the label, save resources, and realize the rapid and effective mining of the target understanding label for all stock and new data content.

[0094] Further, based on the above Figures 2 to 6 One or more embodiments of the specification also provide a storage medium for storing computer executable instruction information. In a specific embodiment, the storage medium can be a U disk, an optical disk, a hard disk, etc. The computer executable instruction information stored in the storage medium can realize the following processes when executed by a processor. obtaining target data to be tagged, the target data comprising one or more of image data, text data, video data, and voice data; determine, based on the target data and a pre-trained large model, a label corresponding to the target data, the large model being a model obtained through model training based on a first loss function constructed in a manner of generative contrast learning on the basis of the constructed interrelated data pairs and a second loss function constructed in a manner of fine-tuning the large model.

[0095] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the above-mentioned one storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0096] The embodiment of the specification provides a storage medium. By obtaining target data to be labeled, the target data including one or more of image data, text data, video data, and voice data, a label corresponding to the target data can be determined based on the target data and a pre-trained large model. The large model is a model obtained through model training based on a first loss function constructed in a manner of generative contrast learning on the basis of the constructed interrelated data pairs and a second loss function constructed in a manner of fine-tuning the large model. In this way, the label of the data is generated through the large model, so as to improve the production efficiency of the label, save resources, and realize the rapid and effective mining of the target understanding label for all stock and new data content.

[0097] Further, based on the above Figures 2 to 6 The one or more embodiments of the specification also provide a computer program product, including a computer program. The computer program in the computer program product can implement the following flow when executed by a processor. Obtain target data to be labeled, the target data including one or more of image data, text data, video data, and voice data; Determine, based on the target data and a pre-trained large model, a label corresponding to the target data, the large model being a model obtained through model training based on a first loss function constructed in a manner of generative contrast learning on the basis of the constructed interrelated data pairs and a second loss function constructed in a manner of fine-tuning the large model.

[0098] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the above-mentioned one storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0099] The embodiment of the specification provides a computer program product, by acquiring target data to be marked, the target data including one or more of image data, text data, video data, voice data, then, the label corresponding to the target data can be determined based on the target data and the pre-trained large model, the large model is a model obtained by model training based on the first loss function constructed by the way of generating contrast learning of the mutually related data pairs, and the second loss function constructed by the way of fine-tuning the large model, in this way, the label of the data is generated through the large model, so as to improve the production efficiency of the label, save the resources, and realize the rapid and effective mining of the target understanding label for all stock and new quantity data content.

[0100] The above describes specific embodiments of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the order in which they are recited in the embodiments and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order in order to achieve the desired results. In some implementations, multitasking and parallel processing can be utilized or can be advantageous.

[0101] In the 1990s, it was possible to distinguish whether an improvement in a technology was a hardware improvement (e.g., an improvement in the circuit structure of a diode, transistor, switch, etc.) or a software improvement (an improvement in a method flow). However, as technology has advanced, many improvements in method flows today can be considered as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into a hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it themselves, without having to ask a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating an integrated circuit chip, this programming is now mostly implemented using "logic compiler" software, which is similar to the software compiler used when developing a program, and the original code before compilation must also be written in a specific programming language, which is called a hardware description language (HDL), and there are many types of HDL, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be aware that it is very easy to obtain a hardware circuit that implements a logical method flow by simply logically programming the method flow in one of the above-mentioned hardware description languages and programming it into an integrated circuit.

[0102] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the microprocessor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to implementing the controller in pure computer readable program code, it is possible to implement the controller in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. to perform the same functions by logically programming the method steps. Such a controller can therefore be considered to be a hardware component, and the means included therein for performing various functions can also be considered to be structures within the hardware component. Alternatively, the means for performing various functions can even be considered to be both a software module implementing the method and a structure within the hardware component.

[0103] The systems, apparatuses, modules or units illustrated by the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0104] For the sake of description, the above apparatuses are described in functional division and are described respectively. Of course, the functions of each unit can be implemented in the same or more software and / or hardware when implementing one or more embodiments of the present specification.

[0105] Those skilled in the art will understand that the embodiments of the present specification can be provided as a method, a system, or a computer program product. Therefore, one or more embodiments of the present specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, one or more embodiments of the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0106] The embodiments of the present specification are described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present specification. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable electronic devices to produce a machine, so that the instructions executed by the computer or other programmable electronic devices generate a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions of one or more flows and / or blocks in the flowcharts and / or block diagrams can be implemented by an apparatus. Figure 1 The functions of one or more flows and / or blocks in the flowcharts and / or block diagrams can be implemented by an apparatus.

[0107] These computer program instructions can also be stored in a computer readable memory that can direct the computer or other programmable electronic devices to work in a specific manner, so that the instructions stored in the computer readable memory produce a product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions of one or more flows and / or blocks in the flowcharts and / or block diagrams can be implemented by an apparatus. Figure 1 The functions of one or more flows and / or blocks in the flowcharts and / or block diagrams can be implemented by an apparatus.

[0108] These computer program instructions can also be loaded into a computer or other programmable electronic devices, so that a series of operation steps are performed on the computer or other programmable electronic devices to produce a computer implemented process, so that the instructions executed on the computer or other programmable electronic devices provide steps for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions of one or more flows and / or blocks in the flowcharts and / or block diagrams can be implemented by an apparatus. Figure 1 The functions of one or more flows and / or blocks in the flowcharts and / or block diagrams can be implemented by an apparatus.

[0109] In a typical configuration, the computing device includes one or more processors (CPU), input / output interface, network interface and memory.

[0110] The memory can include non-persistent memory in the computer readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer readable media.

[0111] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0112] It should also be noted that the terms "comprising", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0113] Those skilled in the art will appreciate that embodiments of the present specification can be provided as methods, systems or computer program products. Therefore, one or more embodiments of the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, one or more embodiments of the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] One or more embodiments of the present specification can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media, including storage devices.

[0115] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0116] The above only describes the embodiments of the specification and is not used to limit the file. The specification can have various changes and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the specification shall be included in the claim range of the file.

Claims

1. A data tagging method, the method comprising: obtaining target data to be tagged, the target data comprising one or more of image data, text data, video data, voice data; determining a label corresponding to the target data based on the target data and a pre-trained large model, the large model being a model obtained by model training based on a first loss function constructed by a way of generative contrastive learning on correlated data pairs and a second loss function constructed by a way of fine-tuning the large model.

2. The method of claim 1, wherein the determining the label corresponding to the target data based on the target data and the pre-trained large model comprises: generating a first data representation for guiding the large model to generate the target data based on the target data, and determining first prompt information of the label corresponding to the target data based on the first data representation; inputting the target data and the first prompt information into the large model to obtain the label corresponding to the target data.

3. The method of claim 1, wherein the determining the label corresponding to the target data based on the target data and the pre-trained large model comprises: generating second prompt information of a second data representation for guiding the large model to generate the target data based on the target data; inputting the target data and the second prompt information into the large model to obtain the second data representation of the target data; based on the second data representation, obtaining one or more similar data representations from a first database having a similarity greater than a first preset similarity threshold with the second data representation; obtaining a first label of data corresponding to each similar data representation, and determining the label corresponding to the target data based on the obtained first label.

4. The method of claim 3, wherein the determining the label corresponding to the target data based on the obtained first label comprises: recommending the obtained first label to a management user; receiving a first label selected by the management user from the obtained first label as the label corresponding to the target data, and taking the selected first label as the label corresponding to the target data.

5. The method of any one of claims 1-4, further comprising: obtaining a second label selected by a user; generating third prompt information of a label representation for guiding the large model to generate the second label based on the second label; inputting the second label and the third prompt information into the large model to obtain a label representation corresponding to the second label; based on the label representation corresponding to the second label, retrieving one or more similar label representations corresponding to the label representation corresponding to the second label from a second database having a similarity greater than a second similarity threshold with the label representation corresponding to the second label; based on the retrieved one or more similar label representations, obtaining corresponding one or more similar labels, and determining corresponding first data based on the obtained similar labels, and recommending the first data to the user.

6. The method of claim 5, further comprising: obtaining a plurality of sample data to be labeled, the sample data comprising one or more of image data, text data, video data, and voice data; generating, based on the sample data, a sample data representation for guiding a large model to generate the sample data, and determining fourth prompt information of a label corresponding to the sample data based on the sample data representation; inputting the sample data and the fourth prompt information into the large model to obtain the label corresponding to the sample data; updating a label representation contained in the second database based on the label corresponding to the sample data.

7. The method of any one of claims 1-4, further comprising: obtaining a third label selected by a user; generating, based on the third label, fourth prompt information of a label representation for guiding a large model to generate the third label; inputting the third label and the fourth prompt information into the large model to obtain a label representation corresponding to the third label; determining, based on the label representation corresponding to the third label, data representations of one or more second data matching the label representation corresponding to the third label from a third database; based on the data representations of the one or more second data matched, obtaining corresponding second data, and recommending the second data to the user.

8. The method of claim 7, further comprising: obtaining a pre-trained large model and training data for visual question answering; fine-tuning the pre-trained large model based on the training data for visual question answering to obtain a fine-tuned large model.

9. A data labeling device, the device comprising: a data obtaining module configured to obtain target data to be labeled, the target data comprising one or more of image data, text data, video data, and voice data; a labeling module configured to determine a label corresponding to the target data based on the target data and a pre-trained large model, the large model being a model obtained by model training based on a first loss function constructed by a way of generating contrastive learning for a pair of interrelated data, and a second loss function constructed by a way of fine-tuning the large model.

10. A data labeling device, the data labeling device comprising: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to: obtain target data to be labeled, the target data comprising one or more of image data, text data, video data, and voice data; determine a label corresponding to the target data based on the target data and a pre-trained large model, the large model being a model obtained by model training based on a first loss function constructed by a way of generating contrastive learning for a pair of interrelated data, and a second loss function constructed by a way of fine-tuning the large model.

Images