Data annotation method, device and system and storage medium

Abstract

The invention discloses a data annotation method, device and system and a storage medium. The method comprises the steps that demand description information input by a user is acquired; identifying the demand description information to obtain cue words in the demand description information, the cue words at least comprising crawler cue words and picture generation cue words; extracting keywords in the crawler prompt words; acquiring a picture element corresponding to the keyword from a specific data source; combining the picture elements corresponding to the keywords to generate a picture corresponding to a picture generation prompt word; and according to the picture, generating prompt words or the picture elements to label the picture. By adopting the scheme provided by the invention, the labor cost can be saved, and the stability and objectivity of the labeling result can be improved.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a data annotation method, device, system and storage medium. Background Art

[0002] Training data plays a vital role in current artificial intelligence research and industry. For machine supervised learning tasks, labeling training data is a necessary step, but labeling training data is a tedious and time-consuming process, which usually requires professional knowledge and the judgment of the annotator. Different tasks may require different labeling methods. For example, pixel-level segmentation labeling, text classification, and sentiment labeling of entities in images require a lot of labor. In addition, since there is no unified labeling standard for training data, when labeling is done by the annotator, the subjective judgment of the annotator has a great influence on the labeling results. Different annotators may have different labeling results for the same training data, resulting in low stability and objectivity of the labeling effect of existing labeling methods. Summary of the invention

[0003] The present application provides a data labeling method, device, system and storage medium to save labor costs and improve the stability and objectivity of labeling results.

[0004] This application provides a data annotation method, including:

[0005] Get the requirement description information entered by the user;

[0006] Identify the requirement description information to obtain prompt words in the requirement description information, where the prompt words at least include crawler prompt words and image generation prompt words;

[0007] Extracting keywords from the crawler prompt words;

[0008] Acquire image elements corresponding to the keywords from a specific data source;

[0009] Combining the image elements corresponding to the keywords to generate an image corresponding to the image generation prompt word;

[0010] Generate prompt words or the picture elements according to the picture to mark the picture.

[0011] The beneficial effects of the present application are: it is capable of obtaining crawler prompt words and image generation prompt words according to the demand description information input by the user, and obtaining image elements corresponding to the keywords in the crawler prompt words from a specific data source; combining the image elements corresponding to the keywords to generate images corresponding to the image generation prompt words; since the image generation prompt words have been obtained through the demand description information, the images can be annotated through the image generation prompt words, and since the elements of the generated images are the image elements corresponding to the keywords in the crawler prompt words, the images can be annotated directly based on the image elements corresponding to the specific keywords, thereby saving labor costs, and since the annotation results are not affected by the subjective judgment of the annotator, the stability and objectivity of the annotation results are improved.

[0012] In one embodiment, combining the picture elements corresponding to the keywords to generate a picture corresponding to the picture generation prompt word includes:

[0013] Combining the image elements corresponding to the keywords to generate groups corresponding to the images, wherein each group contains at least one image element corresponding to all the keywords;

[0014] Combine the image elements in each group to generate images corresponding to each group;

[0015] The matching degree between the pictures corresponding to each group and the picture generation prompt word is calculated, and the pictures whose matching degree with the picture generation prompt word is greater than a preset matching degree are taken as the pictures corresponding to the picture generation prompt word.

[0016] In one embodiment, the prompt word further includes a code-generated prompt word, and the step of generating the prompt word according to the picture or the picture element to mark the picture includes:

[0017] Generate prompt words according to the code to obtain the type of model to be trained;

[0018] When the model to be trained is a picture classification model, generating prompt words according to the picture to label the picture;

[0019] When the model to be trained is a picture element recognition model, the picture is labeled according to the picture elements corresponding to specific keywords among the keywords.

[0020] In one embodiment, the marking the picture according to the picture elements corresponding to the specific keywords among the keywords includes:

[0021] Determine the target keyword corresponding to the image element to be recognized by the model to be trained;

[0022] Matching the target keyword with the keyword in the crawler prompt word to determine a specific keyword in the crawler prompt word corresponding to the target keyword;

[0023] Comparing the image element corresponding to the specific keyword with the image to determine a target area in the image where the similarity between the image element corresponding to the specific keyword and the target area is greater than a preset similarity;

[0024] The edge of the target area is marked.

[0025] In one embodiment, the prompt word further includes generating a first prompt word for a model to be trained, and the method further includes:

[0026] Inputting the first prompt word into a target program or a target neural network model, wherein the target model is used to identify the intention corresponding to the prompt word and generate a code corresponding to the prompt word according to the recognition result;

[0027] Obtain the model code corresponding to the model to be trained output by the target program or target neural network model.

[0028] In one embodiment, the prompt word further includes a second prompt word for generating a training code and / or a test code, and the method further includes:

[0029] Inputting the second prompt word into a target program or a target neural network model;

[0030] Obtain training code and / or test code output by the target program or target neural network model.

[0031] In one embodiment, the method further comprises:

[0032] Executing the training code to train the model to be trained;

[0033] After the model to be trained is trained by the training code, the test code is executed to test the model to be trained.

[0034] The present application also provides a data annotation device, comprising:

[0035] The first acquisition module is used to acquire the requirement description information input by the user;

[0036] An identification module, used for identifying the demand description information to obtain prompt words in the demand description information, wherein the prompt words at least include crawler prompt words and picture generation prompt words;

[0037] An extraction module, used to extract keywords from the crawler prompt words;

[0038] A second acquisition module is used to acquire the image element corresponding to the keyword from a specific data source;

[0039] A generating module, used for combining the picture elements corresponding to the keywords to generate a picture corresponding to the picture generation prompt word;

[0040] The marking module is used to generate prompt words or the picture elements according to the picture to mark the picture.

[0041] In one embodiment, the generating module comprises:

[0042] A first combining submodule, used for combining the picture elements corresponding to the keywords to generate groups corresponding to the pictures, wherein each group contains at least one picture element corresponding to all the keywords;

[0043] A second combining submodule, used to combine the picture elements in each group to generate pictures corresponding to each group;

[0044] The calculation submodule is used to calculate the matching degree between the pictures corresponding to each group and the picture generation prompt word, and take the pictures whose matching degree with the picture generation prompt word is greater than a preset matching degree as the pictures corresponding to the picture generation prompt word.

[0045] In one embodiment, the prompt word further includes a code generation prompt word, and the marking module includes:

[0046] An acquisition submodule, used to acquire the type of model to be trained according to the code-generated prompt word;

[0047] A first annotation submodule is used to generate prompt words according to the picture to annotate the picture when the model to be trained is a picture classification model;

[0048] The second labeling submodule is used to label the picture according to the picture elements corresponding to specific keywords in the keywords when the model to be trained is a picture element recognition model.

[0049] In one embodiment, the second marking submodule is further used to:

[0050] Determine the target keyword corresponding to the image element to be recognized by the model to be trained;

[0051] Matching the target keyword with the keyword in the crawler prompt word to determine a specific keyword in the crawler prompt word corresponding to the target keyword;

[0052] Comparing the image element corresponding to the specific keyword with the image to determine a target area in the image where the similarity between the image element corresponding to the specific keyword and the target area is greater than a preset similarity;

[0053] The edge of the target area is marked.

[0054] In one embodiment, the prompt word further includes a first prompt word for generating a model to be trained, and the device further includes:

[0055] A first input module, used for inputting the first prompt word into a target program or a target neural network model, wherein the target model is used for identifying the intention corresponding to the prompt word and generating a code corresponding to the prompt word according to the recognition result;

[0056] The third acquisition module is used to obtain the model code corresponding to the model to be trained output by the target program or the target neural network model.

[0057] In one embodiment, the prompt word further includes a second prompt word for generating a training code and / or a test code, and the device further includes:

[0058] A second input module, used for inputting the second prompt word into a target program or a target neural network model;

[0059] The fourth acquisition module is used to obtain the training code and / or test code output by the target program or target neural network model.

[0060] In one embodiment, the apparatus further comprises:

[0061] A first execution module, used for executing the training code to train the model to be trained;

[0062] The second execution module is used to execute the test code to test the model to be trained after the model to be trained is trained by the training code.

[0063] The present application also provides a data annotation system, including:

[0064] at least one processor; and,

[0065] a memory communicatively connected to the at least one processor; wherein,

[0066] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to implement the data labeling method recorded in any of the above embodiments.

[0067] The present application also provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor corresponding to the data labeling system, the data labeling system can implement the data labeling method recorded in any of the above embodiments.

[0068] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings.

[0069] The technical solution of the present application is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The accompanying drawings are used to provide a further understanding of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings:

[0071] Figure 1 This is a flow chart of a data labeling method in one embodiment of the present application;

[0072] Figure 2 This is a block diagram of a data labeling device in one embodiment of the present application;

[0073] Figure 3 Schematic diagram of the hardware structure of a data annotation system in one embodiment of the present application. DETAILED DESCRIPTION

[0074] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application and are not used to limit the present application.

[0075] Figure 1 Flow chart of a data labeling method in one embodiment of the present application. Figure 1 As shown, the method can be implemented as the following steps S101-S106:

[0076] In step S101, the requirement description information input by the user is obtained;

[0077] In step S102, the demand description information is identified to obtain prompt words in the demand description information, wherein the prompt words at least include crawler prompt words and image generation prompt words;

[0078] In step S103, extracting keywords from the crawler prompt words;

[0079] In step S104, a picture element corresponding to the keyword is obtained from a specific data source;

[0080] In step S105, the picture elements corresponding to the keywords are combined to generate a picture corresponding to the picture generation prompt word;

[0081] In step S106, the picture is annotated with a prompt word generated from the picture or a picture element corresponding to a specific keyword among the keywords.

[0082] In this application, the requirement description information input by the user is obtained; for example, the requirement description information input by the user is as follows:

[0083] Requirement 1.

[0084] Please help me create a model which is able to recognize the image of a person from an image.

[0085] Requirement 2:

[0086] Please help me generate some training data. The training data is images, which can be crawled from XX image website. The image requirements are as follows:

[0087] This is a city landscape from the future, full of cyberpunk elements. The buildings surrounding the city are towering, and through the transparent glass curtain walls you can see the flickering light and futuristic line design. Augmented reality billboards and holographic signs can be seen everywhere in the city, which display various products and information in colorful lights. In this picture, there is a young girl sitting on a high stool, with one of her legs raised and slightly bent. She is wearing a dress inspired by cyberpunk style, embellished with neon-colored decorations, which is dazzling. Her style is fashionable and avant-garde, exuding a sense of the future.

[0088] The demand description information is identified to obtain prompt words in the demand description information, and the prompt words at least include crawler prompt words and picture generation prompt words; following the example of the demand description information, semantic recognition can be performed when obtaining prompt words, and useless information can be eliminated, such as some information describing the scenery, modifiers, modal particles, lyrical sentences, etc., and only the core content is retained. The prompt words finally obtained are as follows:

[0089] Crawler prompt words: crawl the image elements of future city, cyberpunk, young girl, cyberpunk style clothing, and high stool from XX picture website;

[0090] Image generation prompt words: Generate images containing future cities, augmented reality billboards and holographic signs, cyberpunk style, and young girls sitting on high stools;

[0091] Code generation prompt: Generate a model for identifying human elements in images.

[0092] Extract keywords from the crawler prompt words; specifically, the keywords used to crawl image elements can be determined through semantic recognition, and specifically, the words used to describe entities after "crawl" can be identified, for example, future city, cyberpunk, young girl, cyberpunk style clothing, and high stools are used as keywords.

[0093] Obtain image elements corresponding to the keywords from a specific data source; for example, using future city, cyberpunk, young girl, cyberpunk style clothing, and high stool as keywords, retrieve elements matching the keywords from XX image website.

[0094] The picture elements corresponding to the keywords are combined to generate pictures corresponding to the picture generation prompt words; specifically, the picture elements corresponding to the keywords are combined to generate groups corresponding to the pictures, wherein each group contains at least one picture element corresponding to all the keywords; the picture elements in each group are respectively combined to generate pictures corresponding to each group; the matching degree between the pictures corresponding to each group and the picture generation prompt words is calculated, and the pictures whose matching degree with the picture generation prompt words is greater than a preset matching degree are used as pictures corresponding to the picture generation prompt words.

[0095] The image is labeled according to the prompt words generated by the image or the image elements; specifically, the type of model to be trained is obtained according to the prompt words generated by the code; when the model to be trained is a picture classification model, the image is labeled according to the prompt words generated by the image; when the model to be trained is a picture element recognition model, the image is labeled according to the picture elements corresponding to specific keywords among the keywords.

[0096] The model to be trained can be an image classification model, for example, the model to be trained is a model for identifying cyberpunk style images, and another example is a model for identifying future city type images. Then, the image can be directly annotated according to the prompt words generated from the image, that is, the generated image can be annotated as an image containing a future city, augmented reality billboards and holographic signs, cyberpunk style, and a young girl sitting on a high stool; or the prompt words generated from the image can be simplified according to the semantic recognition model, and simplified to an image containing a future city or a cyberpunk style image.

[0097] The model to be trained may also be a picture element recognition model. For example, the model to be trained may be a model for recognizing human elements in an image. When the picture is annotated according to the picture elements corresponding to specific keywords among the keywords, the target keyword corresponding to the picture element to be recognized by the model to be trained is determined; the target keyword is matched with the keyword in the crawler prompt word to determine the specific keyword in the crawler prompt word corresponding to the target keyword; the picture element corresponding to the specific keyword is compared with the picture to determine a target area in the picture where the similarity of the picture element corresponding to the specific keyword is greater than a preset similarity; and the edge of the target area is annotated.

[0098] For example, when the model to be trained is a model for identifying human elements in an image, it can be determined that the human elements in the generated training set are young girls. Then, the target keyword corresponding to the image element to be identified by the model to be trained is the keyword "young girl" in the crawler prompt word, and all elements searched out based on this keyword are human elements to be identified. Since the operations of obtaining the image elements corresponding to the keywords from a specific data source and the operations of annotating the image according to the image elements corresponding to specific keywords in the keywords are both performed by the execution subject of this application, the step of obtaining the image elements can be traced back to determine the image elements corresponding to the keyword "young girl", and the image elements corresponding to the keyword "young girl" can be compared with the image as the annotation basis, so as to determine whether there is a target area in the image whose similarity with the image elements corresponding to the keyword "young girl" is greater than the preset similarity. If so, the edge of the target area is annotated. Specifically, a contour line can be drawn on the edge of the target area to complete the annotation of the image.

[0099] In this application, after completing the annotation of the image, the training model can be trained and tested. The training model can be trained and tested in a variety of ways. The following are two ways to train and test the training model:

[0100] Method 1

[0101] The prompt words include a second prompt word for generating a training code and / or a test code, inputting the second prompt word into a target program or a target neural network model; obtaining the training code and / or the test code output by the target program or the target neural network model; executing the training code to train the model to be trained; after the model to be trained is trained by the training code, executing the test code to test the model to be trained.

[0102] Method 2

[0103] The annotated images are divided into a training set and a test set; the training set is input into the model to be trained, and the weights and parameters of the model to be trained are adjusted through the back propagation algorithm and the optimizer to obtain the trained model; the trained model is evaluated using the test set, and the accuracy and recall rate of the trained model are calculated; when the accuracy of the trained model is greater than the preset accuracy, it is determined that the trained model test has passed.

[0104] The beneficial effects of the present application are: it is capable of obtaining crawler prompt words and image generation prompt words according to the demand description information input by the user, and obtaining image elements corresponding to the keywords in the crawler prompt words from a specific data source; combining the image elements corresponding to the keywords to generate images corresponding to the image generation prompt words; since the image generation prompt words have been obtained through the demand description information, the images can be annotated through the image generation prompt words, and since the elements of the generated images are the image elements corresponding to the keywords in the crawler prompt words, the images can be annotated directly based on the image elements corresponding to the specific keywords, thereby saving labor costs, and since the annotation results are not affected by the subjective judgment of the annotator, the stability and objectivity of the annotation results are improved.

[0105] In one embodiment, the above step S105 may be implemented as the following steps A1-A3:

[0106] In step A1, the picture elements corresponding to the keywords are combined to generate groups corresponding to the pictures, wherein each group contains at least one picture element corresponding to all the keywords;

[0107] In step A2, the picture elements in each group are respectively combined to generate pictures corresponding to each group;

[0108] In step A3, the matching degree between the pictures corresponding to each group and the picture generation prompt word is calculated, and the pictures whose matching degree with the picture generation prompt word is greater than a preset matching degree are taken as the pictures corresponding to the picture generation prompt word.

[0109] For example, extract at least one image element from each element matched by a keyword. For example, if there are 5 keywords, at least 5 image elements can be extracted. Put the image elements extracted each time into the same group, so that N groups can be combined. Combine the image elements of each group to form N images. When extracting at least one element from each element matched by a keyword, you can extract it randomly or according to specific rules, for example, you can extract it in order. When crawling data, the degree of matching with the crawl prompt word is usually given. Therefore, you can also extract according to the degree of matching between the element and the crawl prompt word.

[0110] In one embodiment, the prompt word further includes a code generation prompt word, and the generation process of the model to be trained can be implemented as the following steps B1-B2:

[0111] In step B1, a code generation prompt word is input into a target program or a target neural network model for automatically generating model code according to a natural description language;

[0112] In step B2, the model to be trained output by the target program or target neural network model is obtained.

[0113] In this embodiment, the code generation prompt is input into the target program or the target neural network model used to automatically generate the model code according to the natural description language. The model code is to encapsulate the code modularly and automatically. By inputting the code generation prompt into the target program, the corresponding code can be automatically generated. Compared with the manual code, the code is more standardized, the style is more unified, and the accuracy is higher. Then, the model to be trained output by the target program or the target neural network model can be obtained, that is, the corresponding model code can be obtained.

[0114] In one embodiment, the prompt word further includes a code generation prompt word, and the above step S106 can be implemented as the following steps C1-C3:

[0115] In step C1, the type of model to be trained is obtained according to the code generation prompt word;

[0116] In step C2, when the model to be trained is a picture classification model, the picture is annotated according to the prompt words generated from the picture;

[0117] In step C3, when the model to be trained is a picture element recognition model, the picture is labeled according to the picture elements corresponding to specific keywords among the keywords.

[0118] The model to be trained can be an image classification model, for example, the model to be trained is a model for identifying cyberpunk style images, and another example is a model for identifying future city type images. Then, the image can be directly annotated according to the prompt words generated from the image, that is, the generated image can be annotated as an image containing a future city, augmented reality billboards and holographic signs, cyberpunk style, and a young girl sitting on a high stool; or the prompt words generated from the image can be simplified according to the semantic recognition model, and simplified to an image containing a future city or a cyberpunk style image.

[0119] The model to be trained may also be a picture element recognition model. For example, the model to be trained may be a model for recognizing human elements in an image. When the picture is annotated according to the picture elements corresponding to specific keywords among the keywords, the target keyword corresponding to the picture element to be recognized by the model to be trained is determined; the target keyword is matched with the keyword in the crawler prompt word to determine the specific keyword in the crawler prompt word corresponding to the target keyword; the picture element corresponding to the specific keyword is compared with the picture to determine a target area in the picture where the similarity of the picture element corresponding to the specific keyword is greater than a preset similarity; and the edge of the target area is annotated.

[0120] In one embodiment, the above step C3 may be implemented as the following steps C31-C34:

[0121] In step C31, the target keyword corresponding to the image element to be identified by the model to be trained is determined;

[0122] In step C32, the target keyword is matched with the keyword in the crawler prompt word to determine a specific keyword in the crawler prompt word corresponding to the target keyword;

[0123] In step C33, the picture element corresponding to the specific keyword is compared with the picture to determine a target area in the picture where the similarity of the picture element corresponding to the specific keyword is greater than a preset similarity;

[0124] In step C34, the edge of the target area is marked.

[0125] Continuing with the above example, when the model to be trained is a model for identifying human elements in an image, it can be determined that the human elements in the generated training set are young girls. Then, the target keyword corresponding to the image element to be identified by the model to be trained is the keyword "young girl" in the crawler prompt words, and all elements searched out based on this keyword are human elements to be identified. Since the operations of obtaining the image elements corresponding to the keywords from a specific data source and the operations of annotating the image according to the image elements corresponding to specific keywords in the keywords are both performed by the execution subject of this application, the step of obtaining the image elements can be traced back to determine the image elements corresponding to the keyword "young girl", and the image elements corresponding to the keyword "young girl" can be compared with the image as the annotation basis, so as to determine whether there is a target area in the image whose similarity with the image elements corresponding to the keyword "young girl" is greater than the preset similarity. If so, the edge of the target area is annotated. Specifically, a contour line can be drawn on the edge of the target area to complete the annotation of the image.

[0126] In one embodiment, the prompt word further includes generating a first prompt word for a model to be trained. The method can also be implemented as the following steps D1-D2:

[0127] In step D1, the first prompt word is input into a target program or a target neural network model, wherein the target model is used to identify the intention corresponding to the prompt word and generate a code corresponding to the prompt word according to the recognition result;

[0128] In step D2, the model code corresponding to the model to be trained output by the target program or the target neural network model is obtained.

[0129] In this embodiment, the first prompt word is input into the target program or the target neural network model, wherein the target model is used to identify the intent corresponding to the prompt word and generate the code corresponding to the prompt word according to the recognition result. There are many methods for identifying the intent of the corresponding prompt word. For example, the correspondence between multiple prompt words and standard prompt words can be stored in a preset database, and the corresponding intent can be obtained by matching the prompt words with the preset database; the prompt words can also be matched in a preset search engine to obtain the corresponding intent. In addition, the model code corresponding to the standard prompt word is also stored in the preset database, and the code corresponding to the prompt word can be generated according to the recognition result; of course, a standard model code template can also be pre-stored, and the corresponding part of the model code template is standardized and supplemented according to the corresponding prompt word, so as to obtain the corresponding model code. Then, the model code corresponding to the model to be trained output by the target program or the target neural network model can be obtained.

[0130] In one embodiment, the prompt word further includes a second prompt word for generating a training code and / or a test code. The method can also be implemented as the following steps E1-E2:

[0131] In step E1, the second prompt word is input into a target program or a target neural network model;

[0132] In step E2, the training code and / or test code output by the target program or target neural network model is obtained.

[0133] In one embodiment, the method can also be implemented as the following steps F1-F2:

[0134] In step F1, executing the training code to train the model to be trained;

[0135] In step F2, after the model to be trained is trained by the training code, the test code is executed to test the model to be trained.

[0136] In one embodiment, the training model is trained through the following steps G1-G2:

[0137] In step G1, the annotated images are divided into a training set and a test set;

[0138] In step G2, the training set, the test set and the model to be trained are imported into a specific hardware device, so as to train the model to be trained through the specific hardware device.

[0139] In one embodiment, the above step G2 may be implemented as the following steps G21-G23:

[0140] In step G21, the training set is input into the model to be trained, and the weights and parameters of the model to be trained are adjusted by the back propagation algorithm and the optimizer to obtain the trained model;

[0141] In step G22, the trained model is evaluated using the test set to calculate the precision and recall of the trained model;

[0142] In step G23, when the accuracy of the trained model is greater than the preset accuracy, it is determined that the trained model test has passed.

[0143] Figure 2 FIG. 1 is a schematic diagram of a data labeling device in an embodiment of the present application. Figure 2 As shown, the device comprises:

[0144] The first acquisition module 201 is used to acquire the demand description information input by the user;

[0145] An identification module 202 is used to identify the demand description information to obtain prompt words in the demand description information, wherein the prompt words at least include crawler prompt words and image generation prompt words;

[0146] An extraction module 203 is used to extract keywords from the crawler prompt words;

[0147] A second acquisition module 204 is used to acquire the image element corresponding to the keyword from a specific data source;

[0148] A generating module 205, configured to combine the picture elements corresponding to the keywords to generate a picture corresponding to the picture generation prompt word;

[0149] The marking module 206 is used to generate prompt words or the picture elements according to the picture to mark the picture.

[0150] In one embodiment, the generating module comprises:

[0151] A first combining submodule, used for combining the picture elements corresponding to the keywords to generate groups corresponding to the pictures, wherein each group contains at least one picture element corresponding to all the keywords;

[0152] A second combining submodule, used to combine the picture elements in each group to generate pictures corresponding to each group;

[0153] The calculation submodule is used to calculate the matching degree between the pictures corresponding to each group and the picture generation prompt word, and take the pictures whose matching degree with the picture generation prompt word is greater than a preset matching degree as the pictures corresponding to the picture generation prompt word.

[0154] In one embodiment, the prompt word further includes a code generation prompt word, and the marking module includes:

[0155] An acquisition submodule, used to acquire the type of model to be trained according to the code-generated prompt word;

[0156] A first annotation submodule is used to generate prompt words according to the picture to annotate the picture when the model to be trained is a picture classification model;

[0157] The second labeling submodule is used to label the picture according to the picture elements corresponding to specific keywords in the keywords when the model to be trained is a picture element recognition model.

[0158] In one embodiment, the second marking submodule is further used to:

[0159] Determine the target keyword corresponding to the image element to be recognized by the model to be trained;

[0160] Matching the target keyword with the keyword in the crawler prompt word to determine a specific keyword in the crawler prompt word corresponding to the target keyword;

[0161] Comparing the image element corresponding to the specific keyword with the image to determine a target area in the image where the similarity between the image element corresponding to the specific keyword and the target area is greater than a preset similarity;

[0162] The edge of the target area is marked.

[0163] In one embodiment, the prompt word further includes a first prompt word for generating a model to be trained, and the device further includes:

[0164] A first input module, used for inputting the first prompt word into a target program or a target neural network model, wherein the target model is used for identifying the intention corresponding to the prompt word and generating a code corresponding to the prompt word according to the recognition result;

[0165] The third acquisition module is used to obtain the model code corresponding to the model to be trained output by the target program or the target neural network model.

[0166] In one embodiment, the prompt word further includes a second prompt word for generating a training code and / or a test code, and the device further includes:

[0167] A second input module, used for inputting the second prompt word into a target program or a target neural network model;

[0168] The fourth acquisition module is used to obtain the training code and / or test code output by the target program or target neural network model.

[0169] In one embodiment, the apparatus further comprises:

[0170] A first execution module, used for executing the training code to train the model to be trained;

[0171] The second execution module is used to execute the test code to test the model to be trained after the model to be trained is trained by the training code.

[0172] Figure 3 FIG. 1 is a schematic diagram of the hardware structure of a data annotation system in one embodiment of the present application. Figure 3 As shown, the data annotation system includes:

[0173] at least one processor 320; and,

[0174] A memory 304 in communication with the at least one processor 320; wherein,

[0175] The memory 304 stores instructions that can be executed by the at least one processor 320, and the instructions are executed by the at least one processor 320 to implement the data labeling method recorded in any of the above embodiments.

[0176] Reference Figure 3 The data annotation system 300 may include one or more of the following components: a processing component 302 , a memory 304 , a power component 306 , a multimedia component 308 , an audio component 310 , an input / output (I / O) interface 312 , a sensor component 314 , and a communication component 316 .

[0177] The processing component 302 generally controls the overall operation of the data annotation system 300. The processing component 302 may include one or more processors 320 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 302 may include one or more modules to facilitate the interaction between the processing component 302 and other components. For example, the processing component 302 may include a multimedia module to facilitate the interaction between the multimedia component 308 and the processing component 302.

[0178] The memory 304 is configured to store various types of data to support the operation of the data annotation system 300. Examples of such data include instructions for any application or method operating on the data annotation system 300, such as text, pictures, videos, etc. The memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0179] The power supply component 306 provides power to various components of the data annotation system 300. The power supply component 306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the data annotation system 300.

[0180] The multimedia component 308 includes a screen that provides an output interface between the data annotation system 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 308 may also include a front camera and / or a rear camera. When the data annotation system 300 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0181] The audio component 310 is configured to output and / or input audio signals. For example, the audio component 310 includes a microphone (MIC), and when the data annotation system 300 is in an operation mode, such as an alarm mode, a recording mode, a speech recognition mode, and a speech output mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 304 or sent via the communication component 316. In some embodiments, the audio component 310 also includes a speaker for outputting audio signals.

[0182] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0183] The sensor assembly 314 includes one or more sensors for providing various aspects of status assessment for the data annotation system 300. For example, the sensor assembly 314 may include a sound sensor. In addition, the sensor assembly 314 may detect the open / closed state of the data annotation system 300, the relative positioning of the components, such as the display and keypad of the data annotation system 300, and the sensor assembly 314 may also detect the operating state of the data annotation system 300 or a component of the data annotation system 300, such as the operating state of the wind distribution plate, the structural state, the operating state of the discharge scraper, etc., the orientation or acceleration / deceleration of the data annotation system 300 and the temperature change of the data annotation system 300. The sensor assembly 314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 314 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 314 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, a material stacking thickness sensor or a temperature sensor.

[0184] The communication component 316 is configured to enable the data annotation system 300 to provide the ability to communicate with other devices and cloud platforms in a wired or wireless manner. The data annotation system 300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0185] In an exemplary embodiment, the data labeling system 300 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the data labeling method described in any of the above embodiments.

[0186] The present application also provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor corresponding to the data labeling system, the data labeling system can implement the data labeling method recorded in any of the above embodiments.

[0187] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0188] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0189] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0190] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0191] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data labeling method, characterized in that: include: Get the requirement description information entered by the user; Identify the requirement description information to obtain prompt words in the requirement description information, where the prompt words at least include crawler prompt words and image generation prompt words; Extracting keywords from the crawler prompt words; Acquire image elements corresponding to the keywords from a specific data source; Combining the image elements corresponding to the keywords to generate an image corresponding to the image generation prompt word; Generate prompt words or the picture elements according to the picture to mark the picture.

2. The method according to claim 1, characterized in that The step of combining the picture elements corresponding to the keywords to generate a picture corresponding to the picture generation prompt word includes: Grouping the image elements corresponding to the keywords, wherein each group contains at least one image element corresponding to all the keywords; Combine the image elements in each group to generate images corresponding to each group; The matching degree between the pictures corresponding to each group and the picture generation prompt word is calculated, and the pictures whose matching degree with the picture generation prompt word is greater than a preset matching degree are taken as the pictures corresponding to the picture generation prompt word.

3. The method according to claim 1, characterized in that The prompt word also includes a code-generated prompt word, and the generating of the prompt word according to the picture or the picture element to mark the picture includes: Generate prompt words according to the code to obtain the type of model to be trained; When the model to be trained is a picture classification model, generating prompt words according to the picture to label the picture; When the model to be trained is a picture element recognition model, the picture is labeled according to the picture elements corresponding to specific keywords among the keywords.

4. The method according to claim 3, characterized in that The marking the picture according to the picture elements corresponding to the specific keywords among the keywords includes: Determine the target keyword corresponding to the image element to be recognized by the model to be trained; Matching the target keyword with the keyword in the crawler prompt word to determine a specific keyword in the crawler prompt word corresponding to the target keyword; Comparing the image element corresponding to the specific keyword with the image to determine a target area in the image where the similarity between the image element corresponding to the specific keyword and the target area is greater than a preset similarity; The edge of the target area is marked.

5. The method according to claim 1, characterized in that The prompt word also includes generating a first prompt word for a model to be trained, and the method further includes: Inputting the first prompt word into a target program or a target neural network model, wherein the target model is used to identify the intention corresponding to the prompt word and generate a code corresponding to the prompt word according to the recognition result; Obtain the model code corresponding to the model to be trained output by the target program or target neural network model.

6. The method according to claim 5, characterized in that The prompt word also includes a second prompt word for generating a training code and / or a test code, and the method further includes: Inputting the second prompt word into a target program or a target neural network model; Obtain training code and / or test code output by the target program or target neural network model.

7. The method according to claim 6, characterized in that The method further comprises: Executing the training code to train the model to be trained; After the model to be trained is trained by the training code, the test code is executed to test the model to be trained.

8. A data labeling device, characterized in that: include: The first acquisition module is used to acquire the requirement description information input by the user; An identification module, used for identifying the demand description information to obtain prompt words in the demand description information, wherein the prompt words at least include crawler prompt words and picture generation prompt words; An extraction module, used to extract keywords from the crawler prompt words; A second acquisition module is used to acquire the image element corresponding to the keyword from a specific data source; A generating module, used for combining the picture elements corresponding to the keywords to generate a picture corresponding to the picture generation prompt word; The marking module is used to generate prompt words or the picture elements according to the picture to mark the picture.

9. A data annotation system, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to implement the data labeling method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor corresponding to the data labeling system, the data labeling system can implement the data labeling method as described in any one of claims 1 to 7.

Images