Data annotation method, device and readable storage medium

Abstract

The present invention provides a data annotation method, device and readable storage medium. The data annotation method is used on the client side, including: issuing a data set creation request to a data service; uploading a file to an object storage service; after uploading the file and annotation information to the object storage service, issuing a data verification request to the data service. In the technical solution of the present invention, the file upload process does not involve code, and can be in a graphical, wizard-like form to simplify the operation process. Users do not need to master other knowledge, and a low threshold and wizard-like interaction based on a fixed process are achieved. It is more concise and friendly for traditional industry users who started late and have poor foundations, and better improves the user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of data annotation, and in particular to a data annotation method, device and readable storage medium. Background Art

[0002] Data labeling is the process of processing unprocessed primary data, including voice, images, text, video, etc., and converting them into machine-readable information. Raw data is generally obtained through data collection, and subsequent data labeling is equivalent to processing the data and then transmitting it to artificial intelligence algorithms and models for call.

[0003] The current AI dataset annotation has the following problems:

[0004] (1) The annotation format is not unified, and format conversion is required when using it, which requires a certain amount of code and manual work. The annotation strategy framework and label template reuse process are cumbersome.

[0005] (2) The data labeling, training, and model validation processes are separated and cannot be automated.

[0006] (3) In scenarios with large data sets, the performance of data labeling and model training is poor.

[0007] (4) How to store unstructured data and performance issues. Summary of the invention

[0008] The present invention aims to solve or improve at least one of the above-mentioned technical problems.

[0009] To this end, a first objective of the present invention is to provide a data labeling method.

[0010] A second object of the present invention is to provide a data labeling method.

[0011] The third object of the present invention is to provide a data labeling method.

[0012] A fourth object of the present invention is to provide a data labeling method.

[0013] A fifth object of the present invention is to provide a data labeling device.

[0014] A sixth object of the present invention is to provide a readable storage medium.

[0015] To achieve the first purpose of the present invention, the technical solution of the present invention provides a data labeling method for use on the client side, including: issuing a data set creation request to a data service; uploading a file to an object storage service; and issuing a data verification request to the data service after uploading the file and labeling information to the object storage service.

[0016] In this embodiment, the file upload process does not involve code and can be in a graphical, wizard-like form to simplify the operation process. Users do not need to master other knowledge, and a low threshold is achieved. Wizard-style interaction based on a fixed process is achieved. It is more concise and friendly for traditional industry users who started late and have poor foundations, and better improves the user experience.

[0017] In addition, the technical solution provided by the present invention may also have the following additional technical features:

[0018] In the above technical solution, uploading files to the object storage service specifically includes: selecting an upload folder, making batch upload requests to the data service for pre-signed links, and then uploading files to the object storage service.

[0019] In this implementation, first select the upload folder, then make a batch request to the data service to upload the pre-signed link, and then upload the file to the object storage service. The pre-signed link can better verify the identity, ensure the security of the data, and meet the data security requirements in multi-tenant scenarios.

[0020] In any of the above technical solutions, the data labeling method also includes: batch requesting data set labeling information and file download pre-signed links from the data service; receiving the download link returned by the data service; downloading files in parallel according to the download link; and training the model in batches based on the files and labeling information.

[0021] In this embodiment, after obtaining the file and annotation information, the model can be trained directly in batches, and data annotation and model training are seamlessly integrated to achieve integrated data annotation, training and model verification.

[0022] To achieve the second purpose of the present invention, the technical solution of the present invention provides a data labeling method for the data service side, including: receiving a request to create a data set, and saving metadata information to a database according to the data set name; receiving a data verification request issued by a client, and performing data set verification on files stored in an object storage service; receiving an unlabeled file list request issued by a labeling platform, searching for unlabeled files according to the metadata information, obtaining unlabeled set files, and returning the unlabeled set files to the labeling platform.

[0023] In this embodiment, the data service implements data set verification of files stored in the object storage service, and can convert third-party format annotation information into its own format annotation information. The operation is simple and a certain degree of automation can be achieved.

[0024] In addition, the technical solution provided by the present invention may also have the following additional technical features:

[0025] In the above technical solution, data set verification is performed on the files stored in the object storage service, specifically including: verifying whether the file exists; based on the existence of the file and annotation information, checking whether the file includes exchangeable image file format information; based on the file including exchangeable image file format information, deleting the exchangeable image file format information of the file; establishing a mapping relationship between third-party annotation information and the annotated file, and based on the mapping relationship, converting the third-party annotation information into annotation information in its own format.

[0026] This embodiment automatically performs conversion and labeling by establishing a mapping relationship, which is more friendly to traditional industry users who started late and have poor foundations.

[0027] In any of the above technical solutions, converting the third-party annotation information into the annotation information in the own format specifically includes: starting a background scheduled task framework job to convert the third-party format annotation information into the annotation information in the own format.

[0028] In this embodiment, the scheduled task framework uses Quartz, and the scheduled task framework can effectively manage tasks, including the start time of tasks, the uniqueness of tasks, strategies after task failure, etc. In addition, the scheduled task framework can also split tasks for large data sets and perform parallel verification.

[0029] In any of the above technical solutions, the data annotation method also includes: receiving data set annotation information and a file download pre-signed link request, obtaining a list of files and annotated files under the path corresponding to the data set based on metadata information, generating a download link, and returning the download link to the client.

[0030] This embodiment uses a pre-signed link method to better verify identity, ensure data security, and meet the data security requirements in multi-tenant scenarios.

[0031] To achieve the third purpose of the present invention, the technical solution of the present invention provides a data labeling method for use on the object storage service side, including: receiving and storing metadata information; receiving and storing files; receiving and storing labeling information.

[0032] In this embodiment, the object storage service can use the current mainstream open source framework MinIO (open source distributed file storage system).

[0033] In addition, the technical solution provided by the present invention may also have the following additional technical features:

[0034] In the above technical solution, receiving and storing the annotation information specifically includes: verifying the files corresponding to the annotation information through batch query of the database, and then saving the annotation information.

[0035] In this embodiment, the file corresponding to the annotation information is first verified, and after the verification is passed, the annotation information is saved. This embodiment implements the above process through batch query of the database, and the method is simple and easy to implement.

[0036] To achieve the fourth purpose of the present invention, the technical solution of the present invention provides a data labeling method for the labeling platform side, including: requesting a list of unlabeled files from the data service; receiving and verifying the labeling set file returned by the data service, parsing the labeling set file, adapting the labeling format, and labeling the labeling set file; sending the labeling information to the object storage service.

[0037] In this embodiment, the annotation platform has the characteristics of high compatibility and supports the current mainstream annotation formats.

[0038] To achieve the fifth purpose of the present invention, the technical solution of the present invention provides a data labeling device, including: a memory and a processor, the memory stores a program or instruction, and the processor executes the program or instruction; wherein, when the processor executes the program or instruction, it implements the steps of the data labeling method of any technical solution of the present invention.

[0039] The data labeling device provided by the present technical solution implements the steps of the data labeling method of any technical solution of the present invention, and thus has all the beneficial effects of the data labeling method of any technical solution of the present invention, which will not be repeated here.

[0040] To achieve the sixth purpose of the present invention, the technical solution of the present invention provides a readable storage medium, which stores a program or instruction. When the program or instruction is executed, the steps of the data marking method of any of the above technical solutions are implemented.

[0041] The readable storage medium provided by the present technical solution implements the steps of the data labeling method of any technical solution of the present invention, and thus has all the beneficial effects of the data labeling method of any technical solution of the present invention, which will not be repeated here.

[0042] Additional aspects and advantages of the present invention will become apparent from the following description or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0044] Figure 1 This is one of the flow charts of the data annotation method according to one embodiment of the present invention;

[0045] Figure 2 This is a second flow chart of a data labeling method according to an embodiment of the present invention;

[0046] Figure 3 This is a third flow chart of a data labeling method according to an embodiment of the present invention;

[0047] Figure 4 This is a fourth flow chart of a data labeling method according to an embodiment of the present invention;

[0048] Figure 5 This is a fifth flow chart of a data labeling method according to an embodiment of the present invention;

[0049] Figure 6 This is a sixth flow chart of a data labeling method according to an embodiment of the present invention;

[0050] Figure 7 This is a seventh flow chart of a data labeling method according to an embodiment of the present invention;

[0051] Figure 8 This is an eighth flow chart of a data labeling method according to an embodiment of the present invention;

[0052] Fig. 9 This is a ninth flow chart of a method for labeling data according to an embodiment of the present invention;

[0053] Fig.10 This is a tenth flow chart of a data labeling method according to an embodiment of the present invention;

[0054] Fig.11 A schematic block diagram of a data labeling device according to an embodiment of the present invention;

[0055] Fig.12 This is one of the data annotation schematic diagrams according to an embodiment of the present invention;

[0056] Fig.13 This is a second diagram of data annotation according to an embodiment of the present invention.

[0057] in, Figures 11 to 13 The corresponding relationship between the reference numerals and the component names is as follows:

[0058] 100: Browser workshop, 102: Data service, 104: Object storage service, 106: MinIO, 108: OSS, 110: MySQL, 112: Annotation platform, 114: Python SDK, 200: Data annotation device, 210: Storage, 220: Processor. DETAILED DESCRIPTION

[0059] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0060] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0061] Refer to the following Figures 1 to 13 The present invention describes a method, device and readable storage medium for labeling data according to some embodiments of the present invention.

[0062] Embodiment 1:

[0063] like Figure 1 As shown, this embodiment provides a data marking method, which is used on the client side and includes the following steps:

[0064] Step S102, sending a request to create a data set to a data service;

[0065] Step S104, uploading the file to the object storage service;

[0066] Step S106, after uploading the file and annotation information to the object storage service, a data verification request is sent to the data service.

[0067] In this embodiment, the client sends a request to create a dataset to the data service. The data service receives the request to create a dataset and saves metadata information to the object storage service according to the dataset name. The client uploads the file to the object storage service. After uploading the file and annotation information to the object storage service, the client sends a data verification request to the data service. The data service receives the data verification request sent by the client and performs dataset verification on the file stored in the object storage service.

[0068] In this embodiment, the file supports all unstructured data, including but not limited to: pictures, videos, audio, text, etc., and can store and manage massive amounts of unstructured data.

[0069] In this embodiment, the file upload process does not involve code and can be in a graphical, wizard-like form to simplify the operation process. Users do not need to master other knowledge, and a low threshold is achieved. Wizard-style interaction based on a fixed process is achieved. It is more concise and friendly for traditional industry users who started late and have poor foundations, and better improves the user experience.

[0070] In this embodiment, the file processing process can be performed in parallel to achieve more efficient data labeling.

[0071] Embodiment 2:

[0072] like Figure 2 As shown, this embodiment provides a data annotation method. In addition to the technical features of the above embodiments, this embodiment further includes the following technical features:

[0073] Uploading a file to the object storage service includes the following steps:

[0074] Step S202, select an upload folder, send a batch upload pre-signed link request to the data service, and then upload the file to the object storage service.

[0075] In this implementation, first select the upload folder, then make a batch request to the data service to upload the pre-signed link, and then upload the file to the object storage service. The pre-signed link can better verify the identity, ensure the security of the data, and meet the data security requirements in multi-tenant scenarios.

[0076] Embodiment 3:

[0077] like Figure 3 As shown, this embodiment provides a data annotation method. In addition to the technical features of the above embodiments, this embodiment further includes the following technical features:

[0078] The data annotation method also includes the following steps:

[0079] Step S302, batch requesting dataset annotation information and file download pre-signed links from the data service;

[0080] Step S304, receiving the download link returned by the data service;

[0081] Step S306, downloading files in parallel according to the download links;

[0082] Step S308: training the model in batches based on the files and annotation information.

[0083] In this embodiment, after the data is labeled, the data service can be requested in batches for the data set labeling information and the file download pre-signed link, and the download link returned by the data service is received, and then the file is downloaded. After the file is downloaded, the model is trained in batches. Among them, this embodiment can better verify the identity and ensure the security of the data through the pre-signed link, and meet the data security requirements in multi-tenant scenarios. In this embodiment, after obtaining the file and labeling information, the model can be trained directly in batches, and the data labeling and model training are seamlessly integrated to achieve integrated data labeling, training and model verification, and realize automation. Moreover, in the scenario of large data sets, data labeling and model training meet the performance requirements of massive data in labeling and training scenarios.

[0084] This embodiment can be applied to model training in a short time to create real AI commercial value.

[0085] Embodiment 4:

[0086] like Figure 4 As shown, this embodiment provides a data annotation method for a data service side, comprising the following steps:

[0087] Step S402, receiving a request to create a data set, and saving metadata information to a database according to the data set name;

[0088] Step S404, receiving a data verification request from the client, and performing data set verification on the file stored in the object storage service;

[0089] Step S406, receiving the unlabeled file list request sent by the labeling platform, searching for unlabeled files according to metadata information, obtaining unlabeled set files, and returning the unlabeled set files to the labeling platform.

[0090] In this embodiment, the client sends a request to create a data set to the data service, the data service receives the request to create a data set, saves metadata information to the database of the object storage service according to the data set name, the client sends a data verification request to the data service, the data service receives the data verification request sent by the client, and performs data set verification on the files stored in the database of the object storage service. The labeling platform requests the data service for a list of unlabeled files, the data service receives the request for a list of unlabeled files sent by the labeling platform, searches for unlabeled files according to the metadata information, obtains unlabeled set files, and returns the unlabeled set files to the labeling platform.

[0091] In this embodiment, the data service implements data set verification of files stored in the database of the object storage service, and can convert third-party format annotation information into its own format annotation information. The operation is simple and a certain degree of automation can be achieved.

[0092] Embodiment 5:

[0093] like Figure 5 As shown, this embodiment provides a data annotation method. In addition to the technical features of the above embodiments, this embodiment further includes the following technical features:

[0094] Perform data set verification on files stored in the object storage service, including the following steps:

[0095] Step S502, verifying whether the file exists;

[0096] Step S504, based on the existence of the file and the annotation information, checking whether the file includes exchangeable image file format information;

[0097] Step S506, based on the fact that the file includes the exchangeable image file format information, deleting the exchangeable image file format information of the file;

[0098] Step S508, establishing a mapping relationship between the third-party annotation information and the annotated file, and based on the mapping relationship, converting the third-party annotation information into annotation information in its own format;

[0099] Step S510, counting the number of labels in the statistical data set;

[0100] Step S512: based on the file being a picture or a video, add a thumbnail of the picture or video.

[0101] In this embodiment, data set verification is performed on the files stored in the database of the object storage service, including verifying whether the files exist and checking whether the files include exchangeable image file format information. If the files have annotation information, the annotation information at this time is third-party annotation information and may not meet the requirements of the proprietary format. A mapping relationship between the third-party annotation information and the annotated files can be established, and then based on the mapping relationship, the third-party annotation information is converted into proprietary format annotation information. Some of the data can have automatic annotation capabilities. When facing enterprise operation scenarios such as marketing, procurement, office, HR management, supply chain, finance, and manufacturing, by establishing a mapping relationship, conversion and annotation are automatically performed, which is more friendly to traditional industry users who started late and have poor foundations.

[0102] In this embodiment, after establishing a mapping relationship between third-party annotation information and the annotated file, and converting the third-party annotation information into annotation information in its own format based on the mapping relationship, the number of dataset labels can also be counted. By counting the number of dataset labels, it can be determined whether the number of each label in the dataset is balanced, thereby avoiding imbalance in training data.

[0103] In this embodiment, thumbnails of pictures and videos can also be generated, and the thumbnails are used to display the file details on the dataset details page.

[0104] Embodiment 6:

[0105] like Figure 6 As shown, this embodiment provides a data annotation method. In addition to the technical features of the above embodiments, this embodiment further includes the following technical features:

[0106] Convert third-party annotation information into annotation information in its own format, including the following steps:

[0107] Step S602, starting a background scheduled task framework job to convert third-party format annotation information into self-owned format annotation information.

[0108] In this embodiment, the scheduled task framework uses Quartz, and the scheduled task framework can effectively manage tasks, including the start time of tasks, the uniqueness of tasks, strategies after task failure, etc. In addition, the scheduled task framework can also split tasks for large data sets and perform parallel verification.

[0109] Embodiment 7:

[0110] like Figure 7 As shown, this embodiment provides a data annotation method. In addition to the technical features of the above embodiments, this embodiment further includes the following technical features:

[0111] The data annotation method also includes the following steps:

[0112] Step S702, receiving the dataset annotation information and the file download pre-signed link request, obtaining the files and annotated file list under the path corresponding to the dataset according to the metadata information, generating a download link, and returning the download link to the client.

[0113] In this embodiment, after receiving the dataset annotation information and the file download pre-signed link request, the data service obtains the files and annotated file list under the corresponding path of the dataset based on the metadata information, generates a download link, and returns the download link to the client to facilitate the client to download. Among them, the pre-signed link method can better verify the identity, ensure the security of the data, and meet the data security requirements in multi-tenant scenarios.

[0114] Embodiment 8:

[0115] like Figure 8 As shown, this embodiment provides a data labeling method for an object storage service side, including the following steps:

[0116] Step S802, receiving and storing metadata information;

[0117] Step S804, receiving and storing the file;

[0118] Step S806, receiving and storing annotation information.

[0119] In this embodiment, the object storage service can use the current mainstream open source framework MinIO (open source distributed file storage system), which supports the S3 (S3 Simple Storage Service, simple storage service) annotation protocol and can be seamlessly adapted to S3 / OSS (Object Storage Service, a massive, secure, low-cost, and highly durable cloud storage service provided by Alibaba Cloud). MinIO is the world's fastest object storage service server, which is very suitable for large private cloud environments with strict security requirements and can ensure high availability under various workloads. Among them, MinIO is an open source free service, and OSS is a paid service. Using MinIO can effectively reduce costs.

[0120] In this embodiment, the object storage service may further include MySQL (a relational database management system) for data storage.

[0121] Embodiment 9:

[0122] like Fig. 9 As shown, this embodiment provides a data annotation method. In addition to the technical features of the above embodiments, this embodiment further includes the following technical features:

[0123] Receiving and storing the annotation information specifically includes the following steps:

[0124] Step S902: After verifying the files corresponding to the annotation information through batch query of the database, the annotation information is saved.

[0125] In this embodiment, the file corresponding to the annotation information is first verified, and after the verification is passed, the annotation information is saved. This embodiment implements the above process through batch query of the database, and the method is simple and easy to implement.

[0126] Embodiment 10:

[0127] like Fig.10 As shown, this embodiment provides a data annotation method for annotating a platform side, comprising the following steps:

[0128] Step S1002, requesting a list of unlabeled files from a data service;

[0129] Step S1004, receiving and verifying the unannotated set file returned by the data service, parsing the unannotated set file, adapting the annotation format, and annotating the unannotated set file;

[0130] Step S1006: Send the annotation information to the object storage service.

[0131] In this embodiment, the annotation platform has the characteristics of high compatibility and supports the current mainstream annotation formats, including but not limited to: voc (Visual Object Classes), coco (Common Objects in COntext, a data set provided by the Microsoft team that can be used for image recognition), labelme (label me), vott (Visual Object TaggingTool), etc.

[0132] This embodiment can realize parallel annotation and task management of data sets, meet the performance requirements of massive data in annotation scenarios, and achieve more efficient data annotation.

[0133] Embodiment 11:

[0134] like Fig.11 As shown, this embodiment provides a data labeling device 200, including: a memory 210 and a processor 220, the memory 210 stores a program or instruction, and the processor 220 executes the program or instruction; wherein, when the processor 220 executes the program or instruction, it implements the steps of the data labeling method as any embodiment of the present invention.

[0135] Embodiment 12:

[0136] This embodiment provides a readable storage medium, which stores a program or instruction. When the program or instruction is executed by a processor, the steps of the data marking method of any of the above embodiments are implemented. Specific embodiment:

[0138] This implementation proposes a data labeling method, which realizes parallel labeling and task management of large-scale data through low-threshold, visualization and wizard-style means, and can be applied to model training in a short time to form real AI commercial value.

[0139] This embodiment intends to solve at least one of the following key problems:

[0140] (1) How to store and manage massive amounts of unstructured data, including but not limited to images, videos, voice, and text;

[0141] (2) How to parallelize dataset annotation and task management;

[0142] (3) How to meet the performance requirements of massive data in annotation and training scenarios;

[0143] (4) How to ensure data security in multi-tenant scenarios.

[0144] The data labeling method of this embodiment is implemented through a labeling platform. The labeling platform is a low-threshold unstructured storage data labeling platform, including a client, data services, and object storage services, which can realize the complete implementation process of data set import and data labeling.

[0145] The data annotation methods include:

[0146] (1) Dataset import: Fig.12 As shown, a request to create a dataset is sent from the browser workshop 100 (or the client), and the data service 102 saves the metadata information according to the dataset name (saves it to MySQL 110 (relational database management system)). The browser workshop 100 selects an upload folder, and batch requests the data service 102 to upload urls (presigned links, Presigned Url), and then directly uploads the files to the object storage service 104.

[0147] The object storage service uses the current mainstream open source framework MinIO106 (open source distributed file storage system), which supports the S3 (S3 Simple Storage Service) annotation protocol and can be seamlessly adapted to S3 / OSS108. MinIO is the world's fastest object storage service server, very suitable for large private cloud environments with strict security requirements, and can ensure high availability under various workloads.

[0148] (2) Dataset verification: Fig.12 As shown, after all files are uploaded, the user clicks the button to verify the data set on the client. After receiving the request, the data service 102 starts the background Quartz (a timed task framework, an open source project of the OpenSymphony open source organization in the field of job scheduling) job and starts to convert the third-party format annotation information into its own format annotation information.

[0149] The data set verification content is as follows:

[0150] 1) Verify that the file exists.

[0151] 2) Delete the image EXIF ​​information and check the signature of the file to determine whether it is an image.

[0152] 3) Convert third-party annotations into annotations in its own format, and establish a mapping relationship between third-party annotations and annotated files.

[0153] 4) Count the number of labels in the statistical data set.

[0154] 5) If the file is a picture or video, add a thumbnail of the picture or video.

[0155] (3) Labeling process: Fig.13 As shown, the annotation platform 112 (annotation service) requests the data service 102 for a list of unannotated files. The data service 102 searches for unannotated files based on the metadata information of the data set and returns them to the annotation platform 112. Externally, the annotation set file, file set name and other parameters are verified. Internally, the annotation set file is parsed and the annotation format is adapted. Through batch query of the database, it is verified whether the file corresponding to the annotation information exists and the annotation information is saved.

[0156] (4) Model training process: Fig.13 As shown, Python SDK 114 (software development kit) batch requests dataset annotation information and file download URL (pre-signed link) from data service 102. Data service 102 lists the files and annotation files under the corresponding path of the dataset according to the dataset metadata information, generates a download link, and returns it to Python SDK 114. Python SDK 114 can break up the returned results, download files in parallel, and perform model training in batches.

[0157] This embodiment does not involve code. The data set import process, data set verification process, and data annotation process are all graphical and wizard-based. This embodiment uses zero-code, graphical, and wizard-based data set import and annotation processes. Compared with the free layout of canvas-based interaction, the wizard-based interaction based on a fixed process is more concise and friendly for traditional industry users who started late and have poor foundations.

[0158] The annotation platform in this embodiment has the characteristics of high compatibility and supports the current mainstream annotation formats, including but not limited to: voc, coco, labelme, vott, etc.

[0159] In this embodiment, some data annotation automation capabilities are aimed at enterprise operation scenarios such as marketing, procurement, office, HR management, supply chain, finance, and manufacturing.

[0160] This embodiment supports all unstructured data, including but not limited to: pictures, videos, audio, text, etc.

[0161] In this embodiment, data annotation and model training are seamlessly integrated.

[0162] In summary, the beneficial effects of this embodiment are:

[0163] 1. In this embodiment, the file supports all unstructured data, including but not limited to: pictures, videos, audio, text, etc., and can store and manage massive unstructured data.

[0164] 2. In this embodiment, the file upload process does not involve code, and can be in a graphical, wizard-like form to simplify the operation process. Users do not need to master other knowledge, and a low threshold is achieved. Wizard-style interaction based on a fixed process is more concise and friendly for traditional industry users who started late and have poor foundations, and better improves the user experience.

[0165] 3. In this embodiment, the labeling process can be performed in parallel to achieve more efficient data labeling.

[0166] In the present invention, the terms "first", "second", and "third" are used for descriptive purposes only and should not be understood as indicating or implying relative importance; the term "plurality" refers to two or more, unless otherwise clearly defined. The terms "installed", "connected", "connected", "fixed", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; "connected" can be a direct connection or an indirect connection through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0167] In the description of the present invention, it is necessary to understand that the directions or positional relationships indicated by terms such as “upper”, “lower”, “left”, “right”, “front” and “back” are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or unit referred to must have a specific direction, be constructed and operated in a specific orientation, and therefore, cannot be understood as a limitation on the present invention.

[0168] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "specific embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0169] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data annotation method, used on a client side, characterized in that: include: Send a create dataset request to the data service; Upload files to the object storage service; After uploading the file to the object storage service, issuing a data verification request to the data service; Wherein, the file supports unstructured data; The data labeling method further includes: Requesting batches of dataset annotation information and file download pre-signed links from the data service; Receiving a download link returned by the data service; Downloading the files in parallel according to the download links; Based on the files, the model is trained in batches.

2. The data labeling method according to claim 1, characterized in that: The uploading of files to the object storage service specifically includes: Select an upload folder, send a batch request to the data service to upload the pre-signed link, and then upload the file to the object storage service.

3. A data annotation method, used on the data service side, characterized in that: include: Receive a request to create a dataset and save metadata information to the database according to the dataset name; Receive the data verification request from the client and perform data set verification on the files stored in the object storage service; Receiving an unlabeled file list request from the labeling platform, searching for unlabeled files according to the metadata information, obtaining unlabeled set files, and returning the unlabeled set files to the labeling platform; The performing data set verification on the files stored in the object storage service specifically includes: Verify that the file exists; Based on the existence of the file, checking whether the file includes exchangeable image file format information; Based on the file including the exchangeable image file format information, deleting the exchangeable image file format information of the file; Establishing a mapping relationship between third-party annotation information and annotated files, and based on the mapping relationship, converting the third-party annotation information into annotation information in its own format; Counting the number of labels of the data set; Based on the file being a picture or a video, adding a thumbnail of the picture or the video; The processing of the files is performed in parallel.

4. The data labeling method according to claim 3, characterized in that: The converting the third-party annotation information into the annotation information in the self-format specifically includes: Start a background scheduled task framework job to convert the third-party format annotation information into the own format annotation information.

5. The data labeling method according to claim 3, characterized in that: Also includes: Receive data set annotation information and a file download pre-signed link request, obtain the files and annotated file list under the path corresponding to the data set according to the metadata information, generate a download link, and return the download link to the client.

6. A data labeling device (200), characterized in that: include: A memory (210) storing programs or instructions; A processor (220) executes the program or instruction; Wherein, when executing the program or instruction, the processor (220) implements the steps of the data labeling method according to claim 1 or 2, or the steps of the data labeling method according to any one of claims 3 to 5.

7. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the data marking method according to claim 1 or 2, or the steps of the data marking method according to any one of claims 3 to 5 are implemented.

Images