Data annotation method, electronic equipment and readable storage medium

Abstract

The invention discloses a data annotation method, electronic equipment and a readable storage medium, and the method comprises the steps: responding to a data annotation request of a user, and based on model identification information specified by the user, determining a target annotation model from a model library; the model library comprises at least one user exclusive annotation model, and the user exclusive annotation model is provided by a user and conforms to an interface specification preset by the data annotation platform; and calling the target annotation model to annotate a to-be-annotated data set corresponding to the data annotation request, and outputting an obtained annotation result. Therefore, the user can directly use the models optimized and trained for the specific scene to label the data, the adaptability advantage of the models to the data features in the vertical field is fully played, and the labeling precision and efficiency are improved.

Description

Technical Field

[0001] This invention relates to the field of data annotation technology, and in particular to a data annotation method, electronic device, and readable storage medium. Background Technology

[0002] Model-based pre-annotation has become a core technology for various data annotation platforms to improve annotation efficiency. Existing pre-annotation models are mainly divided into two categories: general models and vertical models. General models have basic recognition capabilities across scenarios, while vertical models have significantly better recognition performance than general models in specific domains.

[0003] In the existing annotation process, annotators first use the platform's general model to pre-annotate their own datasets; after pre-annotation, the results are manually reviewed and corrected; finally, the annotation results are output for model training. In this process, annotators can only rely on the general model provided by the platform for pre-annotation, and its annotation accuracy in specific domains and scenarios is often insufficient to meet the requirements, resulting in a large number of omissions. This leads to a significant amount of manpower being needed to supplement omissions and correct errors later. Summary of the Invention

[0004] The purpose of this application is to provide a data annotation method, electronic device, and readable storage medium, which enables users to directly use models optimized and trained for specific scenarios to annotate data, thereby improving annotation accuracy and efficiency.

[0005] To achieve the above objectives: In a first aspect, embodiments of this application provide a data annotation method, including: In response to a user's data annotation request, a target annotation model is determined from the model library based on the model identifier information specified by the user; the model library includes at least one user-specific annotation model, which is provided by the user and conforms to the interface specifications preset by the data annotation platform; The target annotation model is invoked to annotate the dataset to be annotated corresponding to the data annotation request, and the obtained annotation results are output.

[0006] In one embodiment, the method further includes: Receive model service images provided by users according to the preset interface specifications of the data annotation platform; Deploy the model service image to a container cluster to generate a callable user-specific labeled model instance; Register the user-specific labeled model instance to the model library.

[0007] In one embodiment, determining the target labeled model from the model library based on the model identifier information specified by the user includes: Based on the annotation mode type of the data annotation request, determine the interface specifications that the required model service should meet; Based on the model identifier information specified by the user, models that conform to the interface specification are selected from the model library and used as the target labeled models.

[0008] In one embodiment, if the annotation mode type is to annotate each data item individually, then the interface specification is a first interface specification; the first interface specification is configured to require the model service to receive a single data item input and return structured annotation information. If the annotation mode type is dataset-wide annotation, then the interface specification is the second interface specification; the second interface specification is configured to require the model service to receive the dataset storage path and sample identifier mapping relationship, and return the storage address of the annotation result file.

[0009] In one embodiment, the step of calling the target annotation model to annotate the dataset corresponding to the data annotation request includes: The annotation mode type in response to the data annotation request is to annotate each data point individually, inputting the data samples to be annotated one by one into the target annotation model for inference, and returning the single annotation result in real time; In response to the data annotation request, the annotation mode type is overall dataset annotation, and the storage path of the dataset to be annotated and the identification mapping relationship of each data sample are sent to the target annotation model; wherein, the dataset to be annotated is a dataset stored according to a specific organizational form, the specific organizational form including at least one of recording time series information, data correlation relationship or multi-source fusion information; the target annotation model is configured to perform overall inference on the dataset to be annotated based on the specific organizational form and return the storage address of the annotation result file; the format of the annotation result file is consistent with the specific organizational form of the dataset to be annotated.

[0010] In one embodiment, before the obtained annotation results are output, the method further includes: Obtain the initial annotation results of the target annotation model on the dataset to be annotated; According to the category mapping rules preset for the target annotation model, the object categories in the annotation results are mapped to the standard annotation categories in the platform annotation system; the category mapping rules define the correspondence between the sub-categories output by the user model and the standard categories in the platform annotation system.

[0011] In one embodiment, the model library further includes a general annotation model provided by a data annotation platform; the method further includes: When the user is using the service for the first time or has not specified a user-specific annotation model, the general annotation model will be determined as the target annotation model.

[0012] In one embodiment, the method further includes: The final annotation results are used as training data to optimize the corresponding user-specific annotation model; The optimized user-specific annotation model is updated to the model library.

[0013] Secondly, embodiments of this application provide an electronic device, including: a processor and a memory storing a computer program, wherein when the processor runs the computer program, the steps of the above-described data annotation method are implemented.

[0014] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described data annotation method.

[0015] This application provides a data annotation method, electronic device, and readable storage medium, comprising: responding to a user's data annotation request, determining a target annotation model from a model library based on user-specified model identifier information; the model library includes at least one user-specific annotation model, which is provided by the user and conforms to the interface specifications preset by the data annotation platform; calling the target annotation model to annotate the dataset corresponding to the data annotation request, and outputting the obtained annotation results. This allows users to directly use their models optimized for specific scenarios to annotate data, fully leveraging the adaptability of these models to the characteristics of vertical domain data, thereby effectively improving the annotation accuracy and efficiency in the corresponding scenario, and making the annotation capabilities more suitable for diverse professional scenario needs. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a data annotation method provided in an embodiment of this application.

[0017] Figure 2 This is a flowchart illustrating the model integration process in an embodiment of this application.

[0018] Figure 3 This is a schematic diagram of the architecture of a data annotation platform provided in an embodiment of this application.

[0019] Figure 4 This is a schematic diagram illustrating the process of the model management module implementing model services in an embodiment of this application.

[0020] Figure 5 This is a flowchart of the data annotation process corresponding to the cold start method in the embodiments of this application.

[0021] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. In the following description relating to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements.

[0023] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0024] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0025] It should be noted that step designations such as S101 and S102 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S102 first and then S101, etc., but these should all be within the protection scope of this application.

[0026] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0027] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0028] Model-based pre-annotation has become a core technology for various data annotation platforms to improve annotation efficiency. Existing pre-annotation models are mainly divided into two categories: general-purpose models and vertical models. General-purpose models possess basic recognition capabilities across different scenarios, while vertical models exhibit significantly better recognition performance in specific domains. In practical applications, mainstream annotation platforms generally adopt general-purpose models for pre-annotation work due to the limitations of vertical models, such as high domain adaptation barriers, high development costs, and narrow applicability, which make it difficult to meet the diverse annotation needs of the platform. For example, in the field of object recognition tasks, platforms often use general-purpose models such as the Centerpoint model for autonomous driving scenarios and the YOLO model for image processing to perform pre-annotation tasks.

[0029] In the existing annotation process, annotators first use the platform's general model to pre-annotate their own datasets; after pre-annotation, the results are manually reviewed and corrected; finally, the annotation results are exported for model training. During this process, annotators can only rely on the platform's general model for pre-annotation, which often fails to meet the required accuracy in specific domains and scenarios, resulting in a large number of missed annotations. Subsequent work requires significant manpower to supplement missing annotations and correct errors.

[0030] Based on this, this application provides a data annotation method that allows users to integrate custom vertical models into a data annotation platform through a standardized interface. This enables users to directly use their models, which are optimized and trained for specific scenarios, to annotate data. This fully leverages the adaptability of these models to the characteristics of data in vertical domains, thereby effectively improving the platform's annotation accuracy and efficiency in corresponding scenarios and making the platform's annotation capabilities more suitable for diverse professional scenario needs.

[0031] The data annotation platform (also referred to as the platform) provided in this application embodiment can be an electronic device with data processing capabilities, such as a server. Alternatively, the electronic device may include a personal computer (PC), tablet computer, laptop computer, portable computer (such as a mobile phone), wearable electronic device (such as a smartwatch), augmented reality (AR) / virtual reality (VR) device, in-vehicle computer, etc. The following embodiments do not impose special limitations on the specific form of the electronic device. The execution subject of the data annotation method provided in this application embodiment can be a data annotation platform, which can be integrated into an electronic device or the processor of an electronic device, specifically a server, etc. For ease of explanation, the following content of this application embodiment embodiment will use a server as the execution subject for description.

[0032] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a data annotation method provided in an embodiment of this application. The data annotation method includes: S101, in response to the user's data annotation request, determines the target annotation model from the model library based on the model identifier information specified by the user. The model library includes at least one user-specific annotation model, which is provided by the user and conforms to the preset interface specifications of the data annotation platform.

[0033] A data annotation request refers to the request information initiated by a user when submitting annotation task requirements through the annotation management interface of the client. The annotation management interface provides model selection options, including user-specific annotation model options and general annotation model options. The data annotation request can include the identifier information of the dataset to be annotated, the annotation task type, the annotation mode type, and the model identifier information specified by the user. The annotation task type can be object detection, semantic segmentation, instance segmentation, etc.; the annotation mode type can be annotating individual data points or annotating the entire dataset.

[0034] Model identification information is used to uniquely identify a specific labeled model in the model library. It can be the model name, model number, model version number, or a combination thereof. User-specified model identification information can be obtained in several ways. Specifically, the server can receive user selections of labeled models in the model library through the annotation management interface; that is, when a user clicks or selects a model from the model list displayed on the interface, the server can obtain the corresponding model identification information. Alternatively, the data annotation request can directly include model identification information, with the user actively filling in or selecting the model identifier when creating an annotation task. Or, the server can determine the model identification information based on the model bound to the user's identity identifier. The platform pre-configures default models for users, and when a user initiates an annotation request, the server automatically reads the model identifier bound to that user.

[0035] A model library is a database or storage system used by a data annotation platform to store and manage labeled models. The model library stores at least one user-specific labeled model. These user-specific labeled models are developed or trained by the user and adapted according to the data annotation platform's pre-defined interface specifications. The interface specifications define the input and output formats, calling methods, and data transmission protocols of the model services, ensuring that user-provided models can interact with the platform and integrate functionally. User-specific labeled models are typically vertical domain models optimized for specific application scenarios or data types, exhibiting higher recognition accuracy in specific scenarios compared to the platform's general-purpose models.

[0036] The server retrieves and determines the target annotation model from the model library based on the model identifier information specified by the user. The target annotation model refers to the model instance that will be used to perform the current annotation task. Determining the target annotation model can be done by looking up the corresponding model in the index table of the model library using the model identifier information, and obtaining information such as the model's service address and configuration parameters.

[0037] S102, call the target annotation model to annotate the dataset to be annotated corresponding to the data annotation request, and output the obtained annotation results.

[0038] The dataset to be labeled refers to the original data set that needs to be labeled, which can include various data types such as point cloud data, image data, and video data. The dataset to be labeled can be a single data sample or a data set containing multiple data samples.

[0039] When a server invokes a target annotation model to annotate a dataset, the process may include sending an annotation request to the model's service interface, passing the data to be annotated or the data storage path, and the model receiving the input data, performing inference calculations, generating annotation results, and returning them to the server. The annotation results contain identification information about the target objects in the data. For example, in an object detection task, the annotation results include the detected target category, the target's spatial coordinates, the target's size information, and the detection confidence score. The output annotation results refer to the server processing the annotation results generated by the annotation model and presenting them to the user, such as sending them to a client for display.

[0040] In one embodiment, the output annotation results may include initial annotation results generated by the target annotation model for user review and correction, and the final annotation result, i.e., the annotation dataset, may be output. Specifically, the server synchronizes the initial annotation results to the annotation management interface, where information such as annotation boxes and annotation categories are displayed visually. Users can review the initial annotation results on the interface, manually correct erroneous or inaccurate annotations, and supplement missing targets. After the user completes the review and correction, the server generates the final annotation dataset, which contains high-quality annotation data that has been manually verified and can be used for model training or other applications.

[0041] In summary, by providing standardized interfaces, users can integrate custom vertical models into the data standard platform. This allows users to directly use models optimized for specific scenarios, fully leveraging the adaptability of these models to the characteristics of vertical domain data. Consequently, the platform's annotation accuracy and efficiency in corresponding scenarios are effectively improved, making the platform's annotation capabilities more aligned with the diverse needs of professional scenarios.

[0042] In some embodiments, please refer to Figure 2 , Figure 2 This is a flowchart illustrating the model integration process in this embodiment. The model integration process performed by the data annotation method provided in this embodiment before responding to a user's data annotation request includes: S201: Receive the model service image provided by the user according to the interface specifications preset by the data annotation platform.

[0043] A model service image refers to a container image file formed by encapsulating a user's labeled model and its runtime environment. The container image includes model code, dependency libraries, runtime environment, and configuration files. After completing model development and interface adaptation locally, users use container technology to package the model into an image file and upload it to the data annotation platform through the "Add Model" entry provided on the annotation management interface.

[0044] S202 deploys the model service image to the container cluster, generating a callable user-specific labeled model instance.

[0045] A container cluster is a pool of computing resources composed of multiple physical servers or virtual machine nodes, managed uniformly through a container orchestration platform. In one specific implementation, the deployment operation includes automated deployment, resource scheduling, and lifecycle management of the model service image through the Kubernetes container orchestration platform. Specifically, the server pushes the model service image to the Kubernetes cluster. Kubernetes creates container instances according to the configuration file, allocates computing resources such as CPU and GPU to the containers, configures network ports and storage volumes, and starts the model service process. Kubernetes is also responsible for monitoring the running status of the container instances, automatically restarting them when they exit abnormally, elastically scaling them up when resources are insufficient, and migrating them to other nodes when nodes fail. After the model instance starts successfully, the server generates a unique service call address for the instance, which includes a network domain name and interface path, used to receive external annotation requests.

[0046] S203 registers user-specific annotated model instances to the model library.

[0047] The user-specific annotation model instance registration process includes creating a model record in the model library, recording information such as the model's identifier, service address, supported annotation types, input / output specifications, and user identity. After registration, the user can select to use the model in subsequent annotation tasks.

[0048] In some embodiments, during the process of determining the target annotation model, the server can determine the interface specifications that the required model service should meet based on the annotation mode type of the data annotation request, and select models that conform to the interface specifications from the model library based on the model identification information specified by the user, as the target annotation model.

[0049] The annotation mode type defines how the annotation task is executed. In data annotation platforms, annotation mode types mainly include two types: individual data annotation and dataset-wide annotation. Individual data annotation refers to annotating each data sample in the dataset separately. The model receives one data input at a time and returns the annotation result for that data. Dataset-wide annotation refers to submitting the entire dataset as input to the model all at once. The model processes all samples in the dataset in batches and returns the annotation results for all samples.

[0050] Different annotation mode types correspond to different interface specifications. The interface specification defines the input parameters, output format, and call flow of the model service. If the annotation mode type is single-data-line annotation, then the interface specification is the first interface specification. The first interface specification is configured to require the model service to receive single-data-line input and return structured annotation information. Specifically, the first interface specification requires the model service's input parameters to include the content or storage path of a single data sample, and the output parameter to be structured annotation information. This annotation information is organized in a specific data format, such as JSON, and includes fields such as the category, location, size, and confidence level of the annotated object.

[0051] If the annotation mode type is dataset-wide annotation, then the interface specification is the second interface specification. The second interface specification is configured to require the model service to receive the dataset storage path and sample identifier mapping relationship, and return the storage address of the annotation result file. Specifically, the second interface specification requires the model service's input parameters to include the dataset's storage path in the storage system, and the sample identifier mapping relationship data. The sample identifier mapping relationship data records the correspondence between the unique identifier of each sample in the dataset and the sample file, enabling the model to accurately locate and process each sample. After completing the annotation, the model service writes the annotation results to a file, which can be a single file or a folder containing multiple files, and uploads the file to the specified storage system, returning the file's storage address. The storage address can be an object path in cloud storage, a path in a network file system, or a path in a distributed file system.

[0052] After determining the interface specification based on the annotation mode type, the server filters models from the model library that conform to that specification. The filtering process includes reading the interface specification attributes of each model in the library and determining whether these attributes match the interface specification required for the current task. In one embodiment, models are annotated with their supported interface specification types during registration. The server filters out models that meet the criteria by comparing the annotation mode type with the supported interface specification types. Then, based on the model identifier information specified by the user, the server further determines a unique target annotation model from the filtered models. When calling the target annotation model to annotate the dataset, the server can use different calling methods depending on the annotation mode type.

[0053] In some embodiments, the annotation mode type in response to the data annotation request is to annotate each data point individually. The server inputs the data samples to be annotated one by one into the target annotation model for inference and returns the single annotation result in real time.

[0054] Specifically, the server sequentially extracts individual data samples from the dataset to be labeled. For each extracted data sample, the server sends a labeling request to the target labeling model's service interface, containing the content or storage path of that data sample. Upon receiving the request, the target labeling model performs inference calculations on the data sample, generates a labeling result, and returns the result to the server via the interface response. After receiving the labeling result, the server immediately outputs it to the labeling interface or stores it in a temporary cache, then continues processing the next data sample until all data samples have been processed.

[0055] In some embodiments, the annotation mode type in response to the data annotation request is overall dataset annotation. The server sends the storage path of the dataset to be annotated and the identification mapping relationship of each data sample to the target annotation model. The dataset to be annotated is a dataset stored in a specific organizational form, which includes at least one of recording time-series information, data correlation relationship or multi-source fusion information. The target annotation model is configured to perform overall inference on the dataset to be annotated based on the specific organizational form and return the storage address of the annotation result file. The format of the annotation result file is consistent with the specific organizational form of the dataset to be annotated.

[0056] Specifically, the server uploads or stores the dataset to be labeled to a designated storage space and obtains the dataset's storage path. Simultaneously, the server constructs sample identifier mapping data, which records the correspondence between the identifier of each sample in the dataset and the sample file path or sample content. The server sends the dataset storage path and sample identifier mapping data to the service interface of the target labeling model. The target labeling model accesses the dataset according to the storage path, locates each sample according to the sample identifier mapping, performs batch inference processing on all samples, generates labeling results, and writes the labeling results to a file. The labeling result file contains the labeling information for each sample, associated through sample identifiers. The target labeling model uploads the labeling result file to the storage system and returns the file's storage address to the server. The server downloads the labeling result file according to the returned storage address, parses the file content, extracts the labeling information for each sample, and outputs it to the labeling interface.

[0057] The specific organizational form refers to a known publicly available dataset format, including at least one of nuScenes, Waymo, KITTI, COCO, or LeRobot formats. This holistic dataset annotation mode differs fundamentally from traditional batch processing. Batch processing simply packages and sends multiple independent samples without any correlation between them; whereas holistic dataset annotation targets a complete dataset with a specific inherent organizational form. Taking the nuScenes dataset as an example, it contains data collected by multiple sensors (LiDAR, cameras) at consecutive time stamps, with strict temporal relationships and cross-sensor calibration relationships between samples. Through holistic annotation, the model can utilize temporal context, mine correlation information, and fuse multi-source data to improve annotation accuracy. Furthermore, the output annotation results fully comply with the format specifications of the original dataset, allowing users to directly use them for model training without cumbersome format conversion and parsing.

[0058] In some embodiments, the server may also perform category mapping processing before outputting the obtained annotation results.

[0059] Obtain the initial annotation results of the target annotation model on the dataset to be annotated. The initial annotation results contain the category information of the annotated objects.

[0060] Based on the pre-defined category mapping rules for the target annotation model, the object categories in the initial annotation results are mapped to the standard annotation categories in the platform annotation system to obtain the annotation results; the category mapping rules define the correspondence between the subcategories output by the user model and the standard categories in the platform annotation system.

[0061] Because different models may have different training datasets and category systems, the object categories output by the model may not match the standard labeling categories defined by the data annotation platform. To achieve unified category management and standardized use of subsequent data, category mapping rules can be one-to-one or many-to-one. For example, the dataset used by the user model during training may subdivide vehicles into multiple categories such as cars, buses, and trucks, while the platform's annotation system uniformly classifies these vehicles into the vehicle category. The category mapping rule defines that the car, bus, and truck categories are all mapped to the vehicle category.

[0062] When registering a target annotation model or configuring an annotation task, the server receives the category mapping rules set by the user and stores these rules in the model library or task configuration. After obtaining the initial annotation results, the server reads the category mapping rules, iterates through each annotated object in the initial annotation results, obtains the object's original category, finds the corresponding standard category in the category mapping rules, and replaces the object's category with the standard category. After category mapping processing, the object categories in the annotation results are consistent with the platform's annotation system, facilitating subsequent data management and model training.

[0063] In some embodiments, the model library also includes general annotation models provided by the data annotation platform. These general annotation models are models developed or integrated by the data annotation platform itself, possessing basic cross-scenario recognition capabilities and applicable to various annotation scenarios and data types. When a user is using the data annotation platform for the first time or has not specified a user-specific annotation model, the server determines the general annotation model as the target annotation model. Specifically, after receiving a data annotation request, the server checks whether the user has specified model identification information. If not, it further checks whether the user has registered a user-specific annotation model. If the user has not registered any user-specific annotation models, or if the user explicitly chooses to use the platform's general model, the server selects a general annotation model suitable for the current annotation task type from the model library as the target annotation model. This mechanism provides convenient annotation services for new users, allowing them to complete data annotation tasks using the platform's annotation functions without having to develop and deploy models themselves in the initial stages.

[0064] After data annotation is completed, the server can use the final annotation results as training data to optimize the corresponding user-specific annotated model. The final annotation results refer to high-quality annotated data that has been reviewed and corrected by the user. The server exports the final annotation results in the data format required for model training. The user uses this annotated data to retrain or fine-tune the user-specific annotated model, improving the model's performance in specific scenarios. After model optimization is complete, the user generates a new model weight file or a new model version.

[0065] The server updates the optimized model to the model library. This update can involve replacing the weight file of the existing model or registering a new model version in the library. In one embodiment, the user uploads the optimized model weight file to the data annotation platform; the server receives the weight file and updates the weight path configuration of the corresponding model in the model library. In another embodiment, the user repackages the optimized model into a model service image; the server receives the image, deploys it as a new model instance, registers the new model instance in the model library, and associates it with the user's account. This iterative optimization mechanism forms a closed loop between data annotation and model training, continuously improving the performance of the user-specific annotated model and the platform's overall annotation capabilities.

[0066] To more clearly illustrate the technical solutions of the embodiments of this application, a detailed description is provided below in conjunction with specific application scenarios.

[0067] In one application scenario, an autonomous driving company needed to perform object detection and annotation on a large amount of collected point cloud data. The company had already trained an object detection model optimized for specific road scenarios based on its own data. This model demonstrated high accuracy in identifying vehicles, pedestrians, and traffic signs in the local road environment. The company's technical staff modified the model service code according to the second interface specification provided by the data annotation platform, enabling it to receive the dataset storage path and sample identifier mapping relationship, and return the storage address of the annotation result file. The technical staff then packaged the model and its dependencies into a Docker image and uploaded it to the data annotation platform.

[0068] After receiving the Docker image, the data annotation platform's server automatically deploys it using the Kubernetes container orchestration platform. The server creates container instances in the Kubernetes cluster, allocates GPU resources to the containers to accelerate model inference, and configures network services so that the containers can receive external requests through a specific port. Once the container starts, the server generates a unique call address for the model service and creates a model record in the model library. The record's model name is "Company Road Scene Detection Model Version 1.0," the supported annotation type is 3D object detection, and the interface specification is the second interface specification.

[0069] Users log in to the data annotation platform, create new annotation tasks, and upload the point cloud dataset to be annotated. The dataset contains 10,000 point cloud samples, each containing point cloud data collected by LiDAR and corresponding camera image data. Annotators select the entire dataset as the annotation mode in the task configuration interface and choose the company's road scene detection model version 1.0 from the model selection list. The server receives the annotation task request and obtains the user-specified model identifier information as the company's road scene detection model version 1.0.

[0070] The server annotates the entire dataset based on the annotation mode type, determining that the required model service should meet the second interface specification. The server then filters models in the model library that conform to the second interface specification, finding that the company's road scene detection model version 1.0 meets the criteria and is therefore selected as the target annotation model.

[0071] The server uploads the point cloud dataset to be labeled to the cloud storage system, and the storage path of the dataset is designated as cloud storage path A. Simultaneously, the server constructs sample identifier mapping data, which records the unique identifier of each point cloud sample and the relative path of the sample file within the dataset. The server sends a labeling request to the service interface of the target labeling model, including the dataset storage path (cloud storage path A) and the sample identifier mapping data.

[0072] The target annotation model's service interface receives the request, and the model service downloads the dataset from cloud storage path A to the local computing node. The model service loads each point cloud sample sequentially according to the sample identifier mapping relationship, preprocesses the point cloud data, performs target detection inference, identifies vehicles, pedestrians, and traffic signs in the point cloud, and records the category, 3D spatial coordinates, size, and orientation angle of each detected target. The model service organizes the annotation results of all samples by sample identifier and writes them to an annotation result file. The annotation result file uses JSON format, with each sample corresponding to a JSON object containing the sample identifier and a list of detected targets. The model service uploads the annotation result file to the cloud storage system, obtains the file's storage address as cloud storage path B, and returns this address to the server.

[0073] After receiving the cloud storage path B, the server downloads the annotation result file from that path, parses the file content, and extracts the annotation information for each sample. The server synchronizes the annotation information to the annotation interface of the data annotation platform, where annotators can see the point cloud data and the overlaid detection boxes and category labels. Annotators review the annotation results and find that some small, distant targets were not detected, and that the positions of some detection boxes were inaccurate. Annotators manually add the missed targets, adjust the position and size of the incorrect detection boxes, and complete the review and correction.

[0074] The server records the annotations made by the annotators, updates the annotation results, and generates the final annotated dataset. The final annotated dataset is exported in its original format, including point cloud files, image files, and annotation result files. Company technicians download the final annotated dataset and use it as training data to retrain the company's road scene detection model, improving the model's ability to detect small and distant targets. After training is complete, technicians generate a new model weight file and upload it to the data annotation platform.

[0075] The server receives the new model weight file, updates the weight path configuration of version 1.0 of the company's road scene detection model in the model library, or creates a new model version record as version 1.1. In subsequent annotation tasks, annotators choose to use the updated model, which improves the model's annotation accuracy, reduces the workload of manual correction, and increases annotation efficiency.

[0076] See Figure 3 This is a schematic diagram of the architecture of a data annotation platform according to an embodiment of this application. The data annotation platform supports users in developing adapted model service interfaces based on their own models (especially domain-specific models) according to the unified input / output specifications provided by the platform, achieving seamless integration of user-defined models with the platform. For example... Figure 3As shown, the data annotation platform comprises five functional modules: data import, data storage, model management, data annotation, and result export. Among them: The data import module in the data annotation platform supports importing various dataset formats. For point cloud data, the platform supports nuScenes, Waymo, and KITTI formats; for image data, it supports COCO and YOLO formats. These formats are mainstream standard formats in the open-source dataset field. By supporting these formats, users can directly use open-source model architectures for subsequent model training and evaluation.

[0077] When users import datasets in different formats, the data storage module in the data annotation platform first stores the complete dataset file to a remote storage device. Each dataset format can contain multiple files, which include metadata such as scene information, sensor calibration information, and sample information. The data storage module uses the parsing program corresponding to the dataset format to extract this metadata information, including the storage path, timestamp, sensor type, and calibration parameters for each sample. Simultaneously, the server processes the data to generate a visual preview image, which displays the spatial distribution of the point cloud in either a top-down or perspective view. After parsing, the server associates the extracted metadata and the generated preview image with the original data to form a dataset to be labeled, and then transmits this dataset to the database.

[0078] To facilitate the integration of user-specific labeled models, the model management module provides users with interface specifications. The model code corresponding to each user-specific labeled model must match these interface specifications. The interface specifications refer to the standardized interface definitions for model input and output defined by the data annotation platform. These definitions specify the request format received by the model service, the response format returned, and the data transmission method. Through these interface specifications, the data annotation platform can interact with user-specific labeled models of different architectures and training methods.

[0079] The model management module automatically builds and manages the entire lifecycle of the model service after the user has modified the model code according to the preset interface format and receives the user's custom-annotated model, based on the user's selected deployment method. Once the model service is deployed and available, the user can access the model service management page of the data annotation platform, fill in the service identifier, call configuration, and other information to integrate their own model into the data annotation platform. Users can choose from two deployment methods: self-deployment of the model service or submission of a Docker image to the data annotation platform.

[0080] The model management module also supports users to manually configure the mapping relationship between model output categories and data annotation platform categories, so as to accurately associate the model pre-annotated data with the annotation categories specified by the data annotation platform.

[0081] In one implementation, please refer to Figure 4 , Figure 4 This is a schematic diagram illustrating the process of the model management module implementing model services in an embodiment of this application. Figure 4 As shown, the model management module in the data annotation platform categorizes interface specifications into different types based on model annotation format and data type, specifically including a first interface specification and a second interface specification. The first interface specification corresponds to the single-sample annotation method, while the second interface specification corresponds to the overall dataset annotation method. The single-sample annotation method means that the model service receives a single data sample as input each time, performs inference on that sample, and returns the annotation result for that sample. The overall dataset annotation method means that the model service receives the entire dataset as input, performs batch inference on all samples in the dataset, and returns the annotation results for all samples.

[0082] When a user selects the single-line annotation method, the model code corresponding to the user-specific annotation model must match the first interface specification. The input parameters defined in the first interface specification can include the storage path, data type identifier, and sample identifier for a single data sample. Output parameters include: annotation result data, confidence information, etc.

[0083] Specifically, for the point cloud target recognition task, the input parameters defined in the first interface specification may include the storage path of the point cloud file, which points to the point cloud data file on a remote storage device. The output parameters include a list of recognized targets, each containing information such as target category, confidence level, 3D spatial coordinates, size parameters, and rotation angles. The 3D spatial coordinates include x-coordinates, y-coordinates, and z-coordinates, representing the target's position in 3D space. The size parameters include dx, dy, and dz, representing the target's length in the three dimensions. The rotation angles include rotation angles around the x-axis, y-axis, and z-axis.

[0084] In this embodiment, when a user selects the overall dataset annotation method, the model code corresponding to the user-specific annotation model needs to match the second interface specification. The input parameters defined in the second interface specification may include the storage path of the complete dataset and a file mapping data identifiers to samples. Output parameters include the storage path of the annotation result file, etc. The mapping relationship between data identifiers and samples refers to the correspondence between the unique identifier of each sample and its location or filename in the dataset. Through this mapping relationship, the model service can associate inference results with specific samples.

[0085] In a specific implementation, when a user selects the overall dataset annotation method, the server first generates a mapping file between data identifiers and samples. This file is stored in JSON or CSV format. For example, each record in the mapping file contains a sample identifier and information such as the path to the corresponding point cloud file and image file. The server uses the storage path of the complete dataset and the mapping file as input parameters and sends a request to the user-specific annotation model service.

[0086] Upon receiving a request, the user-specific annotation model service downloads the complete dataset and mapping relationship file from the remote storage device to the local service node. Based on the mapping relationship file, the user-specific annotation model service reads samples from the dataset one by one or in batches and performs inference processing. After inference is complete, the user-specific annotation model service generates a structured annotation result file, which contains the identifier of each sample and the corresponding annotation result. The user-specific annotation model service uploads the annotation result file to the remote storage device and returns the storage path of the annotation result file as a response to the server.

[0087] If the model code corresponding to the user-specific annotation model matches the first interface specification, the server inputs the unlabeled data from the dataset to the user-specific annotation model one by one. Specifically, the server reads the storage path of the first unlabeled data, constructs a request conforming to the first interface specification, and sends the request to the user-specific annotation model service. After receiving the request, the user-specific annotation model service downloads the corresponding data file from the remote storage device, performs inference processing, generates annotation results, and returns the annotation results as a response to the server. After receiving the annotation results, the server stores the results in the database and associates them with the corresponding unlabeled data. Then, the server reads the storage path of the next unlabeled data and repeats the above process until all unlabeled data has been annotated.

[0088] If the model code corresponding to the user-specific annotation model matches the second interface specification, the server inputs the dataset to be annotated and the mapping relationship between the data identifiers and each sample in the dataset into the user-specific annotation model. Specifically, the server generates a mapping relationship file containing all sample identifiers and their corresponding storage paths, and uploads this file to a remote storage device. Simultaneously, the server constructs a request conforming to the second interface specification, containing the storage path of the complete dataset and the storage path of the mapping relationship file, and sends this request to the user-specific annotation model service. Upon receiving the request, the user-specific annotation model service downloads the complete dataset and mapping relationship file from the remote storage device, performs batch inference processing, generates a result file containing the annotation results of all samples, uploads this result file to the remote storage device, and returns the storage path of the result file as a response to the server. Upon receiving the response, the server downloads the result file from the remote storage device, parses the result file, associates the annotation result of each sample with its corresponding sample, and stores it in the database.

[0089] Furthermore, the process of labeling the dataset to be labeled using the user-specific annotation model also includes a category mapping step. The model management module receives the data category mapping rules input by the user; the data annotation module annotates the dataset to be labeled using the user-specific annotation model, and transforms the data annotation results according to the data category mapping rules to obtain the target dataset.

[0090] Data category mapping rules refer to the conversion rules that transform category identifiers output by user-specific annotation models into category identifiers defined by the data annotation platform. Category identifiers output by user-specific annotation models may differ from platform-defined category identifiers in naming or granularity. Category mapping rules unify category identifiers from different sources, ensuring consistency in annotation results.

[0091] In a specific implementation, users can access the category mapping configuration interface. This interface displays a list of categories supported by the user-specific annotation model and a list of categories defined by the platform. Users select a category from their user-specific annotation model and map it to a category defined by the platform. For example, if the user-specific annotation model was trained using the nuScenes dataset, its output categories might include subcategories such as vehicle.car, vehicle.bus, and vehicle.van. The user's annotation requirement is to unify these subcategories into the "Car" category. In the configuration interface, the user maps the vehicle.car, vehicle.bus, and vehicle.van categories to the "Car" category respectively. After configuration, the client sends the mapping rules to the server, which stores the mapping rules in the target database.

[0092] Once the user-specific annotation model completes data annotation, the server receives the annotation results from the user-specific annotation model service. The annotation results include category identifiers output by the user-specific annotation model. The server reads mapping rules from the target database and transforms the category identifiers in the annotation results according to the mapping rules. Specifically, the server iterates through each target in the annotation results, reads the target's category identifier, finds the corresponding platform category identifier in the mapping rules, and replaces the target's category identifier with the platform category identifier. After the transformation is complete, the server stores the transformed annotation results in the database as the target dataset.

[0093] After the model code corresponding to the user-specific labeled model matches the interface specification, the model management module receives the Docker image of the user-specific labeled model, deploys the user-specific labeled model through a Kubernetes cluster, and integrates the user-specific labeled model into the data annotation platform based on the model configuration information input by the user. A Docker image is a standardized container image that packages the user-specific labeled model and its runtime environment, containing all necessary modules such as model code, dependency libraries, and runtime environment. A Kubernetes cluster is a container orchestration cluster built on Kubernetes technology, responsible for the creation, scheduling, management, and monitoring of containers. Model configuration information refers to the metadata information of the user-specific labeled model, including model name, version number, model weight file path, supported annotation types, and API call address.

[0094] In this specific implementation, the user first develops and tests their own labeled model in a local environment to ensure that the model code conforms to the interface specifications. The user then packages the model code and dependent libraries into a Docker image, which is built based on a standard base image. The user configures a startup script for the model service in the image. This script automatically loads the model weight file, initializes the model, starts the HTTP server, and listens on a specified port to receive inference requests when the container starts.

[0095] Users upload Docker images, the client transfers the Docker images to the server, and the server stores the images in the platform's image repository. The image repository is a centralized image storage system that supports image version management and access control.

[0096] Next, the user configures the model deployment parameters. These parameters include: computing resource quotas, such as the number of CPU cores, memory size, and number of GPUs; network configurations, such as the ports listened to inside the container and the service ports exposed to the outside; storage configurations, such as the storage path for the model weight files and the mount path for the data cache directory; and environment variables, such as the hyperparameters used during model inference and the log level.

[0097] After receiving the deployment parameters, the server creates a container instance through the Kubernetes cluster's API. Based on the deployment parameters, the Kubernetes cluster pulls the Docker image from the image repository, creates the container on a specific node in the cluster, allocates the specified computing resources, mounts storage volumes, configures network ports, and sets environment variables. Once the container starts, the model service's startup script executes automatically, loading the model weight file, initializing the model, and starting the HTTP server.

[0098] The Kubernetes cluster performs full lifecycle management of the model service. This lifecycle management includes: health checks (periodically sending health check requests to containers; if a container does not respond or responds abnormally, it is considered unhealthy); fault recovery (automatically restarting the container or recreating it on another node when a container is unhealthy or a node fails); resource scheduling (dynamically adjusting the allocation of containers based on cluster resource usage to achieve load balancing); and elastic scaling (automatically increasing or decreasing the number of container instances based on the load of the model service to improve the service's concurrent processing capabilities).

[0099] After the model service starts successfully, the Kubernetes cluster generates a unique service call domain name. This service call domain name includes an internal cluster domain name and an external cluster domain name. The internal cluster domain name is used by modules within the platform to access the model service, while the external cluster domain name is used by external systems to access the model service. The server returns the service call domain name to the client, which then displays it on the user interface.

[0100] After confirming that the model service is running normally, users register the model. The model registration interface includes several input fields: Model Name field, where users enter the name of the model, such as "Autonomous Driving Object Detection Model"; Version Number field, where users enter the model's version number, such as "v1.0"; Model Weight Path field, where users enter the path of the model weight file on a remote storage device; Supported Annotation Type field, where users select the annotation types supported by the model, such as object detection, semantic segmentation, etc.; API Call Address field, which is automatically filled with the service call domain name; and Category List field, where users enter the list of categories output by the model.

[0101] After filling in all fields, the user clicks the submit button. The client sends the model configuration information to the server, which then stores it in the target database. Once stored, the user-specific labeled model is officially integrated into the data labeling platform's system, and the user can select this model for pre-labeling in subsequent labeling tasks.

[0102] The data annotation module adopts a "model pre-annotation + manual review" model for data annotation, which can be divided into two usage scenarios: cold start and model iteration. In the cold start scenario, if the user has not uploaded their own model, they can call the platform's general model to complete the pre-annotation, and then train their own model based on the annotation results. In the model iteration scenario, if the user has uploaded their own model, they can directly use that model to pre-annotate their own dataset.

[0103] In one implementation, please refer to Figure 5 , Figure 5 This is a flowchart illustrating the data annotation process corresponding to the cold start method in this application embodiment. For example... Figure 5 As shown, if the data annotation platform does not deploy a user-specific annotation model, the data annotation module uses the general model deployed on the data annotation platform to annotate the dataset to be annotated, and obtains the annotated dataset. Based on the correction instructions input by the user, the annotated dataset is corrected to obtain the target dataset, and the target dataset is exported to be used as a training dataset to train the user-specific annotation model.

[0104] A cold start scenario refers to a situation where a user is using a data annotation platform for the first time and has not yet trained a user-specific annotation model. In this scenario, the user can only rely on the general model provided by the platform for pre-annotation.

[0105] In practice, users create annotation tasks. These tasks include configurations such as selecting the dataset to be labeled, choosing the annotation type, and selecting a pre-labeled model. Since users haven't integrated their own annotation models, they can only select general models provided by the platform from the pre-labeled model selection options. These general models include general models for point cloud object detection, image object detection, and semantic segmentation.

[0106] After the user selects a general model, the client sends the annotation task configuration to the server. Based on the annotation task configuration, the server calls the corresponding general model to pre-annotate the dataset to be labeled. The pre-annotation process is similar to the annotation process of the user-specific annotation model described above. The server sends the data to be labeled one by one or in batches to the general model service, which performs inference processing and returns the annotation results. The server then stores the annotation results in the target database.

[0107] After pre-annotation is completed, the client retrieves the annotation results from the server and displays them on the user interface. The user interface provides visual annotation display functions, such as displaying the detected target bounding boxes in a 3D view of the point cloud, and displaying the detected target bounding boxes or segmentation masks on the image.

[0108] Users manually review the pre-labeled results through the data annotation module. During the review process, users discovered omissions and mislabeling in the general model. Omissions refer to actual targets not being detected by the model, while mislabeling refers to targets detected by the model having incorrect categories or locations. Users input correction commands, including: adding targets (users manually draw new bounding boxes or segmentation masks on the interface to mark omissions); deleting targets (users select incorrect annotation results to delete them); and modifying targets (users adjust the position or size of the bounding boxes and modify the target's category label).

[0109] The client sends a correction command to the server, which then updates the annotation results in the target database according to the command. After the update is complete, the server returns the corrected annotation results to the client, which then refreshes the interface to display them.

[0110] After completing the cold start scenario, users obtain a pre-trained user-specific annotation model. After integrating this model into the data annotation platform, users enter the model iteration scenario. In this scenario, users can directly call their user-specific annotation model to pre-annotate new datasets, leveraging its performance advantages in specific scenarios to improve the accuracy and efficiency of pre-annotation.

[0111] In practice, users create new annotation tasks. Among the pre-annotated models, in addition to the platform-provided general models, user-specific annotation models that the user has integrated are also listed. Users select their user-specific annotation model as the pre-annotated model.

[0112] The client sends the annotation task configuration to the server. Based on the annotation task configuration, the server invokes the corresponding user-specific annotation model to pre-annotate the dataset to be labeled. The pre-annotation process is as described above: the server sends the data to be labeled to the user-specific annotation model service according to the user-specific annotation model's interface specification; the user-specific annotation model service performs inference processing and returns the annotation results; the server performs category mapping transformation on the annotation results and stores them in the target database.

[0113] Because user-specific annotation models are trained for specific scenarios, their recognition performance in those scenarios outperforms general-purpose models. Therefore, the accuracy and recall of pre-annotated results are significantly improved, while missed and incorrect annotations are greatly reduced. This significantly reduces the amount of corrective work required by users during the review process, resulting in a significant improvement in annotation efficiency.

[0114] After the user completes the review and correction, they can export the target dataset through the results export module. This target dataset contains high-quality labeled data, which the user can use for iterative training of their custom labeled model.

[0115] During iterative model training, users merge the newly exported labeled dataset with the previous training dataset to form a larger-scale training dataset. Users then use this training dataset to retrain or incrementally train their custom labeled model, resulting in a new version of the model weight file. This new version of the model further improves recognition performance.

[0116] Users upload the new version of the model weight file to the platform and update the configuration information of their user-specific labeled model through the model management module. If the user chooses the hot update method, the server triggers a weight reload operation for the user-specific labeled model service, and the model service loads the new model weight file to complete the update. If the user chooses the redeployment method, the server recreates the container instance of the user-specific labeled model service and uses the new version of the model weight file.

[0117] After the update is completed, users will continue to use the updated user-specific annotation model for pre-annotation in subsequent annotation tasks. With the continuous accumulation of annotation data and the continuous iteration of the model, the performance of the user-specific annotation model will continue to improve, and the accuracy and efficiency of pre-annotation will continue to be optimized, forming a virtuous cycle of "annotation data generation - model iteration and optimization - annotation capability upgrade".

[0118] After reviewing and correcting all samples, users can export the target dataset using the data export module. The data export module converts the annotation results into the corresponding standard format based on the original format and annotation type of the dataset. For example, for a nuScenes format point cloud dataset, the export module generates standard annotation files such as sample_annotation.json, instance.json, and category.json. Simultaneously, the export module packages the original dataset and the generated annotation files into a complete labeled dataset for users to download. Users can then use the exported labeled dataset to train their custom labeled model. The training process takes place in the user's local environment or on a cloud training platform; the platform does not participate in the actual training process. After training is complete, users obtain the model weight file for their custom labeled model.

[0119] Based on the same inventive concept as the foregoing embodiments, this invention provides an electronic device, please refer to... Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the device includes: a processor 610 and a memory 611 storing a computer program; wherein, Figure 6 The processor 610 shown in the diagram does not indicate that there is only one processor 610, but only indicates the positional relationship of the processor 610 relative to other devices. In practical applications, there can be one or more processors 610; similarly, Figure 6 The memory 611 shown in the diagram has the same meaning, that is, it is only used to indicate the positional relationship of memory 611 relative to other devices. In practical applications, there can be one or more memories 611. When the processor 610 runs the computer program, it implements a data annotation method applied to the above-mentioned device.

[0120] The device may also include at least one network interface 612. The various components of the device are coupled together via a bus system 613. It is understood that the bus system 613 is used to implement communication between these components. In addition to a data bus, the bus system 613 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 6 The general designated all buses as Bus System 613.

[0121] The memory 611 can be volatile or non-volatile, or both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); the magnetic surface memory can be disk storage or magnetic tape storage. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 611 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0122] The memory 611 in this embodiment of the invention is used to store various types of data to support the operation of the device. Examples of this data include: any computer programs used to operate on the device, such as operating systems and applications; contact data; phonebook data; messages; pictures; videos, etc. The operating system includes various system programs, such as the framework layer, core library layer, driver layer, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications, such as media players, browsers, etc., used to implement various application services. Here, the program implementing the method of this embodiment of the invention can be included in the application.

[0123] Based on the same inventive concept as the foregoing embodiments, this embodiment also provides a computer-readable storage medium storing a computer program. The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc. When the computer program stored in the computer-readable storage medium is executed by a processor, it implements the above-described data annotation method. For the specific steps implemented when the computer program is executed by the processor, please refer to [link to relevant documentation]. Figure 1 The description of the illustrated embodiments will not be repeated here.

[0124] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, which includes not only the elements listed but also other elements not expressly listed.

[0126] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A data labeling method, characterized by, The method includes: In response to a user's data annotation request, a target annotation model is determined from the model library based on the model identifier information specified by the user; the model library includes at least one user-specific annotation model, which is provided by the user and conforms to the interface specifications preset by the data annotation platform; The target annotation model is invoked to annotate the dataset to be annotated corresponding to the data annotation request, and the obtained annotation results are output.

2. The data labeling method of claim 1, wherein, The method further includes: Receive model service images provided by users according to the preset interface specifications of the data annotation platform; Deploy the model service image to a container cluster to generate a callable user-specific labeled model instance; Register the user-specific labeled model instance to the model library.

3. The data labeling method of claim 1 or 2, wherein, The step of determining the target labeled model from the model library based on the model identifier information specified by the user includes: Based on the annotation mode type of the data annotation request, determine the interface specifications that the required model service should meet; Based on the model identifier information specified by the user, models that conform to the interface specification are selected from the model library and used as the target labeled models.

4. The data annotation method according to claim 3, characterized in that, If the annotation mode type is to annotate each data item individually, then the interface specification is the first interface specification; the first interface specification is configured to require the model service to receive a single data item input and return structured annotation information. If the annotation mode type is dataset-wide annotation, then the interface specification is the second interface specification; the second interface specification is configured to require the model service to receive the dataset storage path and sample identifier mapping relationship, and return the storage address of the annotation result file.

5. The data labeling method of claim 3, wherein, The step of calling the target annotation model to annotate the dataset corresponding to the data annotation request includes: The annotation mode type in response to the data annotation request is to annotate each data point individually, inputting the data samples to be annotated one by one into the target annotation model for inference, and returning the single annotation result in real time; In response to the data annotation request, the annotation mode type is overall dataset annotation, and the storage path of the dataset to be annotated and the identification mapping relationship of each data sample are sent to the target annotation model; wherein, the dataset to be annotated is a dataset stored according to a specific organizational form, the specific organizational form including at least one of recording time series information, data correlation relationship or multi-source fusion information; the target annotation model is configured to perform overall inference on the dataset to be annotated based on the specific organizational form and return the storage address of the annotation result file; the format of the annotation result file is consistent with the specific organizational form of the dataset to be annotated.

6. The data annotation method according to claim 1, characterized in that, Before the annotation results obtained from the output, the following are also included: Obtain the initial annotation results of the target annotation model on the dataset to be annotated; According to the category mapping rules preset for the target annotation model, the object categories in the initial annotation results are mapped to the standard annotation categories in the platform annotation system to obtain the annotation results; the category mapping rules define the correspondence between the sub-categories output by the user model and the standard categories in the platform annotation system.

7. The data annotation method according to claim 1, characterized in that, The model library also includes general annotation models provided by the data annotation platform; the method further includes: When the user is using the service for the first time or has not specified a user-specific annotation model, the general annotation model will be determined as the target annotation model.

8. The data annotation method according to claim 1, characterized in that, The method further includes: The final annotation results are used as training data to optimize the corresponding user-specific annotation model; The optimized user-specific annotation model is updated to the model library.

9. An electronic device, characterized in that, include: A processor and a memory storing a computer program, wherein, when the processor executes the computer program, the steps of the data annotation method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the steps of the data annotation method according to any one of claims 1 to 8.

Images