Data annotation methods, devices, electronic equipment and storage media

Abstract

This invention discloses a data annotation method, apparatus, electronic device, and storage medium. The method includes: acquiring data to be annotated based on event trigger conditions; displaying the data to be annotated according to a preset display style; acquiring annotation information for the data to be annotated by a user; and, upon receiving the annotation information from the user, annotating the data to be annotated as labeled data based on the annotation information; and determining the user's reward points and achievement points based on the annotation quality of the labeled data. This invention, by acquiring the annotation information for the user's annotated data through event trigger conditions, can obtain information useful for driving, helping the driving assistance system to better improve itself; by determining the user's corresponding points based on the quantity and quality of the annotated data, it can provide positive incentives and appropriate rewards to the user, improve the user experience, ensure continuous user participation, and achieve high-quantity and high-quality data annotation.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to data annotation methods, devices, electronic devices, and storage media. Background Technology

[0002] In an era where intelligent connected vehicles are becoming mainstream, more and more vehicles are equipped with autonomous driving assistance systems (L2 / L2+). However, with the widespread adoption of vehicle intelligence, several problems urgently need to be addressed and improved. Among these, how to effectively ensure human-machine interaction and how to effectively collect data to help autonomous driving continuously improve and iterate have become two important topics. Current technologies suffer from inefficiency in labeling data within the vehicle's detection area. They also fail to provide interactive displays that allow human drivers to clearly understand the working status and behavioral intentions of the driver assistance system, nor can they effectively transmit driver feedback to the system. This hinders the improvement of the autonomous driving assistance system, reduces user experience, and fails to guarantee continuous user engagement. Summary of the Invention

[0003] In view of this, the present invention provides a data annotation method, system, electronic device and storage medium that can obtain information useful for driving, enabling the driving assistance system to better improve itself, enhance the user experience, ensure continuous user engagement, and achieve high quantity and high quality data annotation.

[0004] According to one aspect of the present invention, an embodiment of the present invention provides a data annotation method, the method comprising:

[0005] The data to be labeled is obtained according to the event tracking conditions, and the data to be labeled is displayed according to the preset display style.

[0006] Obtain annotation information for the data to be annotated by the user, and when receiving the annotation information from the user, annotate the data to be annotated as labeled data according to the annotation information;

[0007] The user's reward points and achievement points are determined based on the quality of the labeled data.

[0008] According to another aspect of the present invention, embodiments of the present invention also provide a data annotation apparatus, the apparatus comprising:

[0009] The data display module is used to obtain the data to be labeled according to the data point triggering conditions and display the data to be labeled according to the preset display style.

[0010] The annotation module is used to obtain annotation information of the data to be annotated by the user, and when it receives the annotation information from the user, it annotates the data to be annotated as annotated data according to the annotation information;

[0011] The points determination module is used to determine the quantity and quality of the labeled data, and to determine the user's reward points and achievement points based on the quality of the labeled data.

[0012] According to another aspect of the present invention, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0013] At least one processor; and

[0014] A memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data annotation method according to any embodiment of the present invention.

[0016] According to another aspect of the present invention, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions for causing a processor to execute and implement the auxiliary data annotation method described in any embodiment of the present invention.

[0017] In this embodiment of the invention, by acquiring annotation information of data to be annotated by the user, and annotating the data to be annotated according to the annotation information received from the user, useful information for driving can be obtained, thereby better assisting the driving assistance system and enabling the driving assistance system to improve itself. The reward points and achievement points for the user are determined according to the annotation quality of the data to be annotated, so as to attract the user to annotate the data to be annotated, which can provide the user with sufficient positive incentives and appropriate reward measures, improve the user experience, and thus ensure the user's continuous participation, so as to achieve high quantity and high quality data annotation and information collection.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a data annotation method provided in an embodiment of the present invention;

[0021] Figure 2 A flowchart illustrating another data annotation method provided in an embodiment of the present invention;

[0022] Figure 3 A flowchart illustrating yet another data annotation method provided in an embodiment of the present invention;

[0023] Figure 4 This is a structural block diagram of a data annotation device provided in an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of the structure of an electronic device provided for implementing embodiments of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "target," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] In one embodiment, Figure 1This is a flowchart of a data annotation method provided in an embodiment of the present invention. This embodiment is applicable to situations where users perform auxiliary data annotation on scene data. The method can be executed by a data annotation device, which can be implemented in hardware and / or software.

[0028] like Figure 1 As shown, the method includes:

[0029] S110. Obtain the data to be labeled according to the embedded point trigger conditions, and display the data to be labeled according to the preset display style.

[0030] The event tracking trigger conditions can be understood as the trigger conditions for users to actively or passively annotate relevant data. The data to be annotated can be understood as the data recorded when an object identified by a preset algorithm is at a low confidence level. Of course, the data to be annotated can be image information, video information, etc., and this embodiment does not impose any limitations.

[0031] In this embodiment, the preset display style can represent the relevant display parameters corresponding to the data to be labeled, including the font size, content layout, and text color of the data on the display screen, etc., which are not limited in this embodiment. In this embodiment, through the gamified rendering of the display style and the corresponding page layout, a game-like immersive experience is provided through the display screen, providing users with a gamified interactive experience.

[0032] In this embodiment, the trigger condition for data tracking can be when the vehicle is stationary and the user actively tags the data to be tracked; or it can be when the vehicle is in motion and the user actively or passively tags the data to be tracked. This embodiment does not impose any restrictions on this.

[0033] In this embodiment, corresponding data to be labeled can be obtained based on the corresponding data point triggering conditions, and then displayed according to a preset display style. In some embodiments, when the data point triggering condition is that the vehicle is stationary, the data points triggered by the shadow mode in the cloud server can be used as the data to be labeled. The rendering engine and game engine are called according to the preset display style to generate the display content of the data to be labeled for corresponding display. In other embodiments, when the data point triggering condition is that the vehicle is moving, the target objects or special scenes identified in the vehicle detection area by the machine vision algorithm built into the shadow mode can be obtained in real time, and the target objects or special scenes can be displayed according to the preset display style. Alternatively, by monitoring the labeling command actively triggered by the user, the vehicle scene data collected by the vehicle's own sensors at the current moment can be read according to the labeling command, and the scene data can be used as the information for data point triggering.

[0034] S120. Obtain the annotation information of the data to be annotated by the user, and when the annotation information is received from the user, annotate the data to be annotated as annotated data according to the annotation information.

[0035] The annotation information can be understood as the annotation information of the corresponding object to be annotated displayed by the Game-Assisted Training Software Development Kit (SDK). The annotation information displayed by the Game-Assisted Training SDK can be used to annotate irregularly shaped objects in the data to be annotated. The annotated data refers to the data obtained after annotating the data to be annotated based on the displayed annotation information; it can be understood as the data having been annotated.

[0036] In this embodiment of the invention, the annotation information includes at least label annotations and attribute information annotations for irregularly shaped objects in the annotation data. The label annotation can be understood as the possible corresponding labels displayed when classifying irregularly shaped target objects in the annotation data. Of course, an irregularly shaped target object can correspond to one label or multiple labels; this embodiment does not impose any limitations. The attribute information annotation can be understood as a detailed description of the irregularly shaped objects in the annotation data, which may include descriptions of the target object's state, whether it contains goods, its shape, and its size, etc. This embodiment does not impose any limitations.

[0037] In this embodiment of the invention, by identifying the object type corresponding to the data to be labeled, the preset label option corresponding to the current object type is determined. If the preset label option corresponding to the object type exists, the preset label option is highlighted and the corresponding preset label option is prompted. If the preset label option corresponding to the object type does not exist, the user is prompted to create the label option corresponding to the object type and the corresponding attribute description information. In some embodiments, the selection information and corresponding description information of the preset label option can be collected; or, the user input information received by the input display control can be collected. The corresponding selection information and description information or the user input information can be used as annotation information to annotate the data to be labeled, and the annotation relationship of the data to be labeled is generated as the annotation data. In some embodiments, the annotation icon corresponding to the data to be labeled can also be used to merge the annotation icon with the data to be labeled, and the data to be labeled can be annotated according to the merged information. The data to be labeled and the corresponding annotation information are used as the annotation data. This embodiment does not limit this.

[0038] S130. Determine the quantity and quality of the labeled data, and determine the user's reward points and achievement points based on the quality of the labeled data.

[0039] The number of annotations refers to the quantity of annotations a user adds to the data; annotation quality can be understood as the quality of the annotation information provided by the user. Reward points can be redeemed for physical prizes, and achievement points, also known as credit points, reflect a user's creditworthiness.

[0040] In this embodiment, reward points and achievement points for users can be determined based on the quantity and quality of their annotations. Specifically, the quantity and quality of user-annotated data can be statistically analyzed at a time granularity. Once the quantity and quality reach a certain threshold, the user-reported annotation data results are analyzed and processed accordingly to determine reward points and achievement points for the user, thereby encouraging users to annotate data. Alternatively, reward points and achievement points can be determined through user annotation evaluations; this embodiment does not impose any limitations on this approach.

[0041] The technical solution described above in this invention improves the accuracy of annotation by acquiring annotation information of data to be annotated by the user and annotating the data according to the annotation information received from the user. This better assists the driving assistance system, enabling it to improve itself and enhance the user experience. Furthermore, by determining the user's reward points and achievement points based on the annotation quality of the data, the system attracts users to annotate the data, providing ample positive incentives and appropriate rewards to ensure continuous user participation and achieve high-volume and high-quality data annotation.

[0042] In one embodiment, the data annotation method further includes:

[0043] The labeled data and its corresponding labeling information are packaged and uploaded to the cloud server.

[0044] The cloud server can receive the results corresponding to the labeled data obtained by the user.

[0045] In this embodiment of the invention, the game-based user-assisted training SDK interacts with a cloud server to upload and receive labeled data. The uploaded labeled data may include feedback information from users after they have labeled the data. After the labeled data is uploaded to the cloud server, the cloud server preprocesses the labeled data, including data anonymization, data encryption, and data screening and checking. The anonymized and encrypted labeled data can also be sent to a backend server for data analysis, classification, and as feedback from the intelligent driving system before entering the training platform, thereby helping the algorithm to further optimize and iterate.

[0046] In one embodiment, Figure 2 This is a flowchart of another auxiliary data annotation method provided in an embodiment of the present invention. Based on the above embodiments, this embodiment further refines the following steps: obtaining the data to be annotated according to the event triggering conditions, displaying the data to be annotated according to a preset display style, obtaining the annotation information of the data to be annotated by the user, annotating the data to be annotated as annotated data according to the annotation information, and determining the user's reward points and achievement points according to the annotation quality of the annotated data.

[0047] like Figure 2 As shown, the data annotation method in this embodiment of the invention may specifically include the following steps:

[0048] S210. If the vehicle stops moving, download offline data from the cloud server as the data to be labeled, and display the data to be labeled according to the preset display style.

[0049] The data to be labeled includes long-tail scene data that was triggered by shadow mode and uploaded to the cloud server in historical scenarios.

[0050] The shadow mode can be understood as using built-in algorithms to identify scenes containing irregularly shaped objects or scenes of interest, and to record and store the identified special scenes or scenes of interest.

[0051] In this embodiment of the invention, offline data may include long-tail scene data that was previously triggered in shadow mode and uploaded to the cloud server. Long-tail scene data includes at least one of the following: raw data corresponding to the triggering of the tracking point, structured data, and tag data corresponding to the tracking point. The raw data refers to the original images or videos collected by the autonomous vehicle's own sensors corresponding to the long-tail tracking point; the structured data refers to the data that machine recognition algorithms can read during data transmission; and the tag data corresponding to the tracking point refers to the tag data inherent to the tracking point itself, which can characterize the triggering reason for the tracking point.

[0052] In this embodiment, when the vehicle is stationary, i.e. in an offline data interaction scenario, offline data can be downloaded from the cloud server as data to be labeled. When the user clicks or touches to enter the user-assisted training SDK, the preset display style is read on the display screen in the vehicle's intelligent cockpit domain. The rendering engine and game engine are called according to the preset display style to generate the display content of the data to be labeled, so as to display the labeling information corresponding to the data to be labeled.

[0053] In one embodiment, displaying the data to be labeled according to a preset display style includes:

[0054] Read the preset display style;

[0055] The rendering engine and game engine are invoked according to the preset display style to generate the display content of the data to be labeled;

[0056] Transmit the display content to the display device to display the data to be labeled.

[0057] The rendering engine represents the displayed content, such as images or videos, and determines the display method. It can showcase the realism, smoothness, and attractiveness of the image. The image, scene, and color effects seen by the user are all directly controlled by the engine. A game engine refers to the core component of interactive real-time graphics applications, providing a range of visual development tools and reusable components.

[0058] In this embodiment, by reading a pre-set display style, the rendering engine and game engine are called according to the pre-set display style to generate the display content of the data to be labeled, and the display content is transmitted to the display device to display the data to be labeled, so that the user can label the displayed data accordingly.

[0059] In one embodiment, to facilitate understanding of the interaction when the vehicle is stationary, the following describes the interaction scenario between the game-based user assistance training SDK, the user, the vehicle-side intelligent driving domain, the cloud server, and the backend server when the vehicle is stationary: The data to be labeled (de-sensitized videos and images) collected through the shadow mode in the vehicle-side intelligent driving domain and stored in the cloud server is sent to the game-based user assistance training SDK in the vehicle-side intelligent cockpit domain. Under safe conditions, the user can participate in an interactive game in the vehicle by actively calling the desktop application of the game-based user assistance training SDK in the intelligent cockpit domain to label the data to be labeled.

[0060] For example, when a user enters a parking lot and has completed the driving task and is in a parked state, the system detects that the vehicle has stopped moving. It then downloads the data collected in shadow mode and stores it on the cloud server as the data to be labeled. The user then taps the central control display screen to access the desktop application of the game-based user assistance training SDK. The SDK displays images or videos on the screen and highlights irregularly shaped objects in the data to be labeled, prompting for corresponding label options. The user can provide label information via voice or touchscreen and input relevant descriptive information about the irregularly shaped objects in the data to be labeled. Upon receiving the user's label information and description of the object, the SDK labels the object based on these information. After labeling, the SDK compresses and packages the user-input information, labels, and corresponding images or videos, and sends them to the cloud server.

[0061] S320: Detects vehicle movement, reads long-tail scene data triggered by the shadow mode in real time, and uses the long-tail scene data as data to be labeled, displaying the data to be labeled according to the preset display style.

[0062] In this embodiment, when the vehicle is in normal driving mode, the shadow mode data points in the vehicle-side intelligent driving domain are triggered. The system reads long-tail scene data triggered in shadow mode in real time. The triggered data points are transmitted to the Gamified User Assisted Training SDK. The rendering engine and game engine render the data to be labeled and generate the display content for the labeled data. The SDK then displays the corresponding labeling information to the user, requesting the user to provide corresponding labels and descriptions for the irregularly shaped objects in the labeled data. Upon receiving feedback from the user regarding the irregularly shaped objects in the labeled data, the Gamified User Assisted Training SDK packages the label information and the corresponding labeled data together and uploads them to the cloud, providing the user with certain achievement and reward points.

[0063] In one embodiment, to facilitate understanding of the interaction when the vehicle is in motion, the following describes the interaction scenario between the gamified user assistance training SDK, the user, the vehicle-side intelligent driving domain, the cloud server, and the backend server, under the condition that the long-tail scene data triggered by the shadow mode is annotated in real time when the vehicle is in motion: During vehicle operation, the long-tail scene data triggered by the shadow mode in the vehicle-side intelligent driving domain is recorded in real time and packaged into packets, and then transmitted to the gamified user assistance training SDK in the vehicle-side intelligent cockpit domain; the gamified user assistance training SDK provides the real-time long-tail scene data to the user through the display screen, and the user can choose to participate in the auxiliary annotation of the real-time long-tail scene data.

[0064] For example, during vehicle operation, the shadow mode in the vehicle-side intelligent driving domain discovers and records irregularly shaped target objects in long-tail scene data through embedded points. For instance, it could be a picture of a tricycle loaded with goods. The irregularly shaped target object is then transmitted to the gamified user-assisted training SDK in the vehicle-side intelligent cockpit domain. The gamified user-assisted training SDK then pushes the rendered data containing the irregularly shaped target object to the user's display screen in a gamified manner, requesting the user to label the irregularly shaped target object in the picture with corresponding tags and labeling information. The user provides the labeling information to the gamified user-assisted training SDK, which then packages the labeling information and the data containing the irregularly shaped target object together and sends it to the cloud server, giving the user certain achievement and reward points.

[0065] In one embodiment, determining long-tail scene data includes:

[0066] Acquire the target object within the vehicle detection area, wherein there is at least one target object;

[0067] Obtain the real-time scene recognition results of the vehicle using machine vision algorithms;

[0068] The target object is compared with the recognition result;

[0069] When the confidence level of the target object is lower than the preset confidence threshold, the target object is treated as an irregular target object, and it is determined to meet the long-tail tracking point requirement. The attribute information of the irregular target object is used as long-tail scene data. The long-tail scene data includes at least one of the following: raw data, structured data, and the tag corresponding to the tracking point.

[0070] In this context, irregularly shaped target objects can be understood as objects in the long-tail scene data corresponding to the embedded points. These objects can be dynamic or static, and there must be at least one such object. For example, an irregularly shaped target object could be a tricycle loaded with goods, a truck carrying cars, a car, etc. Pre-stored objects can be understood as objects pre-stored on a cloud server, such as cars, bicycles, pedestrians, etc. Confidence level can be understood as the probability that the target object and the pre-stored objects on the cloud server are within a certain allowable error range during recognition; this probability is the corresponding confidence level. The pre-set confidence threshold refers to the pre-set confidence threshold for the target object. This pre-set confidence threshold can be set according to user needs or experience; this implementation does not impose any restrictions.

[0071] In this embodiment of the invention, machine vision algorithms or AI algorithms can be used to call the sensors, cameras and other acquisition devices configured on the vehicle itself to collect irregular target objects in the long-tail scene data corresponding to the long-tail embedded points within the visual range of the vehicle's acquisition devices, and the shadow mode is used to collect irregular target objects in the long-tail scene data.

[0072] In this embodiment of the invention, the built-in algorithms in the shadow mode, such as AI algorithms or machine recognition algorithms, can identify irregularly shaped target objects in certain special scenarios. These irregularly shaped target objects are then compared with objects pre-stored in a cloud server. For example, during autonomous driving, machine vision perceives irregularly shaped target objects on the road. If the machine learning algorithm recognizes that the current irregularly shaped target object has uncertainty, the confidence level of the irregularly shaped target object will be very low. In this case, the scene is recorded as a special scene containing irregularly shaped target objects.

[0073] In this embodiment of the invention, after identifying and comparing the irregular target object collected by the acquisition device with the object pre-stored in the cloud server through relevant identification algorithms, if the confidence level of the identified irregular target object is lower than the preset confidence threshold, it is determined that the long-tail embedding is satisfied, and the scene information containing the irregular target object is used as long-tail scene data. The long-tail scene data includes at least one of the following: the original data corresponding to the long-tail scene data, the structured data, and the tag corresponding to the embedding.

[0074] For example, during autonomous driving, machine vision perceives an irregularly shaped object on the road as a tricycle loaded with goods. When the machine learning algorithm identifies the tricycle loaded with goods, it may be uncertain what the object is. In this case, the confidence level of the tricycle loaded with goods will be very low. When the confidence level of the tricycle loaded with goods is low, it can be used as a data point. When the confidence level is lower than a preset confidence threshold of 10%, the data point is activated to record the scene information containing the irregularly shaped object as long-tail scene data, including the original data, structured data and the label corresponding to the data point.

[0075] S230: Detect user-initiated annotation information triggered by the user in recognizing an abnormal situation, and read the long-tail scene data collected in shadow mode at the current moment.

[0076] In this embodiment, if the user identifies a relatively abnormal target object or situation while the vehicle is in motion, the user can actively trigger the desktop application of the Gamified User Assisted Training SDK. When the user actively triggers the annotation information command after identifying the abnormal situation, the system reads the vehicle data collected in the shadow mode at the current moment, records the relatively abnormal target object or situation, and adds tags and corresponding descriptive information.

[0077] S240. Set the long-tail scene data as a shadow mode tracking point and display the data to be labeled according to the preset display style.

[0078] In this embodiment, the vehicle scene corresponding to the vehicle data collected in the shadow mode at the current moment is used as the shadow mode's data entry point. The data to be labeled is displayed according to the preset display style, and the relevant data corresponding to the scene is recorded and uploaded to the cloud.

[0079] In this embodiment, to facilitate understanding of the interaction when the vehicle is in motion, and in the case where the user actively triggers real-time data annotation when observing an anomaly, the interaction scenario between the gamified user-assisted training SDK and the user, the vehicle-side intelligent driving domain, the cloud server, and the backend server is described as follows: During vehicle operation, the user actively triggers real-time data annotation when observing an anomaly: When the vehicle is in motion, the user observes a relatively abnormal target object or situation and actively triggers the desktop application of the gamified user-assisted training SDK to record the abnormal target object or situation and add tags and annotation information. At the same time, this triggering node will serve as a shadow mode tracking point, recording the relevant data corresponding to the scenario and uploading it to the cloud server. For example, while the vehicle is in motion, a user observes a tricycle loaded with goods and wants to record this scene. The user can trigger the one-click recording function or click to select the tricycle to record the current image and video of the tricycle loaded with goods. The user adds tags and information to the tricycle loaded with goods in the image and submits it. The gamified user assistance training SDK in the intelligent driving domain packages the tag information, images and videos and sends them to the cloud server. Through the user achievement system in the vehicle-side intelligent cockpit domain, the gamified user assistance training SDK gives the user appropriate rewards and points.

[0080] S250: Identify the object type corresponding to the data to be labeled based on the pre-configured data labels.

[0081] In this context, data tags can be understood as tags corresponding to irregularly shaped target objects in a tag library. For example, if the object type is a tricycle loaded with goods, the corresponding data tag could be a tricycle loaded with goods, etc. This embodiment does not impose any limitations.

[0082] In this embodiment of the invention, different data tags correspond to different irregular target objects, and the object type of the irregular target object corresponding to the data to be labeled is identified according to the pre-configured data tags.

[0083] S260. Determine if there is a preset label option corresponding to the object type. If it exists, execute S270; otherwise, execute S280.

[0084] Among them, the preset label options can be understood as the pre-configured label options corresponding to the data to be labeled.

[0085] In this embodiment, when there are corresponding label options for the irregular target object, the number of label options can be one or more. The user can make the corresponding selection based on the label options corresponding to the irregular target object. When there are no corresponding label options for the irregular target object, the user can be prompted to create preset label options corresponding to the object type.

[0086] S270, Prompt for preset label options corresponding to the object type.

[0087] In this embodiment, if there are corresponding tag options for the target object, one or more tag options corresponding to the object type of the irregular target object will be displayed.

[0088] S280. Prompt the user to create preset label options corresponding to the object type, and generate an input display control at the display position of the irregular target object in the data to be labeled.

[0089] In this embodiment, if there is no corresponding label option for the object type of the irregular target object, the user can be prompted to create a preset label option corresponding to the object type. An input display control is generated at the display position of the irregular target object in the data to be labeled, so that the user can manually create the corresponding label and related description information for the irregular target object.

[0090] S290, Collect the selection information of the preset label options and the description information of the corresponding object type; or, collect the user input information received by the input display control.

[0091] In this embodiment, the selection information of the preset label option of the irregular target object in the data to be labeled corresponding to the user input, as well as the description information of the irregular target corresponding to the user input; or, the user input information received by the input display control is collected. It should be noted that the user input method can be voice input, touch input, or click input, and this embodiment does not limit it.

[0092] S2100, Use the preset label options corresponding to the selected information and the description information of the corresponding object type as annotation information, or use the user input information as annotation information.

[0093] In this embodiment, the user-input selection information of the preset label options for the irregular target objects in the data to be labeled, and the description information of the corresponding object type are used as the labeling information of the irregular target objects in the data to be labeled; or, the user input information is used as the labeling information of the irregular target objects in the data to be labeled.

[0094] S2110. Generate the annotation relationship corresponding to the annotation information for the data to be annotated, and use it as annotation data.

[0095] In this embodiment, based on the annotation information of the irregular target objects in the data to be annotated, the annotation relationship between the data to be annotated and the annotation information is generated as the annotation data.

[0096] S2120. Statistically analyze the number and quality of user-annotated data according to time granularity.

[0097] The time granularity can be in the form of days, hours, or months.

[0098] In this embodiment, the number and quality of annotations provided by users can be statistically analyzed according to time granularity. Once the number of annotations reaches a certain level, the annotation data results reported by users will be analyzed and processed accordingly.

[0099] S2130. Determine the reward points and achievement points for users based on the number and quality of annotations.

[0100] In this embodiment of the invention, reward points and achievement points for users can be determined based on the quantity and quality of user-annotated data, and the user's feedback on the annotation data results. It should be noted that reward points for users will be coupled with corresponding achievements, such as credit points and reward points (which can be used to redeem physical rewards). While users actively participate in providing valid data feedback and tag input, sufficient positive incentives and appropriate reward measures are provided to ensure continuous user engagement, thereby achieving high-quantity and high-quality data annotation and information collection. Conversely, when a user maliciously and / or mislabels target objects in long-tail scene data, targeted feedback will be provided, and the user's credit points will be deducted, thereby removing their participation limit for the feature experience.

[0101] It should be noted that a user evaluation system can be used to screen high-quality annotation users based on the quality of the annotation data. Screening criteria typically include, but are not limited to, driving style, knowledge of autonomous driving, system familiarity, and questionnaire results. At the same time, the evaluation system will regularly push out assessments to ensure the validity of users' qualifications, thereby guaranteeing the professionalism of users participating in auxiliary annotation.

[0102] The above-described technical solution of this invention, by detecting vehicle stoppage and downloading offline data from a cloud server as the data to be labeled; detecting vehicle movement and reading long-tail scene data triggered by shadow mode in real time; detecting user-initiated labeling information triggered by an abnormal situation and reading vehicle data collected in shadow mode at the current moment; setting the vehicle scene of the vehicle data as a shadow mode tracking point, can obtain data useful for the assisted driving system, improve the safety of the assisted driving system, and help the driving assistance system to better improve itself; by statistically analyzing the number and quality of user-labeled data according to time granularity, and determining reward points and achievement points for users based on the number and quality of labels to attract users to label the data to be labeled, it can further provide users with positive incentives and appropriate rewards, improve user experience, ensure continuous user participation, and achieve high-quantity and high-quality data labeling.

[0103] In one embodiment, to facilitate a better understanding of the data annotation method, Figure 3 This is a flowchart illustrating another data annotation method provided in an embodiment of the present invention. The long-tail scene data in this embodiment is the data to be annotated from the above embodiments.

[0104] This invention implements gamified user-assisted training. It collects long-tail scenario data for the algorithm using a shadow mode, records this data after algorithm tracking points are triggered in the intelligent driving domain, and pushes it to the human-computer interaction interface. Gamified interactive experiences are used to attract users to actively or passively participate in interacting with the long-tail scenario data, thus aiding in its annotation. The annotated long-tail scenario data is uploaded to a cloud server via online transmission. Analysis and processing by the backend server then help iterate and optimize the driving assistance system's algorithm.

[0105] Meanwhile, this embodiment of the invention can use the user evaluation system in the vehicle-side intelligent cockpit domain to screen high-quality users and grant them corresponding quotas to participate in the functional experience, thereby ensuring the professionalism of users participating in auxiliary labeling. In addition, this embodiment, while creating a game-like interactive experience, will also be supplemented by a user development and achievement system. While users actively participate in the feedback of effective data and the input of tags, they will be given sufficient positive incentives and appropriate rewards, thereby ensuring the continuous participation of users and achieving high-volume and high-quality data labeling and information collection.

[0106] In this embodiment of the invention, the gamified user assistance training SDK is deployed in the vehicle-side intelligent cockpit domain and interacts with the cloud server, the vehicle-side intelligent driving domain, the vehicle display screen, and the user.

[0107] like Figure 4 As shown, the specific steps of the data annotation method are as follows:

[0108] a1. Collect long-tail scenario data of the algorithm through the shadow mode in the vehicle-side intelligent driving domain, record and store the data after the algorithm is triggered by the embedding point in the intelligent driving domain, and upload the packaged and stored long-tail scenario data to the cloud server in real time through online backhaul.

[0109] In this embodiment of the invention, the gamified user assistance training SDK frequently interacts with the shadow mode in the intelligent driving domain of the vehicle. The shadow mode collects long-tail scenario data (useful data for the driving assistance system) and provides it to the gamified user assistance training SDK. The shadow mode mainly includes the following five core functions: 1) Algorithm long-tail tracking: The algorithm pre-embeds some tracking points in the shadow mode that help with algorithm optimization. These tracking points are triggered in real-world scenarios; 2) Data reading: After a tracking point is triggered, the corresponding long-tail scenario data cached in the system is read based on the trigger timestamp. This data is used to record long-tail scenario data within a certain time period before and after the tracking event; 3) Data recording: The raw data, structured data, and tracking point tags obtained through the long-tail scenario data are recorded; 4) Data storage: The recorded long-tail scenario data is packaged and stored using a specific format and structure; 5) Data upload to the cloud: The packaged and stored long-tail scenario data is uploaded to the cloud server in real time via a 4G network through online backhaul.

[0110] a2. Obtain the annotation information of the corresponding long-tail scene data input by the user through the vehicle-side intelligent cockpit domain, and attract users to actively or passively participate in the interaction with the long-tail scene data through gamified interactive experience settings, perform auxiliary annotation of the long-tail scene data, and upload the annotated data to the cloud through online back-end transmission, thereby helping the algorithm of the driving assistance system to iterate and optimize through back-end analysis and processing.

[0111] In this embodiment of the invention, the gamified user-assisted training SDK is typically deployed in the vehicle's intelligent cockpit domain and includes five core modules: 1) Data storage and packaging: This module provides the ability to package and store user-annotated information along with data to be labeled, as well as the ability to receive and store data received from the intelligent driving domain; 2) Data upload to the cloud: This module is used to interact with the compliant cloud, mainly serving the uploading and receiving of data; 3) User evaluation system: This module is mainly used to screen users who meet the criteria for participating in the functional experience based on specific conditions, which typically include, but are not limited to, driving style, knowledge accumulation of autonomous driving, system familiarity, questionnaire results, etc. At the same time, this evaluation system will periodically push assessments to ensure the valid qualifications of users; 4) User achievement system: This module is used to provide task mode and reward mechanism services. After users complete specific tasks (i.e., data annotation of a specific quantity and quality), the system will provide corresponding achievements (credit points) and reward points (points can be used to redeem physical rewards). Conversely, if a user maliciously mislabels the data, the system will provide targeted feedback and deduct the user's credit score, thereby eliminating their participation in the functional experience; 5) Gamified Interaction System: This module will call the rendering engine and game engine in the intelligent cockpit domain to provide a game-like immersive experience through the vehicle's display screen and provide users with a gamified interactive experience.

[0112] a3. Receive feedback information from the user's data annotation obtained from the vehicle-side intelligent cockpit domain via the cloud server, as well as long-tail scenario data collected from the vehicle intelligent driving domain.

[0113] In this embodiment of the invention, the uploaded data mainly includes packaged data with user annotation information, while the data received from the cloud server is mainly data that has been pre-filtered by big data and is used to push to users for auxiliary annotation. After the data is uploaded to the cloud, the cloud will preprocess the data, including de-identification, encryption, and filtering checks.

[0114] a4. By receiving data from the vehicle-side intelligent cockpit domain, the quantity and quality of long-tail scene data labeled by users are statistically analyzed according to time granularity. The auxiliary data labeling results reported by users are then analyzed to determine the reward and penalty points for users.

[0115] In this embodiment of the invention, the auxiliary data annotation results are sent to the backend server for data analysis and classification, and to the intelligent driving system for annotation feedback and input into the training platform, thereby helping the algorithm to perform further optimization and iteration. At the same time, the auxiliary data annotation results fed back by users are analyzed to determine the reward and penalty points for users.

[0116] In one embodiment, Figure 4This is a structural block diagram of a data annotation device provided in an embodiment of the present invention. This device is suitable for assisting users in annotating scene data, and can be implemented in hardware or software. Figure 4 As shown, the device includes: a data display module 410, a labeling module 420, and an integral determination module 430.

[0117] The data display module 410 is used to obtain the data to be labeled according to the embedded point triggering conditions and display the data to be labeled according to the preset display style.

[0118] The annotation module 420 is used to obtain annotation information of the data to be annotated by the user, and when it receives the annotation information from the user, it annotates the data to be annotated as annotated data according to the annotation information;

[0119] The points determination module 430 is used to determine the number and quality of the annotation data, and to determine the user's reward points and achievement points based on the number and quality of the annotation data.

[0120] In this embodiment of the invention, the annotation module obtains annotation information of the data to be annotated by the user, and annotates the data according to the annotation information, which can improve the accuracy of annotation, thereby better assisting the driving assistance system, enabling the driving assistance system to better improve itself, and enhancing the user experience; the points determination module determines the quantity and quality of the annotated data, and determines the user's reward points and achievement points according to the quantity and quality of the annotated data, which can provide users with sufficient positive incentives and appropriate reward measures, thereby ensuring the user's continuous participation and achieving high quantity and high quality data annotation.

[0121] In one embodiment, the embedded point triggering condition includes the vehicle stopping; correspondingly, the data display module 410 is specifically used for:

[0122] If the vehicle stops moving, offline data is downloaded from the cloud server as the data to be labeled.

[0123] The data to be labeled includes long-tail scene data that was triggered by shadow mode and uploaded to the cloud server in historical scenarios.

[0124] In one embodiment, the data point triggering condition includes vehicle movement; correspondingly, the data display module 410 is specifically used for:

[0125] The vehicle movement is detected, and the long-tail scene data triggered by the shadow mode is read in real time, and the long-tail scene data is used as the data to be labeled.

[0126] In one embodiment, the event tracking triggering condition includes user-initiated triggering of annotation information; correspondingly, the data display module 410 includes:

[0127] The data reading unit is used to detect the user-initiated annotation information issued by the user when an abnormal situation is identified, and to read the vehicle data collected in shadow mode at the current moment;

[0128] The tracking point determination unit is used to set the vehicle scene of the vehicle data as the tracking point of the shadow mode.

[0129] In one embodiment, the method for determining the long-tail scene data includes:

[0130] Acquire target objects within the vehicle detection area, wherein there is at least one target object;

[0131] Obtain the real-time scene recognition results of the vehicle using machine vision algorithms;

[0132] The target object is compared with the recognition result;

[0133] When the confidence level of the target object is lower than the preset confidence threshold, the target object is regarded as an irregular target object, and the event triggering condition is determined to be met. The scene information of the irregular target object is regarded as the long-tail scene data, wherein the long-tail scene data includes at least one of the following: raw data, structured data, and the tag corresponding to the event.

[0134] In one embodiment, the data display module 410 includes:

[0135] Read the preset display style;

[0136] The rendering engine and game engine are invoked according to the preset display style to generate the display content of the data to be labeled;

[0137] The display content is transmitted to the display device to display the data to be labeled.

[0138] In one embodiment, the annotation module 420 includes:

[0139] The identification unit is used to identify the object type corresponding to the irregular target object in the data to be labeled based on the pre-configured data labels;

[0140] An option determination unit is used to determine whether there is a preset label option corresponding to the object type;

[0141] The first option prompting unit is used to prompt the preset label option corresponding to the object type if it exists;

[0142] The second option prompting unit is used to prompt the user to create a preset label option corresponding to the object type if the option does not exist, and to generate an input display control at the display position of the irregular target object in the data to be labeled.

[0143] An information acquisition unit is used to acquire selection information of the preset label options and description information corresponding to the object type; or, to acquire user input information received by the input display control.

[0144] The information determination unit is used to use the preset label option corresponding to the selection information and the description information corresponding to the object type as the annotation information, or to use the user input information as the annotation information;

[0145] The data generation unit is used to generate annotation relationships corresponding to the annotation information for the irregular target objects in the data to be annotated, so as to serve as the annotation data.

[0146] In one embodiment, the integral determination module 430 includes:

[0147] The data statistics unit is used to count the number and quality of user-annotated data according to time granularity.

[0148] The points determination unit is used to determine reward points and achievement points for users based on the number of annotations and the quality of annotations, so as to attract users to annotate the data to be annotated.

[0149] In one embodiment, the device further includes:

[0150] The data upload module is used to package the labeled data and its corresponding labeled information and upload them to the cloud server.

[0151] The data annotation device provided in the embodiments of the present invention can execute the data annotation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0152] In one embodiment, Figure 5 This is a schematic diagram of an electronic device provided for implementing embodiments of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0153] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0154] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0155] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as auxiliary data annotation methods.

[0156] In some embodiments, the auxiliary data annotation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the auxiliary data annotation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the auxiliary data annotation method by any other suitable means (e.g., by means of firmware).

[0157] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0158] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0159] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0160] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0161] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0162] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0163] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0164] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data annotation method, characterized in that, The method includes: The data to be labeled is obtained according to the event tracking conditions, and the data to be labeled is displayed according to the preset display style. Identify the object type corresponding to the irregular target object in the data to be labeled based on the pre-configured data tags; Determine whether a preset label option corresponding to the object type exists; If present, one or more preset label options corresponding to the object type will be displayed so that the user can select one or more preset label options corresponding to the object type. If it does not exist, the user is prompted to create the label option corresponding to the object type, and an input display control is generated at the display position of the irregular target object in the data to be labeled. Collect the selection information of the preset label options and the description information corresponding to the object type; or, collect the user input information received by the input display control; The preset label options corresponding to the selection information and the description information corresponding to the object type are used as annotation information, or the user input information is used as annotation information; For the irregular target objects in the data to be labeled, a labeling relationship corresponding to the labeling information is generated as the labeling data; The quantity and quality of the labeled data are determined, and the user's reward points and achievement points are determined based on the quantity and quality of the labeled data.

2. The method according to claim 1, characterized in that, The triggering condition for the data point includes the vehicle stopping. Correspondingly, obtaining the data to be labeled based on the triggering condition includes: If the vehicle stops moving, offline data is downloaded from the cloud server as the data to be labeled. The data to be labeled includes long-tail scene data that was triggered by shadow mode and uploaded to the cloud server in historical scenarios.

3. The method according to claim 1, characterized in that, The triggering conditions for the data point embedding include vehicle movement. Correspondingly, obtaining the data to be labeled based on the triggering conditions includes: The vehicle movement is detected, and the long-tail scene data triggered by the shadow mode is read in real time, and the long-tail scene data is used as the data to be labeled.

4. The method according to claim 1, characterized in that, The event tracking trigger conditions include user-initiated triggering of annotation information. Correspondingly, obtaining the data to be annotated based on the event tracking trigger conditions includes: The system detects that the user has actively triggered annotation information when identifying an abnormal situation, and reads the long-tail scene data collected in shadow mode at the current moment. Set the scene of the long-tail scene data as the tracking point of the shadow mode.

5. The method according to claim 3, characterized in that, The determination of the long-tail scene data includes: Acquire target objects within the vehicle detection area, wherein there is at least one target object; Obtain the real-time scene recognition results of the vehicle using machine vision algorithms; The target object is compared with the recognition result; When the confidence level of the target object is lower than the preset confidence threshold, the target object is regarded as an irregular target object, and the conditions for triggering the embedding point are met. The scene information containing the irregular target object is regarded as the long-tail scene data. The long-tail scene data includes at least one of the following: raw data, structured data, and the tag corresponding to the embedding point.

6. The method according to claim 1, characterized in that, The step of displaying the data to be labeled according to a preset display style includes: Read the preset display style; The rendering engine and game engine are invoked according to the preset display style to generate the display content of the data to be labeled; The display content is transmitted to the display device to display the data to be labeled.

7. The method according to claim 1, characterized in that, The step of determining the user's reward points and achievement points based on the quantity and quality of the labeled data includes: The quantity and quality of user-annotated data are statistically analyzed based on time granularity. The reward points and achievement points for users are determined based on the number and quality of the annotations.

8. The method according to claim 1, characterized in that, Also includes: The labeled data and its corresponding labeled information are packaged and uploaded to the cloud server.

9. A data annotation device, characterized in that, The device includes: The data display module is used to obtain the data to be labeled according to the data point triggering conditions and display the data to be labeled according to the preset display style. The annotation module is used to obtain annotation information of the data to be annotated by the user, and when it receives the annotation information from the user, it annotates the data to be annotated as annotated data according to the annotation information; The points determination module is used to determine the user's reward points and achievement points based on the quantity and quality of the labeled data. The annotation module includes: The identification unit is used to identify the object type corresponding to the irregular target object in the data to be labeled based on the pre-configured data labels; An option determination unit is used to determine whether there is a preset label option corresponding to the object type; The first option prompting unit is used to prompt one or more preset label options corresponding to the object type if they exist, so that the user can select according to one or more preset label options corresponding to the object type; The second option prompting unit is used to prompt the user to create a preset label option corresponding to the object type if the option does not exist, and to generate an input display control at the display position of the irregular target object in the data to be labeled. An information acquisition unit is used to acquire selection information of the preset label options and description information corresponding to the object type; or, to acquire user input information received by the input display control. The information determination unit is used to use the preset label option corresponding to the selection information and the description information corresponding to the object type as the annotation information, or to use the user input information as the annotation information; The data generation unit is used to generate annotation relationships corresponding to the annotation information for the irregular target objects in the data to be annotated, so as to serve as the annotation data.

10. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data annotation method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the data annotation method according to any one of claims 1-8.

Images