Data annotation management method, device, electronic device and storage medium

Abstract

The present disclosure provides a data annotation management method, apparatus, electronic device, and storage medium. The method is applied to an artificial intelligence platform and includes: in response to a user's data annotation task request, determining first access account information of an annotation component; based on the first access account information, obtaining a session identifier of the annotation component; logging in to the annotation component based on the session identifier of the annotation component; creating a data annotation task by using the annotation component based on the data annotation task request; synchronizing the data to be annotated in the artificial intelligence platform to the data annotation task based on an access interface of the annotation component; and automatically annotating the data to be annotated in the data annotation task. This method does not require the user to input identity information to log in to the annotation component, realizes password-free login of the user to the annotation component, improves the convenience of user operations, and realizes automatic annotation of the data to be annotated, improving the data annotation efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of computers, and in particular, to a data annotation management method, apparatus, electronic device, and storage medium. Background Art

[0002] With the booming development of the artificial intelligence related industries, more and more researchers in scientific research enterprises and universities need a large amount of labeled data when conducting model training and construction. Currently, mainstream artificial intelligence platforms generally do not support the data annotation function. When users of artificial intelligence platforms have the need for data annotation tasks, they need to input their identity information separately to log in to a dedicated annotation platform, resulting in cumbersome user operations. Moreover, for artificial intelligence platforms with data annotation functions, they usually also require manual data annotation, and the data annotation efficiency is relatively low. Summary of the Invention

[0003] The present disclosure provides a data annotation management method, apparatus, electronic device, and storage medium, which do not require users to input identity information to log in to the annotation component, realize password-free login of users to the annotation component, improve the convenience of user operations, and realize automatic annotation of data to be annotated, thereby improving the data annotation efficiency.

[0004] In a first aspect embodiment of the present disclosure, a data annotation management method is proposed, which is applied to an artificial intelligence platform and includes: in response to a data annotation task request of a user, determining first access account information of an annotation component; obtaining a session identifier of the annotation component based on the first access account information; logging in to the annotation component based on the session identifier of the annotation component; creating a data annotation task by using the annotation component based on the data annotation task request; synchronizing data to be annotated in the artificial intelligence platform to the data annotation task based on an access interface of the annotation component; and automatically annotating the data to be annotated in the data annotation task.

[0005] In a second aspect embodiment of the present disclosure, an electronic device is proposed, including: a processor and a memory for storing a computer program that can run on the processor, wherein the processor is configured to execute the method described in the first aspect embodiment of the present disclosure when running the computer program.

[0006] In a third aspect embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is proposed, wherein the computer instructions are used to cause a computer to execute the method described in the first aspect embodiment of the present disclosure.

[0007] In summary, according to a data annotation management method provided by the present disclosure, which is applied to an artificial intelligence platform, it includes: in response to a data annotation task request of a creating user, determining first access account information of an annotation component; based on the first access account information, obtaining a session identifier of the annotation component; based on the session identifier of the annotation component, logging in to the annotation component; based on the data annotation task request, using the annotation component to create a data annotation task; based on an access interface of the annotation component, synchronizing data to be annotated in the artificial intelligence platform to the data annotation task; and automatically annotating the data to be annotated in the data annotation task. The method of the present disclosure realizes password-free login of the user to the annotation component by obtaining the session identifier of the annotation component, so that the artificial intelligence platform uses the session identifier to log in to the above-mentioned annotation component without the user inputting identity information, improving the convenience of user operations; and automatically annotating the data to be annotated through the artificial intelligence platform, improving the annotation efficiency of the data.

[0008] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0010] Figure 1 It is a schematic flowchart of a data annotation management method provided by an embodiment of the present disclosure;

[0011] Figure 2 It is a flow example diagram of a data annotation management method provided by an embodiment of the present disclosure;

[0012] Figure 3 It is a flow example diagram of another data annotation management method provided by an embodiment of the present disclosure;

[0013] Figure 4 It is a flow example diagram of yet another data annotation management method provided by an embodiment of the present disclosure;

[0014] Figure 5 It is a schematic flowchart of yet another data annotation management method provided by an embodiment of the present disclosure;

[0015] Figure 6 It is a schematic flowchart of another data annotation management method provided by an embodiment of the present disclosure;

[0016] Figure 7 It is a flow example diagram of still another data annotation management method provided by an embodiment of the present disclosure;

[0017] Figure 8Schematic flowchart of yet another data annotation management method provided by an embodiment of the present disclosure;

[0018] Figure 9 Flowchart example of another data annotation management method provided by an embodiment of the present disclosure;

[0019] Figure 10 Schematic flowchart of yet another data annotation management method provided by an embodiment of the present disclosure;

[0020] Figure 11 Schematic flowchart of yet another data annotation management method provided by an embodiment of the present disclosure;

[0021] Figure 12 Flowchart example of another data annotation management method provided by an embodiment of the present disclosure;

[0022] Figure 13 Schematic structural diagram of a data annotation management device provided by an embodiment of the present disclosure;

[0023] Figure 14 Schematic diagram of the hardware composition structure of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0024] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted below for clarity and conciseness.

[0025] With the booming development of the artificial intelligence-related industries, more and more researchers in scientific research enterprises and universities need a large amount of labeled data when conducting model training and construction. Currently, mainstream artificial intelligence platforms generally do not support the data annotation function. When users of artificial intelligence platforms have the need for data annotation tasks, they need to input their identity information separately to log in to a dedicated annotation platform, resulting in cumbersome user operations. Moreover, for artificial intelligence platforms with a data annotation function, they usually also require manual data annotation, and the data annotation efficiency is relatively low.

[0026] To solve the technical problems existing in the related art, embodiments of the present disclosure provide a data annotation management method.

[0027] The embodiments of the present disclosure will be introduced in detail below.

[0028] As Figure 1As shown in the figure, an embodiment of the present disclosure provides a data annotation management method, which is applied to an artificial intelligence platform and includes the following steps:

[0029] Step 101, in response to a data annotation task request of a creating user, determine the first access account information of the annotation component.

[0030] In some embodiments, when a creating user initiates a data annotation task request on the artificial intelligence platform for the first time, the artificial intelligence platform creates an account information for the creating user to log in to the annotation component, that is, the first access account information, and stores the first access account information in a corresponding storage table or database of the artificial intelligence platform.

[0031] In some embodiments, when a creating user is not initiating a data annotation task request on the artificial intelligence platform for the first time (that is, the creating user already has the account information for logging in to the annotation component), the artificial intelligence platform can query the corresponding storage table or database through the backend service to determine the first access account information corresponding to the creating user.

[0032] In some embodiments, data can be transmitted between the artificial intelligence platform and the annotation component through an access interface, for example, through an Application Programming Interface (API). Therefore, in order for the artificial intelligence platform to successfully log in to the annotation component using the first access account information, the first access account information may include at least one of the following: the login name of the creating user, the login password of the creating user, and the information of the access interface between the artificial intelligence platform and the annotation component.

[0033] In some embodiments, the annotation component can be a labelU-Kit annotation component, a LabelStudio component, or other components that can be used for data annotation.

[0034] In some embodiments, for the convenience of creating and storing the first access account information, the login name of the creating user can be a rewrite of the user name used by the creating user to log in to the artificial intelligence platform (taking the LabelStudio component as an example for illustration. Since the LabelStudio component requires the login name to be in the email format, an email suffix, such as @label.com, can be added to the user name used to log in to the artificial intelligence platform to generate the login name of the creating user). The login password of the creating user can be an 8-bit character randomly generated, and the information of the access interface between the artificial intelligence platform and the annotation component can be a 32-bit character randomly generated, but it is not limited thereto. It should be understood that the specific form of the above first access account information is only an example and can also be other forms.

[0035] Step 102: Obtain the session identifier of the annotation component based on the first access account information.

[0036] In some embodiments, the artificial intelligence platform can utilize the obtained first access account information to access the first resource address of the annotation component through the backend microservices of the artificial intelligence platform. Thus, when the artificial intelligence platform successfully accesses this first resource address, it receives the cross-site request information sent by the annotation component; and then uses this cross-site request information and the first access account information to access the annotation component. Consequently, when successfully accessing the annotation component, it receives the session identifier sent by the annotation component through the backend microservices of the artificial intelligence platform.

[0037] In some embodiments, the session identifier is used to prevent sensitive data (such as the login name of the creating user, the login password of the creating user, etc.) in the first access account information from being intercepted when the artificial intelligence platform logs in to the annotation component using the first access account information.

[0038] In some embodiments, the first resource address can be the Uniform Resource Locator (URL) address of the annotation component.

[0039] In some embodiments, the session identifier can be sessionid.

[0040] Step 103: Log in to the annotation component based on the session identifier of the annotation component.

[0041] In some embodiments, the artificial intelligence platform can log in to the annotation component from the front-end service of the artificial intelligence platform based on the session identifier of the annotation component using the first domain name proxy module. The first domain name proxy module corresponds to the front-end service of the artificial intelligence platform (i.e., the first domain name proxy module can be integrated in the front-end microservices), and the annotation component contains a second domain name proxy module corresponding to the first domain name proxy module.

[0042] In other words, the artificial intelligence platform can, through the first domain name proxy module in the front-end microservices, transfer the session identifier and the first access account information to the second domain name proxy module in the annotation component to achieve the transfer of the creating user's identity information. Thus, the second domain name proxy module transmits the login name of the creating user and the login password of the creating user included in the first access account information to the login system of the annotation component, realizing passwordless login of the creating user to the annotation component.

[0043] In some embodiments, the first domain name proxy module and the second domain name proxy module can be nginx modules, but are not limited thereto. Any module that can achieve cross-domain transmission of sensitive information can be the first domain name proxy module or the second domain name proxy module.

[0044] In some embodiments, the artificial intelligence platform may execute a first command based on the session identifier of the annotation component to call the remote client component to log in to the annotation component from the backend service of the artificial intelligence platform. The remote client component corresponds to the backend service of the artificial intelligence platform, and the annotation component includes a remote server component corresponding to the remote client component.

[0045] In other words, the artificial intelligence platform may execute the first command in the remote client component, thereby calling the remote client component to transfer the session identifier from the port of the artificial intelligence platform to the port of the annotation component. When the second domain name proxy module in the annotation component is listening on the port of the annotation component and detects that the port has successfully received the session identifier, it may send the session identifier to the remote server component of the annotation component, thereby using the remote server component to transfer the session identifier to the login system of the annotation component, thus bypassing the front ends of the artificial intelligence platform and the annotation component and achieving passwordless login to the annotation component from the backend microservice.

[0046] In some embodiments, the remote client component may be a jsonrpc module integrated into the backend service of the artificial intelligence platform, and the remote measurement server component may be a jsonrpc module integrated into the annotation component, but is not limited thereto. Other modules that can implement session identifier transfer are also possible, and the present disclosure does not limit this.

[0047] In some embodiments, the first command may be a python statement, but is not limited thereto. Other programming statements that can call the remote client component are also possible. It should be understood that when using the first statement, both the artificial intelligence platform and the annotation component are configured with a language environment corresponding to the first statement.

[0048] Step 104, create a data annotation task using the annotation component based on the data annotation task request.

[0049] In some embodiments, when the artificial intelligence platform successfully logs in to the annotation component using the first access account information, the artificial intelligence platform may create a data annotation task in the annotation component according to the relevant information in the data annotation task request.

[0050] In some embodiments, the creating user may also create an annotation task in the annotation component by selecting an annotation directory in the artificial intelligence platform.

[0051] In some embodiments, the data annotation task request may include information such as a task name, data to be annotated, an annotation type (such as text, picture, video, etc.), but is not limited thereto. It may also be only used to create data annotation task matters in the annotation component without including information such as data to be annotated.

[0052] Specifically, the processor resources for creating a data annotation task can be determined based on creating a corresponding candidate resource group for the user in the artificial intelligence platform, where the candidate resource group belongs to the artificial intelligence platform; and then, based on the processor resources and the data path of the data to be annotated corresponding to the data annotation task request, a data annotation task is created in the annotation component.

[0053] In some embodiments, the candidate resource group can be the CPU resources and / or GPU resources corresponding to the artificial intelligence platform.

[0054] In some embodiments, the data annotation task (i.e., the annotation page) created in the annotation component can include information such as the task name, the data to be annotated, the annotation type, etc., but is not limited thereto.

[0055] In some embodiments, after successfully creating a data annotation task in the annotation component, through the program programming API interface between the annotation component and the artificial intelligence platform, information such as the resource configuration, data configuration, data annotation list, and annotation task details for creating the data annotation task as shown in Figure 2 is displayed in the preset display area of the artificial intelligence platform.

[0056] In some embodiments, the corresponding relationship between the artificial intelligence platform and the annotation component can be as shown in Figure 3 shown.

[0057] Specifically, the resource configuration can include the resource group, network type, and CPU or GPU used for creating the data annotation task of the artificial intelligence platform corresponding to this data annotation task; the data configuration can include the directory of the annotation data (i.e., the above-mentioned data to be annotated) in the data annotation task.

[0058] The data annotation list can include the task name, status, running duration, resource information, creation time, annotation page, and running node used by the artificial intelligence platform (i.e., the running node); the creating user can perform functions such as task name retrieval, node retrieval, and user retrieval in the data annotation list, and the creating user can also perform operations such as task deletion, task stop, and task start in the data annotation list.

[0059] When there is a data annotation task, the details of the annotation task can include specific task annotation items (such as item 1, item 2, etc.), the total number of data to be annotated, the number of already annotated data to be annotated, and a data synchronization list. The creating user can perform data synchronization operations between the artificial intelligence platform and the annotation component on the annotation task details interface, and can also export the results of the already annotated data (for example, export the data annotation results in JSON format, but not limited to this); when there is no data annotation task or the data annotation task has all ended, the annotation task details can provide options such as creating task items, creating annotation tools, local data synchronization, and jumping to the annotation component to facilitate the quick start of the creating user.

[0060] In some embodiments, when the annotation component successfully creates a data annotation task, the data structure stored in the data annotation task can be as follows in the table:

[0061]

[0062] Table 1

[0063] It should be understood that the above description is only an example and should not limit the present disclosure. The artificial intelligence platform can implement some or all of the above functions, and can also implement other functions other than the above description; the storage structure of the data annotation task can also be different from the content shown in Table 1, and the present disclosure does not limit this.

[0064] Step 105, based on the access interface of the annotation component, synchronize the data to be annotated in the artificial intelligence platform to the data annotation task.

[0065] In some embodiments, the access interface of the annotation component can be the API interface of the annotation component, but not limited to this.

[0066] In some embodiments, when the task annotation request does not include the data to be annotated, the artificial intelligence platform can synchronize the data to be annotated uploaded by the creating user to the artificial intelligence platform or on the local of the creating user to the data annotation task through the access interface to realize the data flow between the artificial intelligence platform and the annotation component.

[0067] Step 106, automatically annotate the data to be annotated in the data annotation task.

[0068] In some embodiments, the artificial intelligence platform can be communicatively connected to multiple computing power nodes. The intelligent scheduling system of the artificial intelligence platform (such as the Kubernetes system, etc.) can allocate computing power nodes for automatic annotation of users for this data annotation task according to information such as the amount of data to be annotated, the type of data to be annotated, and the user level of the creating user included in the data annotation task, and then annotate the data to be annotated by calling the inference model in the allocated computing power nodes, so as to realize the automatic annotation of the data to be annotated.

[0069] In some embodiments, as Figure 4 shown, the artificial intelligence platform can be connected not only to computing power nodes, but also to training nodes and training-computing power fusion nodes. The training nodes are used to train the inference model so that the inference model can implement the function of automatic annotation. The training-computing power fusion nodes are used to train the inference model and use the trained inference model to automatically annotate the data to be annotated in the data annotation task.

[0070] In summary, in the method of the present disclosure, by obtaining the session identifier of the annotation component, the artificial intelligence platform can log in to the above-mentioned annotation component using this session identifier without the user entering identity information, realizing password-free login of the user to the annotation component and improving the convenience of user operations. And through the artificial intelligence platform, automatic annotation of the data to be annotated is realized, improving the annotation efficiency of the data.

[0071] Figure 5 Further, a flowchart of a data annotation management method proposed by the present disclosure is shown. Based on Figure 1 the embodiments shown, step 102 is further explained. Figure 5 It may include the following steps.

[0072] Step 501, based on the first access account information, access the first resource address of the annotation component.

[0073] In some embodiments, the login name of the creating user and the login password of the creating user in the first access account information can be used to access the URL address (i.e., the first resource address) of the annotation component through the backend service of the artificial intelligence platform.

[0074] Step 502, when the first resource address is successfully accessed, receive the cross-site request information sent by the annotation component.

[0075] In some embodiments, when the artificial intelligence platform successfully accesses the first resource address, the annotation component will return cross-site request information to the artificial intelligence platform so that the artificial intelligence platform can send sensitive information cross-site to the annotation component. At this time, the artificial intelligence platform receives the cross-site request information sent by the annotation component.

[0076] In some embodiments, the cross-site request information is, for example, csrfmiddlewaretoken, but is not limited thereto.

[0077] Step 503, based on the first access account information and the cross-site request information, obtain the session identifier of the annotation component.

[0078] In some embodiments, the artificial intelligence platform may use the login name and login password of the creating user in the first access account, as well as the cross-site request information to log in and access the annotation component. When the artificial intelligence platform successfully logs in and accesses the annotation component, the annotation component will store the access credentials of the artificial intelligence platform (such as the first access account information and cross-site request information, etc.), and return a session identifier to the artificial intelligence platform through the browser, so that the artificial intelligence platform can transfer sensitive information between browsers, solve the browser cross-domain problem between the artificial intelligence platform and the annotation, and lay a foundation for passwordless login to the annotation component.

[0079] In some embodiments, the session identifier is, for example, the sessionid field, but is not limited thereto.

[0080] In summary, the data annotation management method proposed according to the present disclosure, which is applied to an artificial intelligence platform, includes: accessing the first resource address of the annotation component based on the first access account information; when successfully accessing the first resource address, receiving the cross-site request information sent by the annotation component; and obtaining the session identifier of the annotation component based on the first access account information and the cross-site request information. The method of the present disclosure lays a foundation for realizing passwordless login of the user to the annotation component by using the first access account information to determine the session identifier of the annotation component.

[0081] Figure 6 Further shows a flowchart of a data annotation management method proposed by the present disclosure. Based on Figure 1 The embodiment shown further explains step 103. Figure 6 It may include the following steps:

[0082] Step 601, use the first domain name proxy module to map the domain name of the artificial intelligence platform to the domain name address of the second domain name proxy module of the annotation component.

[0083] In some embodiments, the first domain name proxy module may be used to map the domain name (https) of the artificial intelligence platform to the domain name (https) address of the second domain name proxy module, so that the artificial intelligence platform can send the first access account information and the session identifier to the annotation component through the browser.

[0084] Step 602, based on the domain name address, send the first access account information and the session identifier to the annotation component to log in to the annotation component.

[0085] In some embodiments, to ensure the security of the first access account information, the Advanced Encryption Standard (AES) can be used to set the encryption key information for the first access account information. For example, the information can be set to annotationslabelstudio==, and the encryption and decryption of the first access account information are performed through the encrypted information. However, this is not limited thereto, and other encryption methods can also be used. The present disclosure does not limit this.

[0086] In some embodiments, according to the domain name address of the second domain name proxy module, the first access account information and the session identifier can be sent to the annotation component through the browser, so as to achieve passwordless login of the creating user to the annotation component.

[0087] Exemplarily, in an alternative embodiment, it can be as Figure 7 shown that the passwordless login of the artificial intelligence platform and the like to the annotation component from the front end includes the following steps:

[0088] 1. The artificial intelligence platform obtains the username and password for logging in to the annotation component Labelstudio. 2. The artificial intelligence platform queries the annotation component container through the backend service using the obtained username and password, and parses and obtains the cross-site request information of the login component. 3. Since the password may be encrypted, the AES algorithm can be used to decrypt the password, and the username, the password decrypted by AES, and the extended request information are used to access the annotation component, and the credentials for the user's login access are saved in the annotation component. 4. Through the session identifier returned by the annotation component, the artificial intelligence platform obtains the session identifier and the first resource address (i.e., the URL of the annotation component). 5. The front-end service of the artificial intelligence platform is used to access the first resource address using the session identifier. 6. The first domain name proxy module (nginx of the artificial intelligence platform) in the front-end service is used for domain name mapping to map the domain name of the artificial intelligence platform to the second domain name proxy module (nginx of the annotation component). 7. The session identifier, the username, and the password are packaged into a small text (cookie) by the first domain name proxy module and sent to the second domain name proxy module, thereby achieving passwordless login to the annotation component.

[0089] In other words, first, in the backend microservices of the artificial intelligence platform, obtain the user and password of the login access annotation component. Then, access the URL address of the annotation component in the backend microservices to obtain the cross-site request information csrfmiddlewaretoke. Use the login user and password, as well as the cross-site request information, to log in and access the annotation component. After successfully logging in to the annotation component, the microservices of the annotation component will store the access credentials of this user. At the same time, the session identifier sessionid field will be returned to the backend service of the artificial intelligence platform in the browser. If the session identifier is passed in the frontend microservices of the artificial intelligence platform, the browser cross-domain problem needs to be solved. Therefore, install the first domain name proxy module ngnix in the components of the frontend microservices for domain name proxy and information transfer. At the same time, the second domain name proxy module ngnix also needs to be installed in the microservices of the annotation component, and the corresponding keys (for example, the certifaicate parameter and the key parameter) are configured to support the secure https (i.e., the domain name of the above artificial intelligence platform) access of the artificial intelligence platform. At the same time, configure the https domain name proxy / map to the http domain name of the annotation component (i.e., the domain name of the above annotation component), so as to achieve password-free secure access.

[0090] In summary, the method of the present disclosure realizes the transfer of the first access account information and the session identifier at the front end of the artificial intelligence platform through the first domain name proxy module in the front-end service of the artificial intelligence platform and the second domain name proxy module in the annotation component. The annotation component can be logged in password-free without the user entering the first access account information in the annotation component, improving the user experience and data annotation efficiency.

[0091] Figure 8 Further, a flowchart of a data annotation management method proposed by the present disclosure is shown. Based on Figure 1 The embodiment shown further explains step 103. Figure 8 It may include the following steps:

[0092] Step 801, execute the first command to call the remote client component and send the session identifier to the second domain name proxy module to log in to the annotation component.

[0093] In some embodiments, the artificial intelligence platform may execute the first command in the remote client component, thereby calling the remote client component to transfer the session identifier from the port of the artificial intelligence platform to the port of the annotation component. When the second domain name proxy module in the annotation component is listening to the port of the annotation component and detects that the port has successfully received the session identifier, the session identifier can be sent to the remote server component of the annotation component, and then the remote server component is used to transfer the session identifier to the login system of the annotation component, thus bypassing the front ends of the artificial intelligence platform and the annotation component and realizing password-free login to the annotation component from the backend microservices.

[0094] Step 802: In response to creating a user's first task query request, obtain the network address of the annotation component at the time of login.

[0095] In some embodiments, when a user creates a first task query request in the artificial intelligence platform, the artificial intelligence platform can obtain the network address stored in the database at the time of logging in to the annotation component by calling the database or other means. The present disclosure does not limit the method of obtaining the network address at the time of logging in to the annotation component.

[0096] In some embodiments, the first task query request is used to request access to the details interface in the data annotation task.

[0097] In some embodiments, the network address at the time of logging in to the annotation component is the network address of the data annotation task.

[0098] Step 803: Based on the network address, execute a second command to call the remote client component to access the details interface of the data annotation task in the annotation component.

[0099] In some embodiments, the artificial intelligence platform can execute a second command to call the remote client component to access the network address at the time of logging in to the annotation component, thereby opening the details interface of the data annotation task in the annotation component.

[0100] Exemplarily, in an alternative embodiment, as Figure 9 shown, the jsonrpc component can be used to implement the operation of the artificial intelligence platform in the annotation component. The jsonrpc component located in the artificial intelligence platform is the remote client component, and the jsonrpc component located in the annotation component is the remote server component. The port corresponding to the remote client component is port 32107, and the port opened by the annotation component is 8081. Then when the artificial intelligence platform executes the first command (i.e., Figure 9 the command 8199 / jsonrpc shown), to call the remote client component to request port 8199 corresponding to port 8081, the second domain name proxy service module nginx of the annotation component can listen on port 8081, thereby sending the session identifier sent by the artificial intelligence platform request to the remote server component (i.e., the jsonrpc component in the annotation component).

[0101] When the artificial intelligence platform executes the second command (i.e., Figure 9 the command 8199 / project shown), to call the remote client component to request port 8199 corresponding to port 8081, at this time, it is not necessary for the second domain name proxy service module of the annotation component to listen on port 8081. The artificial intelligence platform can directly access the details interface of the data annotation task in the annotation component (i.e., Figure 9the Labelstudio interface shown).

[0102] In summary, the data annotation management method proposed according to the present disclosure, which is applied to an artificial intelligence platform, includes: executing a first command to call a remote client component and sending a session identifier to a second domain name proxy module to log in to an annotation component; in response to a first task query request of a created user, obtaining the network address of the annotation component at the time of login; based on the network address, executing a second command to call the remote client component to access the details interface of the data annotation task in the annotation component. The method of the present disclosure, by using the remote client component, enables the artificial intelligence platform to log in to the annotation component without password through the backend, and can realize the query of the annotation interface in the annotation component by the created user, improving the operation convenience of the user and the data annotation efficiency.

[0103] Figure 10 Further shows a flowchart of a data annotation management method proposed by the present disclosure. Based on Figure 1 the embodiments shown, step 105 is further explained,[ Figure 10 It may include the following steps:

[0104] Step 1001, based on a data annotation task request, determine a computing power node corresponding to the artificial intelligence platform for the data annotation task.

[0105] In some embodiments, the artificial intelligence platform may be communicatively connected to multiple computing power nodes. The intelligent scheduling system of the artificial intelligence platform may, according to information such as the amount of data to be annotated, the type of data to be annotated, and the user level of the created user included in the data annotation task, allocate a computing power node for automatic annotation of the data annotation task for the user. Then, by calling the inference model in the allocated computing power node, the data to be annotated is annotated, thereby realizing the automatic annotation of the data to be annotated.

[0106] Step 1002, configure a second resource address of the computing power node for the data annotation task, so as to automatically annotate the data to be annotated in the data annotation task by using the inference model of the computing power node, the model interface of the inference model, the task address of the data annotation task, and the access information of the data annotation task.

[0107] In some embodiments, the inference model can be configured for the data annotation task through the model interface of the inference model, the task address of the data annotation task, and the access information of the data annotation task, so that the inference model can be used to automatically annotate the data to be annotated.

[0108] In some embodiments, the access information is generated by the annotation component when creating a data annotation task, and the access information is used to allow other devices and / or programs outside the annotation component (such as an artificial intelligence platform, an inference model, etc.) to access the data annotation task.

[0109] In some embodiments, the second resource address may be the URL address of the computing power node.

[0110] In some embodiments, the URL address of the computing power node and the inference model in the computing power node may be configured into the data annotation task, so that after the connection between the computing power node and the annotation component is successful, the annotation component can call the inference model in the computing power node to automatically annotate the data to be annotated.

[0111] In some embodiments, the annotation result of the data to be annotated after automatic annotation can also be corrected manually by the creating user.

[0112] In some embodiments, the artificial intelligence platform can also train the inference model to improve the accuracy of automatic annotation.

[0113] Specifically, in response to the inference model training request of the creating user, the training node for training the inference model in the artificial intelligence platform can be determined; based on the training node, the inference model can be trained by using the preset training data set and / or the data that has been annotated in the data annotation task.

[0114] Among them, a suitable training node can be allocated for the inference model training request according to the user level of the creating user, the requirement for the accuracy of the inference model, etc.

[0115] In some embodiments, the training speeds of different training nodes and the CPU and / or GPU resources used for training can be the same or different.

[0116] In some embodiments, the CPU and / or GPU resources used by different computing power nodes for automatically annotating the data to be annotated can be the same or different.

[0117] In some alternative embodiments, the artificial intelligence platform also has a warning function. Specifically, the artificial intelligence platform can determine the estimated completion time of the data annotation task based on preset rules; when the execution time of the data annotation task exceeds the estimated completion time, a timeout prompt is displayed in the second preset area of the artificial intelligence platform, and / or the data annotation task is terminated, and the computing power node corresponding to the data annotation task is released.

[0118] In other words, in some alternative embodiments, when the execution time of the data annotation task exceeds the estimated completion time, a timeout prompt can be only displayed in the second preset area of the artificial intelligence platform.

[0119] In some alternative embodiments, when the execution time of the data annotation task exceeds the estimated completion time, the data annotation task can be only terminated, and the computing power node corresponding to the data annotation task is released.

[0120] In some alternative embodiments, when the execution time of a data annotation task exceeds the expected completion time, a timeout prompt may be displayed in a second preset area of the artificial intelligence platform, the data annotation task may be terminated, and the computing power node corresponding to the data annotation task may be released.

[0121] Exemplarily, the artificial intelligence platform may perform timeout management for data annotation tasks through an intelligent scheduling system, and may perform functions such as timeout notification and error notification for data annotation tasks. The artificial intelligence platform may set timeout times for different types of tasks (training tasks, inference tasks, data annotation tasks, etc.), and may also set timeout times for different types of tasks of resources. For example, it may be set that the timeout is executed by the inference node with a timeout time of 1 day and the notification times are 3 times. After the data annotation task runs for more than one day, message notifications will be made 3 times in the second preset area of the artificial intelligence platform. At the same time, the actions after timeout may be set, such as terminating the data annotation task and releasing the resources of the occupied resource group after timeout.

[0122] Step 1003: Obtain the annotation progress of the data to display the annotation progress in a first preset area of the artificial intelligence platform.

[0123] In some embodiments, the artificial intelligence platform may obtain the annotation progress of the data in the data annotation task through an API interface, but not limited thereto, in the annotation component, and display the annotation progress in a first preset area of the artificial intelligence platform, so as to facilitate the creator to view the completion status of the data annotation task in real time.

[0124] In some embodiments, when the data to be annotated in the data annotation task is completed with data annotation, the annotation component is used to export the annotation result of the data to be annotated in the data annotation task to the artificial intelligence platform in a preset file format.

[0125] In some embodiments, the preset file format is, for example, file formats such as json, csv, etc., and the present disclosure does not limit this.

[0126] In summary, the data annotation management method proposed according to the present disclosure is applied to an artificial intelligence platform, and includes: based on a data annotation task request, determining a computing power node corresponding to the artificial intelligence platform for the data annotation task; configuring a second resource address of the computing power node for the data annotation task, so as to automatically annotate the data to be annotated in the data annotation task by using the inference model of the computing power node, the model interface of the inference model, the task address of the data annotation task, and the access information of the data annotation task; obtaining the annotation progress of the data to display the annotation progress in a first preset area of the artificial intelligence platform. The method of the present disclosure realizes the automatic annotation of the data to be annotated in the data annotation task by calling the computing power node corresponding to the artificial intelligence platform, thereby improving the data annotation efficiency.

[0127] Figure 11 Further shown is a flowchart of a data annotation management method proposed by the present disclosure. Based on Figure 1 the embodiments shown, further explained Figure 11 it may include the following steps:

[0128] Step 1101, in response to a task sharing request from a creating user, determine the data annotation subtasks corresponding to the sharing user in the data annotation task, and generate second access account information for the annotation component corresponding to the sharing user.

[0129] In some embodiments, the task sharing request is used to allocate some of the data to be annotated in the data annotation task to the sharing user.

[0130] In some embodiments, the task sharing request may also indicate the data annotation subtasks assigned to the sharing user.

[0131] In some embodiments, the method for generating the second access account information is the same as that for generating the first access account information. For details, refer to the embodiments shown in step 101 and the descriptions in related embodiments, which will not be elaborated here.

[0132] Step 1102, in response to a second task query request from the sharing user, log in to the annotation component based on the second access account information to access the details interface of the data annotation subtask in the annotation component.

[0133] In some embodiments, when the artificial intelligence platform receives the second task query request from the sharing user, it can log in to the annotation component without password from the front-end service or back-end service of the artificial intelligence platform according to the second access account information (specifically refer to Figure 6 or Figure 8 the embodiments shown, which will not be elaborated here), to access the details interface of the data annotation subtask in the annotation component.

[0134] In some embodiments, the sharing user may only be able to access the details interface of the data annotation subtasks assigned to him / her, or may also access the details interface of the entire data annotation task, which will not be elaborated here.

[0135] Exemplarily, such as Figure 12As shown in the figure, after a user in the artificial intelligence platform creates a data annotation task, the user can share the data annotation task individually or in batches with other users in the artificial intelligence platform. After successful sharing, other users can view the annotation task. The user can also recycle the permissions of the data annotation task. After recycling, other users cannot view the data annotation task. When sharing a data annotation task, the user needs to set the account password for the shared user to access the annotation component and the internal interface information (i.e., the second access account information mentioned above). The API of the annotation component needs to be called to add users. After adding the accounts of the users, the shared users can use this account information for single sign-on to the annotation component interface of the shared annotation task. Collaborative data annotation requires the use of the system annotation function in the annotation component to allocate projects to the shared users, and at the same time set the annotation content in the annotation component for different users, so that the shared users can receive their own data annotation task work.

[0136] In summary, according to the data annotation management method proposed in the present disclosure, which is applied to an artificial intelligence platform, it includes: in response to a task sharing request of a creating user, determining a data annotation subtask corresponding to the shared user in the data annotation task, and generating second access account information of an annotation component corresponding to the shared user; in response to a second task query request of the shared user, logging in to the annotation component based on the second access account information to access the details interface of the data annotation subtask in the annotation component. The method of the present disclosure can, through the task sharing request of the creating user, allocate data annotation subtasks and second access account information to the shared users, realize password-free login of the shared users to the annotation component, and shared allocation of data annotation tasks, providing a more flexible annotation method for users and improving the convenience of user operations and data annotation efficiency.

[0137] To implement the data annotation management method provided in the embodiments of the present disclosure, the embodiments of the present disclosure also provide a data annotation management device, as Figure 13 shown, the data annotation management device 1300 includes:

[0138] A determination unit 1301, configured to determine first access account information of an annotation component in response to a data annotation task request of a creating user;

[0139] An acquisition unit 1302, configured to acquire a session identifier of the annotation component based on the first access account information;

[0140] A login unit 1303, configured to log in to the annotation component based on the session identifier of the annotation component;

[0141] A creation unit 1304, configured to create a data annotation task by using the annotation component based on the data annotation task request;

[0142] A synchronization unit 1305, configured to synchronize the data to be annotated in the artificial intelligence platform to a data annotation task based on an access interface of an annotation component;

[0143] An annotation unit 1306, configured to automatically annotate the data to be annotated in the data annotation task.

[0144] In some embodiments, the obtaining unit 1302 is further configured to access a first resource address of the annotation component based on first access account information; when the first resource address is successfully accessed, receive cross-site request information sent by the annotation component; and obtain a session identifier of the annotation component based on the first access account information and the cross-site request information.

[0145] In some embodiments, the login unit 1303 is further configured to log in to the annotation component from the front-end service of the artificial intelligence platform by using a first domain name proxy module based on the session identifier, where the first domain name proxy module corresponds to the front-end service of the artificial intelligence platform; and / or execute a first command based on the session identifier to call a remote server component to log in to the annotation component from the back-end service of the artificial intelligence platform, where the remote server component corresponds to the back-end service of the artificial intelligence platform.

[0146] In some embodiments, the login unit 1303 is further configured to map the domain name of the artificial intelligence platform to the domain name address of a second domain name proxy module of the annotation component by using the first domain name proxy module; and send the first access account information and the session identifier to the annotation component based on the domain name address to log in to the annotation component.

[0147] In some embodiments, the login unit 1303 is further configured to execute a first command to call a remote server component to send the session identifier to the second domain name proxy module to log in to the annotation component.

[0148] In some embodiments, the login unit 1303 is further configured to, in response to a first task query request for creating a user, obtain a network address when logging in to the annotation component; and execute a second command based on the network address to call a remote server component to access a details interface of a data annotation task in the annotation component.

[0149] In some embodiments, the creating unit 1304 is further configured to determine processor resources for creating a data annotation task based on a candidate resource group corresponding to the creating user, where the candidate resource group belongs to the artificial intelligence platform; and create a data annotation task in the annotation component based on the processor resources and a data path of the data to be annotated corresponding to the data annotation task request.

[0150] In some embodiments, the annotation unit 1306 is further configured to determine, based on a data annotation task request, a computing power node corresponding to the artificial intelligence platform for the data annotation task; configure a second resource address of the computing power node for the data annotation task, so as to automatically annotate the data to be annotated in the data annotation task by using the inference model of the computing power node, the model interface of the inference model, the task address of the data annotation task, and the access information of the data annotation task; and obtain the annotation progress of the data, so as to display the annotation progress in a first preset area of the artificial intelligence platform.

[0151] In some embodiments, the annotation unit 1306 is further configured to, in response to a request for training an inference model by a creating user, determine a training node in the artificial intelligence platform for training the inference model; and based on the training node, train the inference model by using a preset training data set and / or the data that has been completed and annotated in the data annotation task.

[0152] In some embodiments, the annotation unit 1306 is further configured to determine the estimated completion time of the data annotation task based on a preset rule; when the execution time of the data annotation task exceeds the estimated completion time, display a timeout prompt in a second preset area of the artificial intelligence platform, and / or terminate the data annotation task and release the computing power node corresponding to the data annotation task.

[0153] In some embodiments, the creating unit 1304 is further configured to, in response to a task sharing request by a creating user, determine a data annotation subtask corresponding to the sharing user in the data annotation task, and generate second access account information for an annotation component corresponding to the sharing user; and in response to a second task query request by the sharing user, log in to the annotation component based on the second access account information to access the details interface of the data annotation subtask in the annotation component. In some embodiments, the first access account information at least includes: the login name of the creating user, the login password of the creating user, and information about the access interface between the artificial intelligence platform and the annotation component.

[0154] In some embodiments, the annotation unit 1306 is further configured to, when the data to be annotated in the data annotation task is completed with data annotation, use the annotation component to export the annotation result of the data to be annotated in the data annotation task to the artificial intelligence platform in a preset file format.

[0155] It should be noted that when the data annotation management device provided in the above embodiments performs data annotation management, only the division of the above program modules is used as an example for illustration. In practical applications, the above processing can be allocated to different program modules according to needs, that is, the internal structure of the data annotation management device is divided into different program modules to complete all or part of the processing described above. In addition, the data annotation management device provided in the above embodiments and the data annotation management method embodiments provided in the present disclosure belong to the same concept. For the specific implementation process, please refer to the method embodiments, which will not be elaborated here.

[0156] Figure 14 FIG. is a schematic diagram of the hardware composition structure of the electronic device provided in the embodiments of the present disclosure. As Figure 14 shown, the electronic device 1400 includes at least one processor 1402; and a memory 1401 communicatively connected to the at least one processor 1402; wherein, the memory 1401 stores instructions executable by the at least one processor 1402, and the instructions are executed by the at least one processor 1402 to implement the steps of the data annotation management method described in the embodiments of the present disclosure.

[0157] Optionally, the electronic device may specifically be the data annotation management device of the embodiments of the present application, and the electronic device can implement the corresponding processes implemented by the data annotation management device in the various methods of the embodiments of the present application. For the sake of brevity, it will not be elaborated here.

[0158] It can be understood that the electronic device further includes a communication interface 1403. Each component in the electronic device is coupled together through a bus system 1404. It can be understood that the bus system 1404 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1404 further includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 14 all kinds of buses are labeled as the bus system 1404.

[0159] It can be understood that the memory 1401 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM, Read Only Memory), a programmable read-only memory (PROM, Programmable Read-Only Memory), an erasable programmable read-only memory (EPROM, Erasable Programmable Read-Only Memory), an electrically erasable programmable read-only memory (EEPROM, Electrically Erasable Programmable Read-Only Memory), a ferromagnetic random access memory (FRAM, ferromagnetic random access memory), a flash memory (Flash Memory), a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM, Compact Disc Read-Only Memory); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM, Random Access Memory), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as a static random access memory (SRAM, Static Random Access Memory), a synchronous static random access memory (SSRAM, Synchronous Static Random Access Memory), a dynamic random access memory (DRAM, Dynamic Random Access Memory), a synchronous dynamic random access memory (SDRAM, Synchronous Dynamic Random Access Memory), a double data rate synchronous dynamic random access memory (DDR SDRAM, Double Data Rate Synchronous Dynamic Random Access Memory), an enhanced synchronous dynamic random access memory (ESDRAM, Enhanced Synchronous Dynamic Random Access Memory), a sync link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), a direct rambus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 1401 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memories.

[0160] The methods disclosed in the above embodiments of the present disclosure can be applied to or implemented by the processor 1402. The processor 1402 has the ability to process signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit in hardware or the commands in the form of software in the processor 1402. The above-mentioned processor 1402 can be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 1402 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor can be a microprocessor or any conventional processor, etc. Combining the steps of the method disclosed in the embodiments of the present invention can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by the combination of the hardware and software modules in the decoding processor. The software module can be located in the storage medium, and this storage medium is located in the memory 1401. The processor 1402 reads the information in the memory 1401 and combines its hardware to complete the steps of the foregoing method.

[0161] In an exemplary embodiment, the electronic device can be implemented by one or more application-specific integrated circuits (ASICs, Application Specific Integrated Circuits), DSPs, programmable logic devices (PLDs, Programmable Logic Devices), complex programmable logic devices (CPLDs, Complex Programmable Logic Devices), FPGAs, general-purpose processors, controllers, MCUs, microprocessors (Microprocessors), or other electronic components, and is used to execute the foregoing method.

[0162] The embodiments of the present disclosure also provide a non-transitory computer-readable storage medium storing computer commands, and the computer commands are used to enable the computer to execute the steps of the data annotation management method described in the embodiments of the present disclosure when executed.

[0163] The embodiments of the present disclosure also provide a computer program product, including a computer program, and the computer program implements the steps of the data annotation management method described in the embodiments of the present disclosure when executed by a processor.

[0164] Optionally, the computer-readable storage medium can be applied to the data annotation management device in the embodiments of the present application, and the computer commands enable the computer to execute the corresponding processes implemented by the data annotation management device in each method of the embodiments of the present application. For the sake of brevity, it will not be elaborated here.

[0165] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the couplings, direct couplings, or communication connections between the displayed or discussed components can be through some interfaces. The indirect couplings or communication connections of devices or units can be electrical, mechanical, or other forms.

[0166] The units described above as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0167] In addition, each functional unit in the embodiments of the present invention can be all integrated in a processing unit, or each unit can be separately used as a unit, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of a combination of hardware and software functional units.

[0168] Those of ordinary skill in the art can understand that all or part of the steps to implement the above method embodiments can be completed by hardware related to program commands. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments. The foregoing storage medium includes various media that can store program codes, such as removable storage devices, ROM, RAM, magnetic disks, or optical discs.

[0169] Alternatively, if the above-mentioned integrated units of the present invention are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several commands to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The foregoing storage medium includes various media that can store program codes, such as removable storage devices, ROM, RAM, magnetic disks, or optical discs.

[0170] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claimed rights.

Claims

1. A data annotation management method, characterized in that: The method is applied to an artificial intelligence platform, and the method comprises: In response to creating a data annotation task request of a user, determining first access account information of the annotation component; Based on the first access account information, obtaining a session identifier of the annotation component; Based on the session identifier of the annotation component, logging into the annotation component; Based on the data annotation task request, create a data annotation task using the annotation component; Based on the access interface of the annotation component, the data to be annotated in the artificial intelligence platform is synchronized to the data annotation task; Based on the data labeling task request, determining a computing power node corresponding to the artificial intelligence platform for the data labeling task; Configuring a second resource address of the computing power node for the data labeling task, so as to automatically label the data to be labeled in the data labeling task by using the reasoning model of the computing power node, the model interface of the reasoning model, the task address of the data labeling task, and the access information of the data labeling task; Acquire the labeling progress of the data to be labeled, so as to display the labeling progress in a first preset area of ​​the artificial intelligence platform; Wherein, based on the session identifier of the annotation component, logging into the annotation component comprises: Based on the session identifier, using a first domain name proxy module, logging into the annotation component from a front-end service of the artificial intelligence platform, the first domain name proxy module corresponding to the front-end service of the artificial intelligence platform; and / or, Execute the first command, call the remote client component, send the session identifier to the second domain name proxy module to log in to the annotation component, and the remote client component corresponds to the backend service of the artificial intelligence platform; Wherein, based on the session identifier, using the first domain name proxy module, logging into the annotation component from the front-end service of the artificial intelligence platform includes: Using the first domain name proxy module, mapping the domain name of the artificial intelligence platform to the domain name address of the second domain name proxy module of the annotation component; Based on the domain name address, the first access account information and the session identifier are sent to the annotation component to log in to the annotation component.

2. The method according to claim 1, characterized in that The acquiring, based on the first access account information, a session identifier of the annotation component comprises: Accessing a first resource address of the annotation component based on the first access account information; When the first resource address is successfully accessed, receiving cross-site request information sent by the annotation component; Based on the first access account information and the cross-site request information, a session identifier of the annotation component is obtained.

3. The method according to claim 1, characterized in that The method further comprises: In response to the first task query request of the creating user, obtaining the network address of the annotation component at the time of login; Based on the network address, a second command is executed to call the remote client component to access the details interface of the data annotation task in the annotation component.

4. The method according to claim 1, characterized in that: The creating a data annotation task by using the annotation component based on the data annotation task request includes: Determining processor resources for creating the data annotation task based on a candidate resource group corresponding to the creating user, wherein the candidate resource group belongs to the artificial intelligence platform; Based on the processor resources and the data path of the to-be-annotated data corresponding to the data annotation task request, the data annotation task is created in the annotation component.

5. The method according to claim 1, characterized in that The method further comprises: In response to the inference model training request of the creating user, determining a training node in the artificial intelligence platform for training the inference model; Based on the training node, the inference model is trained using a preset training data set and / or the data that has been labeled in the data labeling task.

6. The method according to claim 1, characterized in that The method further comprises: Based on preset rules, determine the estimated completion time of the data labeling task; When the execution time of the data labeling task exceeds the expected completion time, a timeout prompt is displayed in the second preset area of ​​the artificial intelligence platform, and / or the data labeling task is terminated, and the computing power node corresponding to the data labeling task is released.

7. The method according to claim 1, characterized in that The method further comprises: In response to the task sharing request of the creating user, determining the data annotation subtask corresponding to the sharing user in the data annotation task, and generating second access account information of the annotation component corresponding to the sharing user; In response to the second task query request of the sharing user, based on the second access account information, the annotation component is logged in to access the detail interface of the data annotation subtask in the annotation component.

8. The method according to claim 1, characterized in that: The first access account information includes at least: the login name of the creating user, the login password of the creating user, and information about the access interface between the artificial intelligence platform and the annotation component.

9. The method according to claim 1, characterized in that: The method further comprises: When the data to be labeled in the data labeling task is labeled, the labeling component is used to export the labeling results of the data to be labeled in the data labeling task to the artificial intelligence platform in a preset file format.

10. An electronic device, characterized in that: include: a processor and a memory for storing a computer program capable of being executed on the processor, Wherein, when the processor is used to run the computer program, it executes the data labeling management method according to any one of claims 1-9.

11. A non-transitory computer-readable storage medium storing computer commands, characterized in that: The computer command is used to enable the computer to execute the data labeling management method according to any one of claims 1-9.

Images