Implementation method of multi-person cooperation data annotation based on block chain

Abstract

The invention discloses a method for realizing multi-person collaboration data annotation based on a block chain, and relates to the technical field of data annotation. In order to solve the problems that an existing data annotation system is lack of trust, data are easy to tamper, and the cooperation efficiency is low, the adopted scheme comprises the steps that a Fabric block chain network is built, and the network comprises a team manager organization for chaining and checking task-related information and a team member organization for checking and using the information on the chain; registering a block chain identity used for accessing a block chain network, wherein the block chain identity comprises a team manager and team members; deploying a multi-person labeling smart contract on the block chain network, wherein the smart contract provides a calling function related to the labeling task; calling a function provided by a multi-user labeling smart contract, firstly completing creation of labeling tasks, member maintenance and member permission maintenance, and then completing distribution, execution and verification of the labeling tasks in sequence; and carrying out uplink storage on all the generated operation data to ensure that the data is not tampered and traceable.

Description

Technical Field

[0001] This invention relates to the field of data annotation technology, specifically a method for implementing multi-person collaborative data annotation based on blockchain. Background Technology

[0002] Data annotation is a core step in building high-quality training datasets. By creating or adding labels to raw data, it clearly identifies the target information or content expected to be predicted by the model, providing crucial support for subsequent model training. However, existing data annotation systems generally suffer from problems such as lack of trust, opaque processes, susceptibility to data tampering, and low collaboration efficiency. These issues severely affect the credibility of labeled data and the overall quality of annotation work, hindering the development of related technologies based on labeled data.

[0003] Blockchain technology, as a distributed shared ledger technology with trustworthy characteristics, offers potential technical support for solving the aforementioned problems. This technology has the following core characteristics: First, distributed data storage: data is stored across multiple nodes, and consensus algorithms ensure data consistency among nodes, effectively avoiding risks caused by single node failures or malicious tampering. Second, immutable data: adopting an "append-only, not delete or modify" storage model, it ensures the integrity and traceability of on-chain historical records. Third, programmability: new blockchains can encode business rules into the blockchain through smart contracts. These rules (code) follow the same "not delete, not modify" mechanism as on-chain data, and can be automatically executed without skipping when called, with the execution results synchronously written to the blockchain. Based on these characteristics, blockchain technology can achieve trusted data sharing among nodes, providing a new technical approach to solving trust and security issues in the field of data labeling. Summary of the Invention

[0004] This invention addresses the problems of trust deficiency, lack of transparency, easy data tampering, and low collaboration efficiency in existing data annotation systems. It provides a blockchain-based method for multi-person collaborative data annotation, which, by introducing blockchain technology, enables full lifecycle management of annotation tasks, including automation and trustworthiness of team building, task allocation, task execution, and result verification, thereby improving the quality of annotated data and collaboration efficiency.

[0005] The present invention provides a method for implementing multi-user collaborative data annotation based on blockchain, and the technical solution adopted to solve the above-mentioned technical problems is as follows:

[0006] A method for implementing multi-user collaborative data annotation based on blockchain includes the following steps:

[0007] S1. Build a Fabric blockchain network, which includes team manager organizations and team member organizations. The team manager organization and its peer nodes are used by team manager users to upload and view team information, task information, task allocation information and task verification information on the chain. The team member organization and its peer nodes are used by team member users to view and use the information on the chain.

[0008] S2. Register blockchain identities for accessing the Fabric blockchain network, including team managers and team members;

[0009] S3. Deploy a multi-person annotation smart contract on the Fabric blockchain network. This smart contract provides functions for saving annotation team information, retrieving annotation team information, saving annotation task information, retrieving annotation task information, saving task allocation information, retrieving task allocation information, saving task verification information, and retrieving task verification information.

[0010] S4. Call the functions provided by the multi-person annotation smart contract to first complete the creation of annotation tasks, member maintenance and member permission maintenance, and then complete the allocation, execution and verification of annotation tasks in sequence.

[0011] S5. Store all operation data generated in step S4 on the blockchain to ensure the immutability and traceability of the data.

[0012] Optionally, the blockchain identity involved is implemented based on private keys and certificates to ensure the authenticity of the team manager's identity and the access control of the team members.

[0013] Optionally, the function for saving team information is only available for team managers to call and is used to store basic team information and team member permission information. When a team manager calls this function, they need to pass in two parameters: key and value. The key is the team code prefixed with "TEAM_", and the value is a JSON string.

[0014] The function to retrieve labeled team information is used to query team information. When calling this function, a key parameter must be passed in. The key is a JSON string with the prefix "TEAM_" followed by the team code, and it must be in the same format as the value parameter in the "save labeled team information" function.

[0015] Optionally, the function for saving labeled task information is only available for team managers to call and is used to store basic task information. When the team manager calls the function, they need to pass in two parameters: key and value. The key is the task code prefixed with "TASK_", and the value is a JSON-formatted string.

[0016] The function to retrieve annotation task information is used to query task information. When calling this function, a key parameter must be passed in. The key is a JSON string with the task code appended to the prefix "TASK_" and has the same format as the value parameter in the "save annotation task information" function.

[0017] Optionally, the function for saving task assignment information is only available for team managers to call and is used to store detailed task assignment information. When a team manager calls this function, they need to pass in two parameters: key and value. The key is the task code prefixed with "ALLOC_", and the value is a JSON-formatted string.

[0018] The function to retrieve task allocation information is used to query task allocation status. When calling this function, a key parameter must be passed in. The key is a JSON string with the "ALLOC_" prefix followed by the task code, and it must be in the same format as the value parameter in the "save task allocation information" function.

[0019] Optionally, the function for saving task verification information is only available for team managers to call and is used to store task verification results. When a team manager calls this function, they need to pass in two parameters: key and value. The key is the task code prefixed with "CHECK_", and the value is a JSON-formatted string.

[0020] The function to retrieve task verification information is used to query verification results. When calling this function, a key parameter must be passed in. The key is a JSON string with the "CHECK_" prefix followed by the task code and has the same format as the value parameter in the "Save Task Verification Information" function.

[0021] Optionally, step S4 is executed, which calls the function of the multi-person annotation smart contract to save the annotation team information, completes the creation of the annotation task, member maintenance, and member permission maintenance, specifically including the following operations:

[0022] S4.1.1 The team manager fills in the basic information of the labeling team, including team code, team name and team member information, and calls the function of saving labeling team information of the multi-person labeling smart contract to complete the creation of the labeling team;

[0023] S4.1.2 The team manager adds new members to an existing annotation team or removes members from an existing annotation team, and calls the function of saving annotation team information of the multi-person annotation smart contract to complete the maintenance of team members;

[0024] S4.1.3 The team manager adds or removes permissions for team members and calls the function of saving annotation team information in the multi-person annotation smart contract to complete the maintenance of member permissions.

[0025] Optionally, step S4 is executed, which calls the functions for saving annotation task information and saving task allocation information of the multi-user annotation smart contract to complete the allocation of annotation tasks. Specifically, this includes the following operations:

[0026] S4.2.1 The team manager first fills in the basic information of the annotation task, including: task code, task name, task creator, task type, dataset to be annotated and list of labels to be annotated, and then calls the function of the multi-person annotation smart contract to save the annotation task information to complete the creation of the annotation task;

[0027] S4.2.2 The team manager selects a list of members from the team to perform the annotation task, calculates the number of annotations assigned to each member based on the total number of annotations and the number of members selected, and further assigns a specific data sequence number in the dataset to be annotated to each member based on the assigned number of annotations.

[0028] S4.2.3 The team manager calls the function of saving task allocation information of the multi-person annotation smart contract to complete the allocation of annotation tasks and notify the relevant members.

[0029] Further, optionally, step S4 is executed, which calls the function for obtaining task allocation information of the multi-person annotation smart contract to complete the execution of the annotation task, specifically including the following operations:

[0030] S4.3.1 After receiving the assigned annotation task, team members call the task assignment information retrieval function of the multi-person annotation smart contract to query the specific assignment information of the annotation task.

[0031] S4.3.2 Team members begin executing their assigned annotation tasks based on the specific assignment information.

[0032] S4.3.3 After a team member completes their assigned annotation task, they should notify the creator of that annotation task, i.e., the team manager who created the annotation task.

[0033] Optionally, step S4 is executed, which calls the function for obtaining task allocation information and the function for saving task verification information of the multi-person annotation smart contract to complete the verification of the annotation task. Specifically, this includes the following operations:

[0034] S4.4.1 After receiving a notification that a team member has completed the annotation task, the team manager calls the function to obtain task allocation information for multi-person annotation smart contracts and views the task allocation information of the team member.

[0035] S4.4.2 Based on the task allocation information of the team members, the team manager further reviews the corresponding annotation results and verifies and scores them;

[0036] S4.4.3 The team manager calls the function to save the verification information of the smart contract labeled by multiple people to maintain the aforementioned verification results.

[0037] The present invention provides a blockchain-based multi-user collaborative data annotation method, which has the following advantages compared with existing technologies:

[0038] 1. This invention introduces blockchain technology, builds a Fabric blockchain network, and deploys multi-person annotation smart contracts to achieve trusted on-chain storage and management of information such as annotation teams, annotation tasks, task allocation, and task verification, thereby improving the quality of annotation data and collaboration efficiency. Through smart contract functions, it controls the permissions and data operations of team managers and members. Utilizing the immutability, distributed consensus, and automatic execution characteristics of blockchain and smart contracts, it ensures the transparency, traceability, and data security of the annotation process, effectively improving the efficiency and trust of multi-person collaborative data annotation. It aims to solve the problems of lack of trust, opaque processes, susceptibility to data tampering, and low collaboration efficiency existing in data annotation systems.

[0039] 2. This invention utilizes the distributed ledger characteristics of blockchain, storing all annotation-related operations on the chain, ensuring immutability and data authenticity and integrity. All operation records are publicly verifiable, and the entire process of task allocation, execution, and verification is traceable, enhancing trust in collaboration. Smart contracts automatically execute task allocation, access control, and verification processes, reducing human intervention and improving efficiency. It supports various permission settings (such as ADMIN, TASK_CLAIM, TASK_EXEC) for flexible team management and task control. Based on the Fabric multi-organizational architecture, it supports trusted collaboration between different organizations, making it suitable for distributed team scenarios. Through verification mechanisms and scoring feedback, it promotes continuous improvement in annotation quality, making it suitable for constructing high-quality machine learning training data. Attached Figure Description

[0040] Appendix Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0041] To make the technical solution, the technical problem solved, and the technical effect of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments.

[0042] Example 1:

[0043] Reference Appendix Figure 1 This embodiment proposes a method for implementing multi-person collaborative data annotation based on blockchain, which includes the following steps:

[0044] S1. Build a Fabric blockchain network, which includes team manager organizations and team member organizations. The team manager organization and its peer nodes are used by team manager users to upload and view team information, task information, task allocation information and task verification information on the chain. The team member organization and its peer nodes are used by team member users to view and use the information on the chain.

[0045] S2. Register blockchain identities for accessing the Fabric blockchain network, including team managers and team members.

[0046] Blockchain identity is based on private keys and certificates, ensuring the authenticity of team administrators' identities and access control for team members.

[0047] S3. Deploy a multi-person annotation smart contract on the Fabric blockchain network. This smart contract provides functions for saving annotation team information, retrieving annotation team information, saving annotation task information, retrieving annotation task information, saving task allocation information, retrieving task allocation information, saving task verification information, and retrieving task verification information.

[0048] (A) The function for saving team information is only available for use by team managers to store basic team information and team member permission information. When a team manager calls this function, they need to pass in two parameters: key and value. The key is a string of team codes prefixed with "TEAM_" (the team code is a randomly generated unique number), and the value is a JSON string. The specific format is as follows:

[0049]

[0050]

[0051] The field information in the above JSON is described as follows:

[0052] team_id represents the code of a multi-person collaborative data labeling team. This code is a unique code randomly generated when the team is created.

[0053] team_name represents the name of the team;

[0054] leader_id represents the user code of the team creator / manager;

[0055] members represents all team members;

[0056] user_id represents the user code of the team member;

[0057] user_name represents the username of the team member;

[0058] Permissions represent the permissions that team members have. Currently, three types of permissions are defined: ADMIN (team administrator) permission, TASK_CLAIM (task claiming) permission, and TASK_EXEC (task execution) permission.

[0059] The function for retrieving labeled team information is used to query team information. When a team manager / team member calls this function, they need to pass in the key parameter, which is a JSON string with the "TEAM_" prefix followed by the team code, and has the same format as the value parameter in the "save labeled team information" function.

[0060] (B) The function for saving labeled task information is only for team managers to call and is used to store basic task information. When calling this function, team managers need to pass two parameters: key and value. The key is a string with the "TASK_" prefix followed by the task code (the task code is a randomly generated unique number), and the value is a JSON-formatted string. The specific format is as follows:

[0061]

[0062] The field information in the above JSON is described as follows:

[0063] task_id represents the code of the data labeling task, which is a unique code randomly generated when the task is created;

[0064] task_name represents the name of the labeled task;

[0065] task_creator represents the user code of the task creator (i.e., the team manager);

[0066] task_type represents the type of annotation task, where 01 is text type, 02 is image type, 03 is video type, and 04 is audio type;

[0067] dataset_id represents the dataset's encoding;

[0068] `tags` represents the list of tags used in this data annotation task.

[0069] The function to retrieve annotation task information is used to query task information. When team managers / team members call this function, they need to pass in the key parameter, which is a JSON string with the "TASK_" prefix followed by the task code, and has the same format as the value parameter in the "Save Annotation Task Information" function.

[0070] (C) The function for saving task assignment information is only for team managers to call and is used to store detailed task assignment information. When calling this function, team managers need to pass two parameters: key and value. The key is the task code prefixed with "ALLOC_" (the task code is a randomly generated unique number), and the value is a JSON-formatted string. The specific format is as follows:

[0071]

[0072] The field information in the above JSON is described as follows:

[0073] user_id represents the code of the user (usually a team member) who was assigned the task;

[0074] task_id represents the task code for labeling;

[0075] start_index represents the starting index of the labeled data in the dataset for the assigned labeling task;

[0076] data_size represents the total amount of labeled data assigned to the labeling task.

[0077] The function to retrieve task assignment information is used to query task assignment status. When team managers / team members call this function, they need to pass in the key parameter, which is a JSON string with the prefix "ALLOC_" followed by the task code, and has the same format as the value parameter in the "save task assignment information" function.

[0078] (D) The function for saving task verification information is only for team managers to call and is used to store task verification results. When calling this function, team managers need to pass two parameters: key and value. The key is a string of the task code prefixed with "CHECK_" (the task code is a randomly generated unique number), and the value is a string in JSON format. The specific format is as follows:

[0079]

[0080] The field information in the above JSON is described as follows:

[0081] check_result represents the results of all task verifications;

[0082] user_id represents the user code of the task executor (usually a team member);

[0083] annotation_size represents the total number of annotations;

[0084] The score represents the verification score, with a maximum score of 100.

[0085] remark represents the remark information; check_time represents the verification time.

[0086] The checker represents the user code of the verifier (usually a team manager).

[0087] The function to retrieve task verification information is used to query verification results. When team managers / team members call this function, they need to pass in the key parameter, which is a JSON string with the "CHECK_" prefix followed by the task code, and has the same format as the value parameter in the "Save Task Verification Information" function.

[0088] S4. Call the functions provided by the multi-person annotation smart contract to first complete the creation of annotation tasks, member maintenance and member permission maintenance, and then complete the allocation, execution and verification of annotation tasks in sequence.

[0089] S4.1 The team manager calls the function of saving annotation team information in the multi-person annotation smart contract to complete the creation of annotation tasks, member maintenance, and member permission maintenance, specifically including the following operations:

[0090] S4.1.1 The team manager fills in the basic information of the labeling team, including the team code (automatically and randomly generated), team name and team member information, and calls the function of saving labeling team information of the multi-person labeling smart contract to complete the creation of the labeling team;

[0091] S4.1.2 The team manager adds new members to an existing annotation team or removes members from an existing annotation team, and calls the function of the multi-person annotation smart contract to save annotation team information to complete the maintenance of team members;

[0092] S4.1.3 The team manager adds or removes permissions for team members and calls the function of saving the annotation team information of the multi-person annotation smart contract to complete the permission maintenance of team members.

[0093] S4.2 The team manager calls the functions for saving annotation task information and saving task allocation information of the multi-person annotation smart contract to complete the allocation of annotation tasks, specifically including the following operations:

[0094] S4.2.1 The team manager first fills in the basic information of the annotation task, including: task code (automatically and randomly generated), task name, task creator (i.e., team manager), task type, dataset to be annotated and list of labels to be annotated, and then calls the function of saving annotation task information of the multi-person annotation smart contract to complete the creation of the annotation task.

[0095] S4.2.2 The team manager selects a list of members from the team to perform the annotation task, calculates the number of annotations assigned to each member based on the total number of annotations and the number of members selected, and further assigns each member a specific data sequence number (starting sequence number plus quantity) in the dataset to be annotated based on the assigned number of annotations.

[0096] S4.2.3 The team manager calls the function of saving task allocation information of the multi-person annotation smart contract to complete the allocation of annotation tasks and notify the relevant members (i.e. the team members who are assigned annotation tasks).

[0097] S4.3 Call the function to obtain task allocation information of the multi-person annotation smart contract to complete the execution of the annotation task, specifically including the following operations:

[0098] S4.3.1 After receiving the assigned annotation task, team members call the task assignment information retrieval function of the multi-person annotation smart contract to query the specific assignment information of the annotation task.

[0099] S4.3.2 Team members begin executing their assigned annotation tasks based on the specific assignment information.

[0100] S4.3.3 After a team member completes their assigned annotation task, they should notify the creator of that annotation task, i.e., the team manager who created the annotation task.

[0101] S4.4 Call the functions for obtaining task allocation information and saving task verification information of the multi-person annotation smart contract to complete the verification of the annotation task, specifically including the following operations:

[0102] S4.4.1 After receiving a notification that a team member has completed the annotation task, the team manager calls the function to obtain task allocation information for multi-person annotation smart contracts and views the task allocation information of the team member.

[0103] S4.4.2 Based on the task allocation information of the team members, the team manager further reviews the corresponding annotation results and verifies and scores them;

[0104] S4.4.3 The team manager calls the function to save the verification information of the smart contract labeled by multiple people to maintain the aforementioned verification results.

[0105] S5. Store all operation data generated in step S4 on the blockchain to ensure the immutability and traceability of the data.

[0106] In summary, the blockchain-based multi-person collaborative data annotation method of this invention, by introducing blockchain technology, realizes full lifecycle management of annotation tasks, including the automation and trustworthiness of team building, annotation task allocation, annotation task execution and result verification, thereby improving the quality of annotated data and collaboration efficiency. It solves the problems of lack of trust, opaque process, easy data tampering and low collaboration efficiency in existing data annotation systems.

[0107] The above specific examples illustrate the principles and implementation methods of the present invention in detail. These embodiments are merely for the purpose of helping to understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made to the present invention by those skilled in the art without departing from the principles of the present invention should fall within the patent protection scope of the present invention.

Claims

1. A method for implementing multi-user collaborative data annotation based on blockchain, characterized in that, Includes the following steps: S1. Build a Fabric blockchain network, which includes team manager organizations and team member organizations. The team manager organization and its peer nodes are used by team manager users to upload and view team information, task information, task allocation information and task verification information on the chain. The team member organization and its peer nodes are used by team member users to view and use the information on the chain. S2. Register blockchain identities for accessing the Fabric blockchain network, including team managers and team members; S3. Deploy a multi-person annotation smart contract on the Fabric blockchain network. This smart contract provides functions for saving annotation team information, retrieving annotation team information, saving annotation task information, retrieving annotation task information, saving task allocation information, retrieving task allocation information, saving task verification information, and retrieving task verification information. S4. Call the functions provided by the multi-person annotation smart contract to first complete the creation of annotation tasks, member maintenance and member permission maintenance, and then complete the allocation, execution and verification of annotation tasks in sequence. S5. Store all operation data generated in step S4 on the blockchain to ensure the immutability and traceability of the data.

2. The method for implementing multi-user collaborative data annotation based on blockchain according to claim 1, characterized in that, The blockchain identity is implemented based on private keys and certificates, ensuring the authenticity of the team manager's identity and the access control of team members.

3. The method for implementing multi-user collaborative data annotation based on blockchain according to claim 1, characterized in that, The function for saving labeled team information is only available for call by team managers and is used to store basic team information and team member permission information. When calling this function, team managers need to pass two parameters: key and value. The key is a team code prefixed with "TEAM_", and the value is a JSON-formatted string. The function to retrieve labeled team information is used to query team information. When calling this function, a key parameter must be passed in. The key is a JSON string with the "TEAM_" prefix followed by the team code and has the same format as the value parameter in the "save labeled team information" function.

4. The method for implementing blockchain-based multi-user collaborative data annotation according to claim 3, characterized in that, The function for saving labeled task information is only available for team managers to call and is used to store basic task information. When a team manager calls this function, they need to pass in two parameters: key and value. The key is the task code prefixed with "TASK_", and the value is a JSON-formatted string. The function to retrieve annotation task information is used to query task information. When calling this function, a key parameter must be passed in. The key is a JSON string with the "TASK_" prefix followed by the task code and has the same format as the value parameter in the "save annotation task information function".

5. The method for implementing blockchain-based multi-user collaborative data annotation according to claim 4, characterized in that, The function for saving task assignment information is only available for team managers to call and is used to store detailed task assignment information. When a team manager calls this function, they need to pass in two parameters: key and value. The key is the task code prefixed with "ALLOC_", and the value is a JSON-formatted string. The function for obtaining task allocation information is used to query task allocation status. When calling this function, a key parameter needs to be passed in. The key is a JSON string with the "ALLOC_" prefix followed by the task code and has the same format as the value parameter in the "save task allocation information" function.

6. The method for implementing multi-user collaborative data annotation based on blockchain according to claim 5, characterized in that, The function for saving task verification information is only available for team managers to call and is used to store task verification results. When a team manager calls this function, they need to pass in two parameters: key and value. The key is the task code prefixed with "CHECK_", and the value is a JSON-formatted string. The function for obtaining task verification information is used to query the verification results. When calling this function, a key parameter needs to be passed in. The key is a JSON string with the "CHECK_" prefix followed by the task code and has the same format as the value parameter in the "save task verification information" function.

7. The method for implementing multi-user collaborative data annotation based on blockchain according to claim 6, characterized in that, Execute step S4, call the function of the multi-person annotation smart contract to save annotation team information, complete the creation of annotation tasks, member maintenance, and member permission maintenance, specifically including the following operations: S4.1.1 The team manager fills in the basic information of the labeling team, including team code, team name and team member information, and calls the function of saving labeling team information of the multi-person labeling smart contract to complete the creation of the labeling team; S4.1.2 The team manager adds new members to an existing annotation team or removes members from an existing annotation team, and calls the function of the multi-person annotation smart contract to save annotation team information to complete the maintenance of team members; S4.1.3 The team manager adds or removes permissions for team members and calls the function of saving annotation team information in the multi-person annotation smart contract to complete the maintenance of member permissions.

8. The method for implementing multi-user collaborative data annotation based on blockchain according to claim 7, characterized in that, Execute step S4, call the functions for saving annotation task information and saving task allocation information of the multi-user annotation smart contract to complete the allocation of annotation tasks, specifically including the following operations: S4.2.1 The team manager first fills in the basic information of the annotation task, including: task code, task name, task creator, task type, dataset to be annotated and list of labels to be annotated, and then calls the function of the multi-person annotation smart contract to save the annotation task information to complete the creation of the annotation task; S4.2.2 The team manager selects a list of members from the team to perform the annotation task, calculates the number of annotations assigned to each member based on the total number of annotations and the number of members selected, and further assigns a specific data sequence number in the dataset to be annotated to each member based on the assigned number of annotations. S4.2.3 The team manager calls the function of saving task allocation information of the multi-person annotation smart contract to complete the allocation of annotation tasks and notify the relevant members.

9. The method for implementing multi-user collaborative data annotation based on blockchain according to claim 8, characterized in that, Step S4 involves calling the function to obtain task allocation information for the multi-user annotation smart contract, thus completing the annotation task execution. This includes the following operations: S4.3.1 After receiving the assigned annotation task, team members call the task assignment information retrieval function of the multi-person annotation smart contract to query the specific assignment information of the annotation task. S4.3.2 Team members begin executing their assigned annotation tasks based on the specific assignment information. S4.3.3 After a team member completes their assigned annotation task, they should notify the creator of that annotation task, i.e., the team manager who created the annotation task.

10. The method for implementing multi-user collaborative data annotation based on blockchain according to claim 9, characterized in that, Execute step S4, call the function to obtain task allocation information and the function to save task verification information for the multi-person annotation smart contract, and complete the verification of the annotation task. Specifically, this includes the following operations: S4.4.1 After receiving a notification that a team member has completed the annotation task, the team manager calls the function to obtain task allocation information for multi-person annotation smart contracts and views the task allocation information of the team member. S4.4.2 Based on the task allocation information of the team members, the team manager further reviews the corresponding annotation results and verifies and scores them; S4.4.3 The team manager calls the function to save the verification information of the smart contract labeled by multiple people to maintain the aforementioned verification results.

Images