Data labeling method and system based on industry lexicon and label library

Abstract

The invention provides a data annotation method and system based on an industry word library and a tag library in the technical field of data annotation, and the method comprises the steps: S1, obtaining a large amount of historical annotation data, and grouping the historical annotation data based on the industry; step S2, creating an industry word library and a tag library based on each piece of historical annotation data; s3, identifying the to-be-labeled file to obtain to-be-labeled data, and identifying the industry and project numbers of the to-be-labeled data; s4, through an industry word bank corresponding to industry matching, the natural language word segmentation tool performs word segmentation operation on the text data based on industry word segmentation of the industry word bank to obtain a word segmentation result; matching a corresponding label library through the item number, and pre-labeling the word segmentation result based on an item label of the label library to obtain a pre-labeling result; and S5, after the pre-labeling result is audited and corrected, a labeling result is generated. The method has the advantages that the data annotation quality and efficiency are greatly improved, and the annotation cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of data annotation, and in particular to a data annotation method and system based on an industry vocabulary library and a tag library. Background Art

[0002] With the widespread application of artificial intelligence technology, the demand for high-quality labeled data has increased dramatically. As a key link in converting raw data into machine-readable information, data labeling has become increasingly important. Traditionally, data labeling has been done manually one by one, which has the following disadvantages: 1. It is easily affected by the subjective judgment of the labelers, resulting in inconsistent labeling results from different people, making it difficult to ensure the quality of labeling. 2. It is time-consuming and labor-intensive, and the labeling efficiency is low. 3. Different industries have unique characteristics and professional terminology, and traditional methods often find it difficult to accurately capture these characteristics, which limits the depth and breadth of data labeling, especially in areas with more professional knowledge, such as medicine and law. 4. The cost of manual labeling remains high.

[0003] Therefore, how to provide a data annotation method and system based on industry vocabulary and tag library to improve the quality and efficiency of data annotation and reduce the annotation cost has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a data annotation method and system based on an industry vocabulary library and a tag library, so as to improve the quality and efficiency of data annotation and reduce the annotation cost.

[0005] In a first aspect, the present invention provides a data annotation method based on an industry vocabulary library and a tag library, comprising the following steps: Step S1, obtaining a large amount of historical annotated data covering different industries, and grouping each of the historical annotated data based on the industry; Step S2: extract the corresponding industry segmentation words from the historical annotation data of each group, and store the industry segmentation words of each group in a pre-created industry word library; extract project labels from each of the historical annotation data, and store each of the project labels, the project numbers corresponding to the project labels, and the association relationship with the industry segmentation words in a pre-created label library; Step S3: obtaining the file to be annotated through the workflow engine, identifying the file to be annotated to obtain the data to be annotated, and identifying the industry and project number of the data to be annotated; Step S4: Match the industry to the corresponding industry vocabulary, and the natural language word segmentation tool performs a word segmentation operation on the text data based on the industry word segmentation of the industry vocabulary to obtain a word segmentation result; match the project number to the corresponding tag library, and pre-label the word segmentation result based on the project tag of the tag library to obtain a pre-labeled result; Step S5: after reviewing and correcting the pre-annotation results, generate an annotation result, display and export the annotation result.

[0006] Furthermore, the step S2 further includes: Based on the latest annotation data or annotation requirements, the industry vocabulary and label library are updated and maintained in real time.

[0007] Furthermore, the step S3 is specifically as follows: The files to be labeled are obtained through the workflow engine, corresponding storage space and computing resources are allocated to each file to be labeled, each file to be labeled is stored in the corresponding storage space, each file to be labeled is pre-processed for image enhancement, each file to be labeled is identified through OCR technology to obtain the data to be labeled, the industry of the data to be labeled is automatically identified through a pre-trained recurrent neural network, and the project number is automatically identified from the data to be labeled according to the numbering rules.

[0008] Furthermore, the step S4 is specifically as follows: The industry is automatically matched to the corresponding industry vocabulary, and the natural language word segmentation tool hanLP automatically performs word segmentation operations on the text data based on the industry word segmentation of the industry vocabulary to obtain the word segmentation results; the project number is matched to the corresponding tag library, and the word segmentation results are automatically pre-annotated based on the association relationship and the number of annotations of the project tags in the tag library to obtain the pre-annotated results.

[0009] Furthermore, the step S5 is specifically as follows: After the pre-annotation results are automatically reviewed and corrected based on preset annotation rules, annotation results are generated, displayed and exported, stored and backed up, and the annotation progress is tracked through the workflow engine during the annotation process.

[0010] In a second aspect, the present invention provides a data annotation system based on an industry vocabulary and a tag library, comprising the following modules: A historical annotation data acquisition module is used to acquire a large amount of historical annotation data covering different industries, and group the historical annotation data based on the industry; An industry word library and label library creation module is used to extract the corresponding industry segmentation words from the historical annotation data of each group, and store the industry segmentation words of each group in a pre-created industry word library; extract project labels from each of the historical annotation data, and store each of the project labels, the project numbers corresponding to the project labels, and the association relationship with the industry segmentation words in a pre-created label library; A module for identifying files to be annotated, used to obtain files to be annotated through a workflow engine, identify the files to be annotated to obtain data to be annotated, and identify the industry and project number of the data to be annotated; A pre-marking module is used to match the industry with the corresponding industry vocabulary library, and the natural language word segmentation tool performs word segmentation operation on the text data based on the industry word segmentation of the industry vocabulary library to obtain a word segmentation result; match the corresponding tag library with the project number, and pre-mark the word segmentation result based on the project tag of the tag library to obtain a pre-marking result; The annotation review module is used to review and correct the pre-annotation results, generate annotation results, and display and export the annotation results.

[0011] Furthermore, the industry vocabulary and label library creation module is also used for: Based on the latest annotation data or annotation requirements, the industry vocabulary and label library are updated and maintained in real time.

[0012] Furthermore, the to-be-annotated file identification module is specifically used for: The files to be labeled are obtained through the workflow engine, corresponding storage space and computing resources are allocated to each file to be labeled, each file to be labeled is stored in the corresponding storage space, each file to be labeled is pre-processed for image enhancement, each file to be labeled is identified through OCR technology to obtain the data to be labeled, the industry of the data to be labeled is automatically identified through a pre-trained recurrent neural network, and the project number is automatically identified from the data to be labeled according to the numbering rules.

[0013] Furthermore, the pre-marking module is specifically used for: The industry is automatically matched to the corresponding industry vocabulary, and the natural language word segmentation tool hanLP automatically performs word segmentation operations on the text data based on the industry word segmentation of the industry vocabulary to obtain the word segmentation results; the project number is matched to the corresponding tag library, and the word segmentation results are automatically pre-annotated based on the association relationship and the number of annotations of the project tags in the tag library to obtain the pre-annotated results.

[0014] Furthermore, the annotation review module is specifically used to: After the pre-annotation results are automatically reviewed and corrected based on preset annotation rules, annotation results are generated, displayed and exported, stored and backed up, and the annotation progress is tracked through the workflow engine during the annotation process.

[0015] The advantages of the present invention are: 1. Obtain a large amount of historical annotated data containing different industries, group each historical annotated data based on the industry, extract the corresponding industry segmentation words from the historical annotated data of each group, store the industry segmentation words of each group in a pre-created industry vocabulary, extract project labels from each historical annotated data, store each project label, the project number corresponding to the project label, and the association relationship with the industry segmentation words in a pre-created label library; then obtain the file to be annotated through the workflow engine, identify the file to be annotated to obtain the data to be annotated, and identify the industry and project number of the data to be annotated; then match the industry to the corresponding industry vocabulary, and the natural language segmentation tool will use the industry segmentation words in the industry vocabulary to match the text data. The word segmentation operation is performed according to the data to obtain the word segmentation result; the corresponding tag library is matched by the project number, and the word segmentation result is pre-annotated based on the project tag of the tag library to obtain the pre-annotation result; finally, the pre-annotation result is reviewed and corrected to generate the annotation result, and the annotation result is displayed and exported; that is, the data to be annotated obtained by identifying the annotated file is automatically annotated through the created industry thesaurus and tag library, overcoming the subjective judgment influence of manual annotation, low efficiency, and high annotation cost problems, and the word segmentation operation is performed based on the industry word segmentation of the industry thesaurus, that is, targeted word segmentation is performed based on each industry, making the word segmentation more professional and accurate, ultimately greatly improving the quality and efficiency of data annotation, and greatly reducing the annotation cost.

[0016] 2. Real-time update and maintenance of various industry lexicons and label libraries based on the latest annotation data or annotation requirements enable industry lexicons and label libraries to keep pace with the times, and project labels in the label library can be customized according to user annotation requirements, thereby greatly improving the quality and flexibility of data annotation.

[0017] 3. By allocating corresponding storage space and computing resources to each file to be labeled, each file to be labeled can be labeled simultaneously, greatly improving the efficiency of data labeling.

[0018] 4. After preprocessing the image enhancement of each file to be labeled, the OCR technology is used to identify each file to be labeled to obtain the data to be labeled. That is, OCR recognition is performed based on high-quality files to be labeled, which effectively improves the accuracy of identifying the data to be labeled, thereby greatly improving the quality of data labeling.

[0019] 5. Automatically identify the industry of the data to be labeled through pre-trained recurrent neural networks, and automatically identify the project number from the data to be labeled through numbering rules, which greatly improves the recognition speed of industry and project numbers, thereby greatly improving the efficiency of data labeling.

[0020] 6. By performing word segmentation operations through the natural language word segmentation tool hanLP, the word boundaries in the data to be labeled can be accurately identified. Even when faced with complex sentence structures and polysemous words, reasonable word segmentation decisions can be made to ensure the accuracy of the word segmentation results, thereby greatly improving the quality of data labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.

[0022] Figure 1 It is a flow chart of a data labeling method based on an industry vocabulary library and a tag library of the present invention.

[0023] Figure 2 It is a structural schematic diagram of a data annotation system based on an industry vocabulary library and a tag library of the present invention. DETAILED DESCRIPTION

[0024] The overall idea of ​​the technical solution in the embodiments of the present application is as follows: the data to be annotated obtained by identifying the annotated file is automatically annotated by creating an industry vocabulary and label library, thereby overcoming the problems of subjective judgment, low efficiency and high annotation costs of manual annotation, and performing word segmentation operations based on industry word segmentation of the industry vocabulary, making word segmentation more professional and accurate, thereby improving the quality and efficiency of data annotation and reducing annotation costs.

[0025] Please refer to Figure 1 to Figure 2 As shown, a preferred embodiment of a data labeling method based on an industry vocabulary library and a tag library of the present invention includes the following steps: Step S1, obtaining a large amount of historical annotated data covering different industries, and grouping each of the historical annotated data based on the industry; Step S2: extract the corresponding industry segmentation words from the historical annotation data of each group, and store the industry segmentation words of each group in a pre-created industry word library; extract project labels from each of the historical annotation data, and store each of the project labels, the project numbers corresponding to the project labels, and the association relationship with the industry segmentation words in a pre-created label library; in specific implementation, support multi-project word library information sharing, different projects can share the same industry word library, and perform professional vocabulary (industry segmentation) recognition according to the industry to which they belong and the selected word library; Step S3: obtaining the file to be annotated through the workflow engine, identifying the file to be annotated to obtain the data to be annotated, and identifying the industry and project number of the data to be annotated; Step S4: Match the industry to the corresponding industry vocabulary, and the natural language word segmentation tool performs a word segmentation operation on the text data based on the industry word segmentation of the industry vocabulary to obtain a word segmentation result; match the project number to the corresponding tag library, and pre-label the word segmentation result based on the project tag of the tag library to obtain a pre-labeled result; Step S5: after reviewing and correcting the pre-annotation results, generate an annotation result, display and export the annotation result.

[0026] The present invention constructs an industry vocabulary and applies industry word segmentation to the data annotation process. The industry word segmentation is organically integrated with the general word segmentation, so that the word segmentation is more professional and accurate when annotating industry data. At the same time, a flexible project label library management function is provided, allowing users to customize project labels according to annotation requirements to better meet project-level data annotation services. The industry word segmentation and project labels are organically combined. For industry projects, industry professional word segmentation is first performed according to the industry vocabulary, and then statistics are performed according to the annotation relationship (association relationship) between the industry word segmentation and the project label. The high-frequency annotation results are used as a reference to automatically pre-annotate each word segmentation result to improve the efficiency and accuracy of annotation. The workflow engine is used for annotation management, including annotation progress tracking, annotation task assignment, annotation review, etc., to improve the management efficiency and usability of annotation, and also support parallel annotation processing to meet the needs of large-scale annotation tasks.

[0027] The step S2 further comprises: Based on the latest annotation data or annotation requirements, the industry vocabulary and label library are updated and maintained in real time.

[0028] Through the latest annotation data or annotation requirements, the industry vocabulary and label library are updated and maintained in real time, so that the industry vocabulary and label library can keep pace with the times, and the project labels in the label library can be customized according to the user's annotation needs, thereby greatly improving the quality and flexibility of data annotation.

[0029] The step S3 is specifically as follows: The workflow engine is used to obtain the files to be annotated, and corresponding storage space and computing resources are allocated to each file to be annotated, and each file to be annotated is stored in the corresponding storage space. After preprocessing each file to be annotated with image enhancement, each file to be annotated is identified by OCR technology to obtain the data to be annotated, and the industry of the data to be annotated is automatically identified by the pre-trained recurrent neural network, and the project number is automatically identified from the data to be annotated by the numbering rule. The file to be annotated is the project file, which can be in the format of PDF, WORD, picture, etc.

[0030] By allocating corresponding storage space and computing resources to each file to be labeled, each file to be labeled can be labeled simultaneously, which greatly improves the efficiency of data labeling.

[0031] After preprocessing the image enhancement of each file to be labeled, the OCR technology is used to identify each file to be labeled to obtain the data to be labeled. That is, OCR recognition is performed based on high-quality files to be labeled, which effectively improves the accuracy of identifying the data to be labeled, thereby greatly improving the quality of data labeling.

[0032] The industry of the data to be labeled is automatically identified through a pre-trained recurrent neural network, and the project number is automatically identified from the data to be labeled through numbering rules, which greatly improves the recognition speed of the industry and project number, and thus greatly improves the efficiency of data labeling.

[0033] The step S4 is specifically as follows: The industry is automatically matched to the corresponding industry vocabulary, and the natural language word segmentation tool hanLP automatically performs word segmentation operations on the text data based on the industry word segmentation of the industry vocabulary to obtain the word segmentation results; the project number is matched to the corresponding tag library, and the word segmentation results are automatically pre-annotated based on the association relationship and the number of annotations of the project tags in the tag library to obtain the pre-annotated results.

[0034] By performing word segmentation operations through the natural language word segmentation tool hanLP, we can accurately identify the vocabulary boundaries in the data to be labeled. Even when faced with complex sentence structures and polysemous words, we can make reasonable word segmentation decisions to ensure the accuracy of the word segmentation results, thereby greatly improving the quality of data labeling.

[0035] The step S5 is specifically as follows: After automatically reviewing and correcting the pre-annotation results based on the preset annotation rules, the annotation results are generated, displayed and exported, stored and backed up, and the annotation progress is tracked through the workflow engine during the annotation process. In specific implementation, keyword fuzzy query is supported for the annotation results for rapid positioning. In specific implementation, the audit task can be executed based on the preset permissions and roles.

[0036] A preferred embodiment of a data annotation system based on an industry vocabulary and a tag library of the present invention includes the following modules: A historical annotation data acquisition module is used to acquire a large amount of historical annotation data covering different industries, and group the historical annotation data based on the industry; The industry word library and label library creation module is used to extract the corresponding industry segmentation words from the historical annotation data of each group, and store the industry segmentation words of each group in a pre-created industry word library; extract project labels from each of the historical annotation data, and store each of the project labels, the project numbers corresponding to the project labels, and the association relationship with the industry segmentation words in a pre-created label library; in specific implementation, it supports the sharing of multi-project word library information, and different projects can share the same industry word library, and professional vocabulary (industry segmentation) is recognized according to the industry to which they belong and the selected word library; A module for identifying files to be annotated, used to obtain files to be annotated through a workflow engine, identify the files to be annotated to obtain data to be annotated, and identify the industry and project number of the data to be annotated; A pre-marking module is used to match the industry with the corresponding industry vocabulary library, and the natural language word segmentation tool performs word segmentation operation on the text data based on the industry word segmentation of the industry vocabulary library to obtain a word segmentation result; match the corresponding tag library with the project number, and pre-mark the word segmentation result based on the project tag of the tag library to obtain a pre-marking result; The annotation review module is used to review and correct the pre-annotation results, generate annotation results, and display and export the annotation results.

[0037] The present invention constructs an industry vocabulary and applies industry word segmentation to the data annotation process. The industry word segmentation is organically integrated with the general word segmentation, so that the word segmentation is more professional and accurate when annotating industry data. At the same time, a flexible project label library management function is provided, allowing users to customize project labels according to annotation requirements to better meet project-level data annotation services. The industry word segmentation and project labels are organically combined. For industry projects, industry professional word segmentation is first performed according to the industry vocabulary, and then statistics are performed according to the annotation relationship (association relationship) between the industry word segmentation and the project label. The high-frequency annotation results are used as a reference to automatically pre-annotate each word segmentation result to improve the efficiency and accuracy of annotation. The workflow engine is used for annotation management, including annotation progress tracking, annotation task assignment, annotation review, etc., to improve the management efficiency and usability of annotation, and also support parallel annotation processing to meet the needs of large-scale annotation tasks.

[0038] The industry vocabulary and label library creation module is also used for: Based on the latest annotation data or annotation requirements, the industry vocabulary and label library are updated and maintained in real time.

[0039] Through the latest annotation data or annotation requirements, the industry vocabulary and label library are updated and maintained in real time, so that the industry vocabulary and label library can keep pace with the times, and the project labels in the label library can be customized according to the user's annotation needs, thereby greatly improving the quality and flexibility of data annotation.

[0040] The to-be-annotated file identification module is specifically used for: The workflow engine is used to obtain the files to be annotated, and corresponding storage space and computing resources are allocated to each file to be annotated, and each file to be annotated is stored in the corresponding storage space. After preprocessing each file to be annotated with image enhancement, each file to be annotated is identified by OCR technology to obtain the data to be annotated, and the industry of the data to be annotated is automatically identified by the pre-trained recurrent neural network, and the project number is automatically identified from the data to be annotated by the numbering rule. The file to be annotated is the project file, which can be in the format of PDF, WORD, picture, etc.

[0041] By allocating corresponding storage space and computing resources to each file to be labeled, each file to be labeled can be labeled simultaneously, which greatly improves the efficiency of data labeling.

[0042] After preprocessing the image enhancement of each file to be labeled, the OCR technology is used to identify each file to be labeled to obtain the data to be labeled. That is, OCR recognition is performed based on high-quality files to be labeled, which effectively improves the accuracy of identifying the data to be labeled, thereby greatly improving the quality of data labeling.

[0043] The industry of the data to be labeled is automatically identified through a pre-trained recurrent neural network, and the project number is automatically identified from the data to be labeled through numbering rules, which greatly improves the recognition speed of the industry and project number, and thus greatly improves the efficiency of data labeling.

[0044] The pre-marking module is specifically used for: The industry is automatically matched to the corresponding industry vocabulary, and the natural language word segmentation tool hanLP automatically performs word segmentation operations on the text data based on the industry word segmentation of the industry vocabulary to obtain the word segmentation results; the project number is matched to the corresponding tag library, and the word segmentation results are automatically pre-annotated based on the association relationship and the number of annotations of the project tags in the tag library to obtain the pre-annotated results.

[0045] By performing word segmentation operations through the natural language word segmentation tool hanLP, we can accurately identify the vocabulary boundaries in the data to be labeled. Even when faced with complex sentence structures and polysemous words, we can make reasonable word segmentation decisions to ensure the accuracy of the word segmentation results, thereby greatly improving the quality of data labeling.

[0046] The annotation review module is specifically used for: After automatically reviewing and correcting the pre-annotation results based on the preset annotation rules, the annotation results are generated, displayed and exported, stored and backed up, and the annotation progress is tracked through the workflow engine during the annotation process. In specific implementation, keyword fuzzy query is supported for the annotation results for rapid positioning. In specific implementation, the audit task can be executed based on the preset permissions and roles.

[0047] In summary, the advantages of the present invention are: 1. Obtain a large amount of historical annotated data containing different industries, group each historical annotated data based on the industry, extract the corresponding industry segmentation words from the historical annotated data of each group, store the industry segmentation words of each group in a pre-created industry vocabulary, extract project labels from each historical annotated data, store each project label, the project number corresponding to the project label, and the association relationship with the industry segmentation words in a pre-created label library; then obtain the file to be annotated through the workflow engine, identify the file to be annotated to obtain the data to be annotated, and identify the industry and project number of the data to be annotated; then match the industry to the corresponding industry vocabulary, and the natural language segmentation tool will use the industry segmentation words in the industry vocabulary to match the text data. The word segmentation operation is performed according to the data to obtain the word segmentation result; the corresponding tag library is matched by the project number, and the word segmentation result is pre-annotated based on the project tag of the tag library to obtain the pre-annotation result; finally, the pre-annotation result is reviewed and corrected to generate the annotation result, and the annotation result is displayed and exported; that is, the data to be annotated obtained by identifying the annotated file is automatically annotated through the created industry thesaurus and tag library, overcoming the subjective judgment influence of manual annotation, low efficiency, and high annotation cost problems, and the word segmentation operation is performed based on the industry word segmentation of the industry thesaurus, that is, targeted word segmentation is performed based on each industry, making the word segmentation more professional and accurate, ultimately greatly improving the quality and efficiency of data annotation, and greatly reducing the annotation cost.

[0048] 2. Real-time update and maintenance of various industry lexicons and label libraries based on the latest annotation data or annotation requirements enable industry lexicons and label libraries to keep pace with the times, and project labels in the label library can be customized according to user annotation requirements, thereby greatly improving the quality and flexibility of data annotation.

[0049] 3. By allocating corresponding storage space and computing resources to each file to be labeled, each file to be labeled can be labeled simultaneously, greatly improving the efficiency of data labeling.

[0050] 4. After preprocessing the image enhancement of each file to be labeled, the OCR technology is used to identify each file to be labeled to obtain the data to be labeled. That is, OCR recognition is performed based on high-quality files to be labeled, which effectively improves the accuracy of identifying the data to be labeled, thereby greatly improving the quality of data labeling.

[0051] 5. Automatically identify the industry of the data to be labeled through pre-trained recurrent neural networks, and automatically identify the project number from the data to be labeled through numbering rules, which greatly improves the recognition speed of industry and project numbers, thereby greatly improving the efficiency of data labeling.

[0052] 6. By performing word segmentation operations through the natural language word segmentation tool hanLP, the word boundaries in the data to be labeled can be accurately identified. Even when faced with complex sentence structures and polysemous words, reasonable word segmentation decisions can be made to ensure the accuracy of the word segmentation results, thereby greatly improving the quality of data labeling.

[0053] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A data annotation method based on an industry vocabulary and a tag library, characterized by: The steps include: Step S1, obtaining a large amount of historical annotated data covering different industries, and grouping the historical annotated data based on the industry; Step S2: extract the corresponding industry segmentation words from the historical annotation data of each group, and store the industry segmentation words of each group in a pre-created industry word library; extract project labels from each of the historical annotation data, and store each of the project labels, the project numbers corresponding to the project labels, and the association relationship with the industry segmentation words in a pre-created label library; Step S3: obtaining the file to be annotated through the workflow engine, identifying the file to be annotated to obtain the data to be annotated, and identifying the industry and project number of the data to be annotated; Step S4: Match the industry to the corresponding industry vocabulary, and the natural language word segmentation tool performs a word segmentation operation on the text data based on the industry word segmentation of the industry vocabulary to obtain a word segmentation result; match the project number to the corresponding tag library, and pre-label the word segmentation result based on the project tag of the tag library to obtain a pre-labeled result; Step S5: after reviewing and correcting the pre-annotation results, generate an annotation result, display and export the annotation result.

2. A data labeling method based on an industry vocabulary and a tag library as claimed in claim 1, characterized in that: The step S2 further comprises: Based on the latest annotation data or annotation requirements, the industry vocabulary and label library are updated and maintained in real time.

3. The data labeling method based on an industry vocabulary and a tag library as claimed in claim 1, characterized in that: The step S3 is specifically as follows: The files to be labeled are obtained through the workflow engine, corresponding storage space and computing resources are allocated to each file to be labeled, each file to be labeled is stored in the corresponding storage space, each file to be labeled is pre-processed for image enhancement, each file to be labeled is identified through OCR technology to obtain the data to be labeled, the industry of the data to be labeled is automatically identified through a pre-trained recurrent neural network, and the project number is automatically identified from the data to be labeled according to the numbering rules.

4. The data labeling method based on an industry vocabulary and a tag library as claimed in claim 1, characterized in that: The step S4 is specifically as follows: The industry is automatically matched to the corresponding industry vocabulary, and the natural language word segmentation tool hanLP automatically performs word segmentation operations on the text data based on the industry word segmentation of the industry vocabulary to obtain the word segmentation results; the project number is matched to the corresponding tag library, and the word segmentation results are automatically pre-annotated based on the association relationship and the number of annotations of the project tags in the tag library to obtain the pre-annotated results.

5. The data labeling method based on an industry vocabulary and a tag library as claimed in claim 1, characterized in that: The step S5 is specifically as follows: After the pre-annotation results are automatically reviewed and corrected based on preset annotation rules, annotation results are generated, displayed and exported, stored and backed up, and the annotation progress is tracked through the workflow engine during the annotation process.

6. A data annotation system based on an industry vocabulary and a tag library, characterized by: Includes the following modules: A historical annotation data acquisition module is used to acquire a large amount of historical annotation data covering different industries, and group the historical annotation data based on the industry; An industry word library and label library creation module is used to extract the corresponding industry segmentation words from the historical annotation data of each group, and store the industry segmentation words of each group in a pre-created industry word library; extract project labels from each of the historical annotation data, and store each of the project labels, the project numbers corresponding to the project labels, and the association relationship with the industry segmentation words in a pre-created label library; A module for identifying files to be annotated is used to obtain files to be annotated through a workflow engine, identify the files to be annotated to obtain data to be annotated, and identify the industry and project number of the data to be annotated; A pre-marking module is used to match the industry with the corresponding industry vocabulary library, and the natural language word segmentation tool performs word segmentation operation on the text data based on the industry word segmentation of the industry vocabulary library to obtain a word segmentation result; match the corresponding tag library with the project number, and pre-mark the word segmentation result based on the project tag of the tag library to obtain a pre-marking result; The annotation review module is used to review and correct the pre-annotation results, generate annotation results, and display and export the annotation results.

7. A data annotation system based on an industry vocabulary and a tag library as claimed in claim 6, characterized in that: The industry vocabulary and label library creation module is also used for: Based on the latest annotation data or annotation requirements, the industry vocabulary and label library are updated and maintained in real time.

8. The data annotation system based on the industry vocabulary and tag library as claimed in claim 6, characterized in that: The to-be-annotated file identification module is specifically used for: The files to be labeled are obtained through the workflow engine, corresponding storage space and computing resources are allocated to each file to be labeled, each file to be labeled is stored in the corresponding storage space, each file to be labeled is pre-processed for image enhancement, each file to be labeled is identified through OCR technology to obtain the data to be labeled, the industry of the data to be labeled is automatically identified through a pre-trained recurrent neural network, and the project number is automatically identified from the data to be labeled according to the numbering rules.

9. The data annotation system based on the industry vocabulary and tag library as claimed in claim 6, characterized in that: The pre-marking module is specifically used for: The industry is automatically matched to the corresponding industry vocabulary, and the natural language word segmentation tool hanLP automatically performs word segmentation operations on the text data based on the industry word segmentation of the industry vocabulary to obtain the word segmentation results; the project number is matched to the corresponding tag library, and the word segmentation results are automatically pre-annotated based on the association relationship and the number of annotations of the project tags in the tag library to obtain the pre-annotated results.

10. The data annotation system based on industry vocabulary and tag library as claimed in claim 6, characterized in that: The annotation review module is specifically used for: After the pre-annotation results are automatically reviewed and corrected based on preset annotation rules, annotation results are generated, displayed and exported, stored and backed up, and the annotation progress is tracked through the workflow engine during the annotation process.

Images