A data labeling method, apparatus, device, medium, and product

Abstract

The application discloses a data labeling method, device, equipment, medium and product. The data labeling method comprises the following steps: obtaining first rich text document data; splitting the first rich text document data according to structure elements of the rich text, so as to obtain a plurality of data segments corresponding to a plurality of data elements one by one; then displaying the obtained data segments; and in response to a labeling operation of a user on to-be-labeled data in the data segments, generating a label of the to-be-labeled data. According to the embodiment of the application, the rapid convergence of model training can be realized, and the efficiency of model training is improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically to the fields of machine learning and deep learning, and in particular to a data annotation method, apparatus, device, medium and product. Background Technology

[0002] The label data in the training data used for model training needs to be labeled. Currently, all data labeling tools on the market can only label plain text documents.

[0003] Therefore, before labeling rich text data, it is necessary to convert the rich text to plain text format. However, a small number of samples will always have format conversion errors. In addition, the converted plain text document loses the prior structural information of the rich text. For example, when the rich text is an HTML document, the converted plain text document will lose the original HTML tag information, such as h1-h4 (representing headings), bold, italics, and other rich text information, which often helps the model converge faster during training.

[0004] Therefore, existing data labeling methods are not conducive to the rapid convergence of model training, resulting in low model training efficiency. Summary of the Invention

[0005] This application provides a data annotation method, apparatus, device, medium, and product that can achieve rapid convergence of model training and improve the efficiency of model training.

[0006] In a first aspect, embodiments of this application provide a data annotation method, the method comprising:

[0007] Retrieve the first rich text document data;

[0008] The first rich text document data is split according to the structure elements of the rich text, resulting in multiple data fragments that correspond one-to-one with multiple data elements;

[0009] Display data fragments;

[0010] In response to user annotation operations on the data to be labeled in the data segment, generate labels for the data to be labeled.

[0011] Secondly, embodiments of this application provide a data annotation apparatus, which includes:

[0012] The acquisition module is used to acquire the data from the first rich text document.

[0013] The splitting module is used to split the first rich text document data according to the structure elements of the rich text, and obtain multiple data fragments that correspond one-to-one with multiple data elements.

[0014] The display module is used to display data fragments;

[0015] The generation module is used to generate labels for the data to be labeled in response to the user's labeling operation on the data to be labeled in the data segment.

[0016] Thirdly, embodiments of this application provide an electronic device, which includes:

[0017] Processor and memory storing computer program instructions;

[0018] When the processor executes computer program instructions, it implements the steps of the data annotation method as described in any of the embodiments of the first aspect.

[0019] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the steps of the data annotation method as described in any embodiment of the first aspect.

[0020] Fifthly, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform the steps of the data annotation method as described in any embodiment of the first aspect.

[0021] The data annotation method in this embodiment, after obtaining the first rich text document data to be annotated, splits it according to the structural elements of the rich text, obtaining multiple data segments that correspond one-to-one with multiple data elements. Then, the user annotates the data to be annotated based on these data segments. In this method, because the data segments are split according to the structural elements of the rich text, the resulting annotated data includes the structural elements of the rich text. Therefore, the annotated data obtained through this data annotation method retains the structural element information of the rich text, i.e., rich text label information. Rich text label information is beneficial for model convergence during model training. Therefore, model training based on the annotated data obtained through this data annotation method improves the efficiency of model training. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a data annotation method provided in an embodiment of this application;

[0024] Figure 2This is a flowchart illustrating another data annotation method provided in an embodiment of this application;

[0025] Figure 3 This is a schematic diagram illustrating a specific implementation of S270 in this application;

[0026] Figure 4 This is a flowchart illustrating another data annotation method provided in an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of the structure of a data annotation device provided in an embodiment of this application;

[0028] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0030] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0031] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0032] HTML: It is a standard markup language for creating web pages, a fundamental technology that is often used by many websites, along with CSS and JavaScript, to design the user interfaces of web pages, web applications, and mobile applications.

[0033] To facilitate understanding of the technical solutions of the embodiments of this application, a brief introduction to the background technology corresponding to the embodiments of this application will be given first.

[0034] As described in the background section, the inventors discovered that all data annotation tools currently available on the market can only annotate plain text documents. Therefore, before annotating rich text data, it is necessary to convert the rich text to plain text format. However, during the conversion process, a small number of samples with format conversion errors will appear, and the prior structural information in the rich text will be lost. Such a data annotation method is not conducive to the rapid convergence of model training and will result in low model training efficiency.

[0035] To address the issue of rapid convergence during model training, this application provides a data annotation method. This method involves acquiring a first rich text document (rich text document) to be annotated, then splitting it according to the document's structural elements to obtain multiple data segments, each corresponding to a specific data element. The user then uses these data segments to annotate the data to be annotated. The annotated data obtained after this process includes the rich text document's structural elements; therefore, the annotated data obtained through this method retains the rich text document's structural element information, i.e., rich text label information. This allows for the addition of feature dimensions during subsequent model training, improving training efficiency.

[0036] The data annotation method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0037] Figure 1 This is a flowchart illustrating a data annotation method provided in an embodiment of this application, such as... Figure 1 As shown, the data annotation method provided in this application embodiment may include the following steps:

[0038] S110, Obtain the data of the first rich text document;

[0039] S120. Split the first rich text document data according to the structure elements of the rich text to obtain multiple data fragments that correspond one-to-one with multiple data elements.

[0040] S130, Display data segment;

[0041] S140. In response to the user's annotation operation on the data to be annotated in the data segment, generate labels for the data to be annotated.

[0042] Therefore, in the embodiments of this application, after obtaining the first rich text document data to be labeled, it is split according to the structural elements of the rich text to obtain multiple data segments that correspond one-to-one with multiple data elements. Then, the user labels the data to be labeled based on these data segments. In this data labeling method, since the data segments are split according to the structural elements of the rich text, the resulting labeled data includes the structural elements of the rich text. Therefore, the labeled data obtained by this data labeling method retains the structural element information of the rich text, i.e., the rich text label information. The rich text label information is beneficial to model convergence during model training. Therefore, model training based on the labeled data obtained by this data labeling method improves the efficiency of model training.

[0043] The specific implementation methods for each of the above steps are described below.

[0044] In some embodiments, in S110, the first rich text document data involved in the embodiments of this application may be HTML text document data.

[0045] As an example, the HTML text document here could be, for instance, a news article or paper in HTML format.

[0046] In some embodiments, in S120, the first rich text document data is split according to the structure elements of the rich text.

[0047] As an example, the structural elements could be the title, paragraphs, subscript text, superscript text, standard cells, and document title of an HTML news article, with each of the split structural elements recorded as a data fragment.

[0048] In some embodiments, in S130, the displayed data fragment may be, for example, a text fragment of an HTML news article, for data annotation operations.

[0049] In some embodiments, in S140, the user performs annotation operations on the data to be annotated in the data segment. The user can operate on each line of text in the data segment. After completing this operation, the label of the data to be annotated is generated and further saved to the database.

[0050] To achieve rapid convergence of model training and improve the efficiency of model training, this application also provides another implementation of the data annotation method, as detailed in the following embodiments.

[0051] Please see Figure 2 In addition to steps S110 to S120, the method also includes the following steps after S120:

[0052] S210. Record the data segment identifier, structural element information, and style of the corresponding rich text document data for each data segment. The data segment identifier is determined according to the order of the data segments in the first rich text document data.

[0053] In some embodiments, after splitting the rich text according to its structural elements, data fragments corresponding to each structural element are obtained, and the data fragment's data sequence number, structural element information, and the style of the rich text document data need to be recorded. When the rich text is an HTML document, as an example, the data fragments can be text fragments corresponding to any of the following: HTML format news titles, paragraphs, subscript text, superscript text, standard cells, and document titles.

[0054] As an example, the data style of a rich text document can be, for example, font, color, and font size, and the paragraph numbering of the data fragments is determined according to the top-to-bottom order of paragraphs in an HTML news article.

[0055] S220. Identify the punctuation marks contained in multiple clauses within each data segment;

[0056] In some embodiments, the punctuation marks included in the data fragment may be, for example, commas, periods, semicolons, and line breaks in a paragraph of an HTML news article. These punctuation marks are determined for the next step of sentence segmentation.

[0057] S230. The data segment is divided into sentences according to punctuation marks, and each sentence is identified by a sentence identifier, which is determined according to the order in the data segment.

[0058] In some embodiments, such as after determining the punctuation marks in an HTML news article, the text can be split according to these punctuation marks, and each split sentence can be recorded with a sentence number, where the sentence number is determined according to the order of the text in the data paragraph.

[0059] S240. Record the annotation data corresponding to the data to be annotated. The annotation data includes: the data segment identifier where the data to be annotated is located, structural elements, the style of the rich text document data, and sentence identifiers.

[0060] In some embodiments, the annotation data corresponding to the data to be annotated is recorded, and the annotation data is further saved to the database.

[0061] As an example, the labeled data could be any data segment from an HTML news article, such as paragraph numbers, structural elements, rich text document data styles, and sentence numbers.

[0062] S250: Based on the data segment identifier, structural element information, rich text document data style, and sentence identifier of each data segment, construct the first rich text document data into a scalable vector graphics SVG screen.

[0063] In some embodiments, after a user annotates each data fragment, structural element information, rich text document data style, and sentence identifier, each result after the above annotation operation is constructed into an SVG screen, which makes it easier for the user to annotate the data of the rich text document. For example, after annotating an HTML news article, each result is constructed into an SVG screen, allowing the user to more clearly understand the data information of the entire rich text document, which facilitates subsequent annotation operations.

[0064] S260. Convert the SVG background into SVG text;

[0065] In some embodiments, after each result following the annotation operation is constructed into an SVG canvas, the entire SVG canvas is then converted into SVG text.

[0066] S270. Based on the text in the SVG, the user performs annotation operations on the data to be annotated in each line of text in the SVG, and determines the start and end positions of the data to be annotated.

[0067] In some embodiments, users are allowed to annotate each line of text in the SVG, thus achieving a user annotation process for rich text document data. After the user performs annotation operations on the SVG, the start and end positions of the annotated characters are recorded.

[0068] As an example, a user can annotate data segments displayed in a document, such as... Figure 3 As shown, Figure 3 This is a schematic diagram illustrating the specific implementation of S270 in this application.

[0069] If a user wants to annotate the "project name", they can see that "one two three four five six seven" is the fourth sentence in the data segment in the document. They can also see that "one two three four five six seven" is in the second segment from the fifth character to the twelfth character. Therefore, users can annotate multiple data segments and save the starting and ending character positions of the annotated data.

[0070] To better distinguish data fragments from sentences, S230 includes the following:

[0071] Label the data segment attributes based on the number of sentences included in the data segment;

[0072] In some embodiments, such as an HTML news article, each data segment has a different number of words and a different number of sentences, so it is necessary to distinguish and record the special attributes of the data segments.

[0073] If the first data segment contains only one sentence, the attribute of the first data segment is recorded as the first preset attribute;

[0074] In some embodiments, such as a news article where any data segment contains only one sentence, this sentence needs to be specially processed and can be recorded as "1". "1" indicates that there is only one sentence in a single paragraph.

[0075] If the first data segment includes at least two sentences, the attributes of the first data segment are recorded as the second preset attributes;

[0076] In some embodiments, such as news articles, if any data segment has at least two sentences and no longer contains only one sentence, it can be recorded as "0", where "0" indicates that there are at least two sentences in a single paragraph.

[0077] Therefore, based on sentence segmentation, further refinement is used to distinguish between cases where a data segment contains only one sentence and cases where a segment contains multiple sentences, thus ensuring the accuracy of data labeling.

[0078] To further achieve rapid convergence in model training, data fragments and their corresponding labeled data are aggregated and processed in batches. Based on these labels, labeled data is obtained, which can then be used for model training, improving training efficiency. This application also provides another implementation of the data labeling method, as detailed in the following embodiments.

[0079] Please see Figure 4 In addition to steps S110 to S140, the method also includes the following steps:

[0080] S410. Take each data segment and its corresponding labeled data as elements in the sequence to construct a sequence containing multiple elements;

[0081] In some embodiments, such as for any HTML-formatted news article, the annotation data corresponding to each paragraph in the news article can be used as a set of sequence data. A set of sequence data will contain multiple elements, and each node in the sequence is a sequence. The annotation data is recorded in S280 and will not be described in detail here.

[0082] S420. Train the model based on the sequence to obtain the target model for extracting elements from rich text document data;

[0083] In some embodiments, the multiple sequences constructed above are input into the model in batches to start training. After training, a trained target model can be obtained, in which multiple elements from the rich text document have been extracted.

[0084] It should be noted that there are many models to choose from here. You can either stack two conventional models or perform mixed training on a dual-output layer neural network. Taking the pre-trained language model BERT as an example, you can use the BERT model to determine whether the sentence contains elements based on the predicted CLS vectors of the text, and then use the BERT model to determine whether the character vector of each character is the beginning or end of an element.

[0085] Therefore, by constructing multiple sets of sequence data from the first data segment and the corresponding labeled data, and then inputting these multiple sets of sequence data into the model for training, the model training results can be more accurate, the model training efficiency can be improved, and the training cost can be reduced.

[0086] To make the elements extracted by the model more accurate and improve the robustness of model training, the following steps can be included after S420:

[0087] Retrieve the data from the second rich text document;

[0088] In some embodiments, the second rich text document data obtained may be HTML text document data, such as a news article or paper in HTML format.

[0089] The second rich text document data is split according to the structure elements of the rich text, resulting in multiple second data fragments that correspond one-to-one with multiple data elements;

[0090] In some embodiments, for example, the structural elements may be the title, paragraphs, subscript text, superscript text, standard cells, and document title of an HTML news article, and each of the split structural elements is recorded as a second data fragment.

[0091] Based on the model constructed above, elements can be extracted from rich text, thus providing another embodiment. Specifically, after obtaining multiple second data fragments, the process may include:

[0092] The second data fragment is input into the target model, and the element information contained in the second data fragment is extracted through the target model. The location information of the target position containing the element information in the text of the second data fragment is also extracted.

[0093] Output element information and position information.

[0094] In some embodiments, the trained model is used as a target model and input into the second data segment again. The second data segment also contains multiple elements. The element information contained in each second data segment can be extracted according to the target model. Furthermore, the position information of the element information in the current data segment can be extracted, that is, the position information of the element information in the second data segment can be extracted. The extracted element information and position information are then output.

[0095] Therefore, by inputting the second data fragment into the target model, the element information and the location information of the element information are extracted, and the accuracy of the model training is verified again, thereby improving the accuracy of the model training.

[0096] It should be noted that the application scenarios described in the above embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0097] Based on the same inventive concept, this application also provides a data annotation device, specifically combined with... Figure 5 Please provide a detailed explanation.

[0098] Figure 5 This is a schematic diagram of the structure of a data annotation device provided in an embodiment of this application.

[0099] like Figure 5 As shown, the data annotation device 500 may include:

[0100] Module 501 is used to acquire the data of the first rich text document;

[0101] The splitting module 502 is used to split the first rich text document data according to the structure elements of the rich text, and obtain multiple data fragments that correspond one-to-one with multiple data elements.

[0102] Display module 503 is used to display data fragments;

[0103] The generation module 504 is used to generate labels for the data to be labeled in response to the user's labeling operation on the data to be labeled in the data segment.

[0104] In some embodiments, to facilitate data generation, the data annotation apparatus may further include the following modules:

[0105] The first recording module is used to record the data segment identifier, structural element information, and style of the corresponding rich text document data for each data segment after obtaining multiple data segments that correspond one-to-one with multiple data elements. The data segment identifier is determined according to the order of the data segments in the first rich text document data.

[0106] A determination module is used to determine the punctuation marks contained in multiple clauses in each of the data segments;

[0107] The sentence segmentation module is used to segment the data segment into sentences according to punctuation marks and record the sentence identifier of each sentence, wherein the sentence identifier is determined according to the order in the data segment;

[0108] The second recording module is used to record the annotation data corresponding to the data to be annotated. The annotation data includes: the data segment identifier where the data to be annotated is located, the structural element, the style of the rich text document data, and the sentence identifier.

[0109] In some embodiments, to facilitate user operation of each line of text in the data to be annotated, and to further determine the start and end positions of the characters in the data to be annotated, the data annotation device further includes the following modules:

[0110] The building module is used to construct a scalable vector graphics SVG backdrop from the first rich text document data based on the data fragment identifier, structural element information, rich text document data style, and sentence identifier of each data fragment.

[0111] The conversion module is used to convert SVG backgrounds into SVG text;

[0112] The determination module is used to determine the start and end positions of the data to be annotated based on the text in the SVG and the annotation operations performed by the user on the data to be annotated in each line of the SVG.

[0113] The labeled data also includes the start and end positions of the characters in the data to be labeled.

[0114] In some embodiments, to further refine sentence segmentation and better distinguish data fragments from sentences, the data annotation device further includes the following modules:

[0115] The annotation module is used to annotate the fragment attributes of a data fragment based on the number of sentences included in the data fragment;

[0116] The first recording module is used to record the attributes of the first data segment as the first preset attributes when the first data segment contains only one sentence.

[0117] The second recording module is used to record the attributes of the first data segment as a second preset attribute when the first data segment includes at least two sentences.

[0118] In some embodiments, to further achieve rapid convergence of model training, the data annotation device further includes the following modules for batch processing of data fragments and corresponding labeled data:

[0119] The building module is used to construct a sequence containing multiple elements by taking each data fragment and its corresponding labeled data as elements in the sequence;

[0120] The extraction module is used to train the model based on the sequence to obtain the target model for extracting elements from rich text document data.

[0121] In some embodiments, to make the elements extracted by the model more accurate and improve the robustness of model training, the data annotation device further includes the following modules:

[0122] Module 501 is used to acquire the second rich text document data;

[0123] The splitting module 502 is used to split the second rich text document data according to the structure elements of the rich text, and obtain multiple second data fragments that correspond one-to-one with multiple data elements.

[0124] The input module is used to input the second data fragment into the target model, extract the element information contained in the second data fragment through the target model, and extract the location information of the target position containing the element information in the text of the second data fragment.

[0125] The output module is used to output element information and position information.

[0126] Therefore, after obtaining the first rich text document data to be labeled, it is split according to the structural elements of the rich text, resulting in multiple data fragments that correspond one-to-one with multiple data elements. The user then labels the data based on these data fragments. In this method, because the data fragments are split according to the structural elements of the rich text, the resulting labeled data includes these structural elements. Therefore, the labeled data obtained through this method retains the structural element information of the rich text, i.e., rich text label information. Rich text label information is beneficial for model convergence during model training. Therefore, training the model based on the labeled data obtained through this data labeling method improves the efficiency of model training.

[0127] Figure 6 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application.

[0128] The electronic device 600 may include a processor 601 and a memory 602 storing computer program instructions.

[0129] Specifically, the processor 601 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0130] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 602 may include removable or non-removable (or fixed) media. Where appropriate, memory 602 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 602 is non-volatile solid-state memory.

[0131] In a particular embodiment, memory 602 includes read-only memory (ROM). Where suitable, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these. Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical / tangible memory storage devices. Thus, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.

[0132] The processor 601 implements any of the data annotation methods described in the above embodiments by reading and executing computer program instructions stored in the memory 602.

[0133] In one example, the electronic device 600 may further include a communication interface 603 and a bus 610. For example... Figure 6 As shown, the processor 601, memory 602, and communication interface 603 are connected through bus 610 and complete communication with each other.

[0134] The communication interface 603 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0135] Bus 610 includes hardware, software, or both, that couples components of a data labeling device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 610 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0136] For example, the electronic device 600 can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc.

[0137] The electronic device 600 can execute the data annotation method in the embodiments of this application, thereby achieving the combination Figure 1 and Figure 5 The data annotation methods and apparatus described.

[0138] Furthermore, in conjunction with the data annotation methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the data annotation methods in the above embodiments.

[0139] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0140] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0141] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0142] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0143] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A data annotation method, characterized in that, include: Retrieve the first rich text document data, which is an HTML text document data; The first rich text document data is split according to the structure elements of rich text to obtain multiple data segments that correspond one-to-one with multiple data elements. The structure elements are the title, paragraph, subscript text, superscript text, standard cell and document title of the first rich text document data. Display the data fragment; In response to a user's annotation operation on the data to be annotated in the data segment, a label is generated for the data to be annotated; Each data segment's data segment identifier, structural element information, and corresponding rich text document data style are recorded. The data segment identifier is determined according to the order of the data segments in the first rich text document data. The rich text document data style includes font, color, and font size. In each of the data segments, the punctuation marks contained in multiple clauses are identified; The data segment is divided into sentences according to punctuation marks, and the sentence identifier of each sentence is recorded. The sentence identifier is determined according to the order in the data segment. Record the annotation data corresponding to the data to be annotated, the annotation data including: the data segment identifier where the data to be annotated is located, the structural element, the style of the rich text document data, and the sentence identifier; Based on the data segment identifier, structural element information, rich text document data style, and sentence identifier of each data segment, the first rich text document data is constructed into a scalable vector graphics SVG screen. Convert the SVG background into SVG text; Based on the text in the SVG, the user performs annotation operations on the data to be annotated in each line of text in the SVG, and determines the start and end positions of the data to be annotated. The annotation data also includes the start and end positions of the characters in the data to be annotated.

2. The method according to claim 1, characterized in that, After segmenting the data fragment into sentences according to punctuation marks, the method further includes: The fragment attributes of the data fragment are labeled according to the number of sentences included in the data fragment; If the first data segment contains only one sentence, the attribute of the first data segment is recorded as the first preset attribute; If the first data segment includes at least two sentences, the attributes of the first data segment are recorded as the second preset attributes.

3. The method according to claim 1, characterized in that, Also includes: Each data fragment and its corresponding labeled data are used as elements in a sequence to construct a sequence containing multiple elements; The model is trained based on the sequence to obtain a target model for extracting elements from rich text document data.

4. The method according to any one of claims 1-3, characterized in that, After obtaining the target model for extracting elements from rich text document data, the method further includes: Retrieve the data from the second rich text document; The second rich text document data is split according to the structure elements of the rich text to obtain multiple second data fragments that correspond one-to-one with multiple data elements; For each of the second data segments, perform the following operations: The second data fragment is input into the target model, and the element information contained in the second data fragment is extracted through the target model. The location information of the target position containing the element information in the text of the second data fragment is also extracted. Output the element information and the position information.

5. A data annotation device, characterized in that, The device includes: The acquisition module is used to acquire the first rich text document data, which is HTML text document data. The splitting module is used to split the first rich text document data according to the structure elements of the rich text, to obtain multiple data segments that correspond one-to-one with multiple data elements. The structure elements are the title, paragraph, subscript text, superscript text, standard cell and document title of the first rich text document data. A display module is used to display the data fragments; The generation module is used to generate labels for the data to be labeled in response to the user's labeling operation on the data to be labeled in the data segment; The first recording module is used to record the data segment identifier, structural element information, and style of the corresponding rich text document data for each data segment after obtaining multiple data segments that correspond one-to-one with multiple data elements. The data segment identifier is determined according to the order of the data segments in the first rich text document data. The style of the rich text document data is the font, color, and font size. A determination module is used to determine the punctuation marks contained in multiple clauses in each of the data segments; The sentence segmentation module is used to segment the data segment into sentences according to punctuation marks and record the sentence identifier of each sentence, wherein the sentence identifier is determined according to the order in the data segment; The second recording module is used to record the annotation data corresponding to the data to be annotated. The annotation data includes: the data segment identifier where the data to be annotated is located, the structural element, the style of the rich text document data, and the sentence identifier. The construction module is used to construct a scalable vector graphics SVG backdrop from the first rich text document data based on the data segment identifier, structural element information, rich text document data style, and sentence identifier of each data segment. A conversion module is used to convert the SVG background into SVG text; The determination module is used to determine the start and end positions of the data to be annotated based on the text in the SVG and the annotation operations performed by the user on the data to be annotated in each line of text in the SVG. The annotation data also includes the start and end positions of the characters in the data to be annotated.

6. An electronic device, characterized in that, The electronic device includes: Processor and memory storing computer program instructions; When the processor executes the computer program instructions, it implements the steps of the data annotation method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the steps of the data annotation method as described in any one of claims 1-4.

8. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device is able to perform the steps of the data annotation method as described in any one of claims 1-4.

Images