Data labeling method, device and electronic equipment for engineering field HTML document

Abstract

The application provides a data labeling method and device for an engineering field HTML document and electronic equipment, and relates to the technical field of data processing. The method comprises the following steps: obtaining to-be-labeled data, performing named entity recognition marking on a preset target phrase of a target engineering contained in the to-be-labeled data, and obtaining an entity mark corresponding to the preset target phrase; performing offset calculation according to the entity mark, and obtaining an offset corresponding to the preset target phrase; performing relationship matching on the preset target phrase according to the offset and a context semantic relationship of the to-be-labeled data, and obtaining a labeling result of the relationship matching. The device is used for executing the above method. The application obtains a more accurate entity and entity relationship extraction method for the engineering field by automatically identifying entities and calculating the relationship between entities, and improves the efficiency of data labeling.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically, to a data annotation method, apparatus, and electronic device for HTML documents in the engineering field. Background Technology

[0002] The rapid development of the Internet has led to a dramatic increase in data volume, and the arrival of the big data era has also led to a continuous improvement in the level of informatization. In order to more efficiently mine more valuable and accurate information, Natural Language Processing (NLP) technology has emerged.

[0003] In existing NLP technologies, the key steps of Named Entity Recognition (NER) annotation and NER relation annotation heavily rely on manual labor, resulting in low annotation efficiency. Summary of the Invention

[0004] The purpose of this application is to provide a data annotation method, apparatus, and electronic device for HTML documents in the engineering field, which can improve the efficiency of data annotation for entity annotation and relationship extraction between entities in the engineering field.

[0005] In a first aspect, embodiments of this application provide a data annotation method for HTML documents in the engineering field, comprising: first, acquiring data to be annotated, and performing named entity recognition tagging on a preset target phrase of a target project contained in the data to be annotated to obtain entity tags corresponding to the preset target phrases; wherein, the data to be annotated is HTML document data; then, performing offset calculation based on the entity tags to obtain the offset corresponding to the preset target phrases; wherein, the offset refers to the relative positional distance of the entity tags relative to the data to be annotated; finally, performing relation matching on the preset target phrases based on the offset and the contextual semantic relationship of the data to be annotated to obtain the relation matching annotation result.

[0006] In the technical solution of this application embodiment, since the acquired data to be labeled may contain not only the required data but also a lot of invalid data, the required data entities are first labeled, and then the relationships between the labeled entities are obtained according to the offset and contextual semantic relationships, so that the relationships between entities can be extracted in a targeted manner, thereby improving the efficiency of data labeling.

[0007] In some embodiments, naming entity recognition is performed on a preset target phrase of the target project contained in the data to be labeled to obtain entity tags corresponding to the preset target phrase. This includes: segmenting the data to be labeled to obtain multiple segments and identifying the attribute value of each segment; if the attribute value is a preset attribute value, then the segment is determined to be a preset target phrase; and generating entity tags for the preset target phrase.

[0008] Since a sentence or paragraph in the data to be labeled may contain the required data, but directly labeling a sentence or paragraph would lead to inaccurate labeling, it is necessary to segment the data into words. In addition, it is also necessary to label the attribute values ​​of each word segment and determine whether it is a preset target phrase based on the attribute values. In this process, the attribute values ​​of each word segment are used to determine whether it is the required data, thereby eliminating data that does not need to be analyzed. Therefore, the accuracy of the data to be analyzed is improved, further increasing the efficiency of data labeling.

[0009] In some embodiments, the preset attribute values ​​include company name attribute and ranking attribute; the preset target phrase is matched for relation based on the offset and the contextual semantic relationship of the data to be labeled, and the labeling result of relation matching is obtained, including: traversing entity tags; if the entity tag contains preset key attribute values, and the entity tag contains ranking attribute and company name attribute, then the target ranking corresponding to the ranking attribute and the target company corresponding to the company name attribute are extracted; the distance between the target ranking and the target company is determined according to the offset of the entity tag; if the target ranking and the target company satisfy the contextual semantic relationship, then the target ranking and the target company with the closest distance are taken as the labeling result of relation matching.

[0010] The system identifies the company corresponding to the "company name" attribute and the ranking corresponding to the "rank" attribute. When traversing the tagged entities, a further filtering is performed based on preset key attribute values. Only entities exhibiting these preset key attribute values ​​are traversed, and only entities exhibiting both the "rank" and "company name" attributes are tagged and extracted. Finally, the distance between the extracted target ranking and target company is calculated based on their offsets. The target ranking and target company with the closest distance and satisfying the contextual semantic relationship are selected as the relationship matching result. Therefore, by further filtering the tagged entity data using preset key attribute values, invalid data is eliminated, effectively reducing noise and thus improving the efficiency of data annotation.

[0011] In some embodiments, the contextual semantic relationship includes the table semantic relationship; the distance between the target ranking and the target company is determined based on the offset of the entity tag; if the target ranking and the target company satisfy the contextual semantic relationship, the target ranking and the target company with the closest distance are used as the labeling result of the relationship matching, including: if the entity tag exists in the table, the target ranking and the target company in the entity tag are extracted based on the relative position of the entity tag in the table; the labeling result of the relationship matching is obtained based on the minimum distance between the table semantic relationship and the relative position.

[0012] Since the data objects for entity labeling may also exist in tables, entity labels are extracted from tables based on their unique characteristics. In this process, corresponding data extraction is performed according to different data formats, further improving the efficiency of data labeling.

[0013] In some embodiments, after performing relation matching on the preset target phrases based on the offset to obtain the annotation results of the relation matching, the method further includes: determining whether the annotation results need to be deduplicated based on the number of sections corresponding to the target project, and if so, deduplicating the annotation results; wherein: the preset conditions include: the number of target companies corresponding to the target ranking in the annotation results is greater than the number of sections.

[0014] After obtaining the labeled data, the final bidding results differ depending on whether the project involves multiple bidding sections or single bidding sections. For example, in a single bidding section project, each ranking corresponds to only one company, while in a multi-bidding section project, which includes multiple sub-projects, a single ranking may correspond to multiple companies. Therefore, it is necessary to determine the number of bidding sections in the target project based on the labeled data to decide whether deduplication is required. In this process, deduplication methods are used for different project types to obtain the final labeled results, improving the accuracy of the labeled data and further enhancing the efficiency of data labeling.

[0015] In some embodiments, deduplication of the annotation results includes: calculating the absolute distance between the target ranking and each target company based on the offset of the entity tag corresponding to the target ranking and the offset of the entity tag corresponding to the target company; obtaining the minimum value of the absolute distance; and deduplicating the annotation results that do not meet the relationship matching corresponding to the minimum value.

[0016] Following Chinese writing conventions, two related entities are likely to be located close to each other in the text. Therefore, by calculating the absolute distance between the target ranking and each target company, the target companies ultimately related to the target ranking are determined. The deduplication process improves the accuracy of the annotation results, thereby further improving the efficiency of data annotation.

[0017] In some embodiments, before obtaining the data to be labeled, performing named entity recognition tagging on the preset target phrase of the target project contained in the data to be labeled, and obtaining the entity tag corresponding to the preset target phrase, the method further includes: obtaining web data, removing the styles, scripts, and annotations of the web data, and merging inline tags and synonyms to obtain the data to be labeled.

[0018] Web data is processed to remove meaningless and distracting data, such as styles, scripts, and documentation source code annotations. By processing the raw web data, meaningful data to be labeled is retained, reducing the time spent traversing invalid data and thus further improving the efficiency of data labeling.

[0019] Secondly, embodiments of this application provide a data processing apparatus for HTML documents in the engineering field, comprising: a tagging module, configured to acquire data to be annotated, and perform named entity recognition tagging on a preset target phrase of a target project contained in the data to be annotated to obtain an entity tag corresponding to the preset target phrase; wherein, the data to be annotated is HTML document data; a calculation module, configured to perform offset calculation based on the entity tag to obtain an offset corresponding to the preset target phrase; wherein, the offset refers to the relative position distance of the entity tag relative to the data to be annotated; and a matching module, configured to perform relationship matching on the preset target phrase based on the offset and the contextual semantic relationship of the data to be annotated to obtain the annotation result of the relationship matching.

[0020] The tagging module marks predefined target phrases in the data to be labeled with named entities. The calculation module calculates the offset of each entity tag, and the matching module performs relationship matching of the predefined target phrases based on the offsets. In this process, since the acquired data to be labeled may contain a lot of invalid data in addition to the required data, the necessary data entities are marked first, and then the relationships between the marked entities are obtained based on the offsets and contextual semantic relationships. This allows for targeted extraction of relationships between entities, thus improving the efficiency of data labeling.

[0021] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, a storage medium, and a bus, wherein the processor and the memory communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor can execute the method steps of the first aspect by calling the program instructions.

[0022] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium, comprising: the computer-readable storage medium storing computer instructions, the computer instructions causing the computer to perform the method steps of the first aspect.

[0023] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart illustrating a data annotation method provided in an embodiment of this application;

[0026] Figure 2 A technical route overview diagram provided for embodiments of this application;

[0027] Figure 3 An overview diagram of relationship matching for labeled data provided in an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of a data processing device structure provided in an embodiment of this application;

[0029] Figure 5 This is a schematic diagram of the electronic device structure provided in an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0031] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0032] It should be noted that all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0033] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0034] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0035] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0036] It is understood that the data processing method provided in this application embodiment can be applied to terminal devices (also known as electronic devices) and servers; wherein the terminal device can specifically be a smartphone, tablet computer, computer, personal digital assistant (PDA), etc.; the server can specifically be an application server or a web server.

[0037] To facilitate understanding, the technical solutions provided in the embodiments of this application will be described below using a server as the execution subject as an example to illustrate the application scenarios of the data annotation method provided in the embodiments of this application.

[0038] For ease of understanding, the relevant technologies involved in the technical solutions provided in the embodiments of this application will be introduced.

[0039] The HanLP model is a toolkit composed of a series of model-based computational methods. Combined with distributed natural language processing using deep neural networks, it features comprehensive functionality, high performance, a clear architecture, up-to-date corpora, and customizability. This model provides lexical analysis, syntactic analysis, text analysis, and sentiment analysis capabilities, and is a highly active natural language processing technology on GitHub.

[0040] HanLP is completely open source, including its dictionary. It does not depend on other JAR files and uses a series of high-speed data structures at the underlying level, such as double array Trie trees, DAWG, and AhoCorasick DoubleArrayTrie, all of which are open source.

[0041] The HanLP utility class allows for one-line access to all functions, with detailed documentation and out-of-the-box usability. Its underlying algorithm has been meticulously optimized, achieving up to 20 million characters per second in ultra-fast word segmentation mode, while requiring only 120MB of memory. In terms of I / O, dictionary loading is extremely fast, requiring only 500ms to start. HanLP has undergone multiple refactorings and is currently updated to version 1.7, adding and improving functions such as Chinese word segmentation, named entity recognition, information extraction, text classification, text clustering, and graphical analysis, significantly improving its efficiency and applicability.

[0042] Figure 1 A flowchart of a data annotation method provided in an embodiment of this application is shown below. Figure 1 As shown, the method includes:

[0043] Step 101: Obtain the data to be labeled, and perform named entity recognition tagging on the preset target phrases of the target project contained in the data to be labeled to obtain the entity tags corresponding to the preset target phrases.

[0044] In the specific implementation process, after the server obtains the data to be labeled, if the data to be labeled contains a preset target phrase of the target project, then the preset target phrase is marked by named entity recognition. The marking mainly includes part-of-speech analysis, position analysis, and analysis of the relationship between each preset target phrase. Finally, the preset target phrase is marked according to the analysis results.

[0045] In this context, "data to be labeled" refers to HTML document data that has undergone preprocessing of web data, and this data can be text data, tabular data, or image data, etc. "Target project" refers to the engineering project for which the user wants to know the bidding information. "Preset target phrase" refers to all suspicious target entities, i.e., specific information about the bidding process in the engineering project that the user wants to know, including but not limited to: project name, bid type, bid number, company name of the winning candidate, ranking, performance, winning bid price, etc. This application uses text data and tabular data as examples to perform named entity recognition tagging of preset target phrases on text data and tabular data. It should be noted that text data refers to pure text data, which does not include images, tables, or other types of data formats; tabular data refers to data displayed in tabular form. It is understood that the data to be labeled can also be a combination of text data and tabular data; this embodiment of the application does not specifically limit this.

[0046] Step 102: Calculate the offset based on the entity marker to obtain the offset corresponding to the preset target phrase.

[0047] In practical implementation, offset refers to the relative positional distance of an entity marker with respect to the data to be labeled. It means treating the data to be labeled as a whole and determining the starting position of the pre-defined target phrase corresponding to the entity marker within the data. For example, if the first character of the pre-defined target phrase appears at the position corresponding to the second character in the data to be labeled, starting with an index of 0, then the index of the pre-defined target phrase is 1, and its offset is 1.

[0048] For the entity tags obtained in the above steps, the offset of the preset target phrase corresponding to all entity tags can be calculated in the following way: traverse the preset target phrases after tagging, obtain the index value of each preset target phrase in the entire data to be labeled, and use the index value as the offset of the preset target phrase.

[0049] Step 103: Perform relation matching on the preset target phrase based on the offset and the contextual semantic relationship of the data to be labeled, and obtain the labeling result of the relation matching.

[0050] In the specific implementation process, the preset target phrases obtained in the above steps are matched with the contextual semantic relationships of the data to be labeled to obtain the labeling results. It should be noted that relation matching refers to extracting the preset target phrases and then matching related preset target phrases based on the contextual semantic relationships and their offsets to find implicit relational information in a large amount of data. Semantic relationships refer to the semantic relationships between linguistic units, mainly manifested as vertical aggregation relationships, horizontal combination relationships, and logical relationships. "Context" refers to the contextual information of a word or character that needs to be understood using natural language processing algorithms, and contextual semantic relationship analysis is the processing of context-related situations. It should be noted that natural language processing algorithms can include AST syntax tree analysis, table direction detection, paragraph segmentation, etc., and can be selected according to specific circumstances; this application does not impose specific limitations on this. Finally, based on the matching results, the successfully matched relationships are used as the labeling results. That is, if the preset target phrases to be labeled are "rank" and "company," the labeling result is a correspondence between "rank" and "company."

[0051] In the specific application of bidding projects in the engineering field, if a project requires bidding, many companies can submit bids. The bidding party will ultimately rank the bidding companies based on the bidding results and issue an announcement. In addition to some relevant project information, the announcement will also state the winning bids: Company A is first, Company B is second, and Company C is third. The preset target phrases for entity tagging in this announcement are "first place", "second place", "third place", "Company A", "Company B", and "Company C", and the tagging results are "first place - Company A", "second place - Company B", and "third place - Company C".

[0052] In the technical solution of this application embodiment, named entity tags are applied to preset target phrases in the data to be labeled, and the offset of each entity tag is calculated. Relationship matching of the preset target phrases is then performed based on the offset. In this process, since the acquired data to be labeled may contain a lot of invalid data in addition to the required data, the required data entities are first labeled, and then the relationships between the labeled entities are obtained based on the offset and contextual semantic relationships. This allows for targeted extraction of relationships between entities, thus improving the efficiency of data labeling.

[0053] Based on the above embodiments, named entity recognition and marking are performed on the preset target phrases of the target project included in the data to be annotated, and entity marks corresponding to the preset target phrases are obtained, including: performing word segmentation on the data to be annotated to obtain multiple word segments, and identifying the attribute values of each word segment; if the attribute value is a preset attribute value, determining that the word segment is a preset target phrase; generating an entity mark for the preset target phrase.

[0054] In the specific implementation process, word segmentation is performed on the data to be annotated, and the attribute values of each word segment are identified, and corresponding labels are marked for the required word segments. It should be noted that the attribute value is the word nature of each word segment. For example, xxxx Co., Ltd. is the name of a certain company, and its corresponding attribute value is company name. The preset attribute value is the attribute value that needs to be marked and set in advance, including but not limited to: ranking attribute, company name attribute, performance attribute, prompt word attribute (HIT, WIN), etc. For example, the company name attribute is a preset attribute value. If a company name is traversed, the corresponding span label is marked for it.

[0055] The word segmentation and marking of the data to be annotated are mainly implemented in the following way: First, text parsing is performed on the data to be annotated, and the etree.HTML is used to parse the text to form an Element object, and then the nodes are traversed. Since not all texts are in a complete node, word segmentation and marking need to be performed on the text within each node and the text after the node.

[0056] When performing word segmentation and marking, the "ner / pku" mode of the HanLP model is mainly used to identify nt (company name) in the text. In addition, word segments such as order (ranking), YeJi (performance), HIT (prompt word), and WIN (prompt word) are also customized. For example, for the word segmentation result 'tok / fine' = {list}[('主', ':', '广东', '华盈', '建设', '集团', '有限公司', '成', ':', '浙江', '恒欣', '设计', '集团', '股份', '有限公司']], its recognition result can be expressed as: 'ner / pku' = {list}[('广东华盈建设集团有限公司', 'nt', 2, 7), ('浙江恒欣设计集团股份有限公司', 'nt', 9, 15)]. Among them, "tok / fine" is the word segmentation result, "ner / pku" is the result after marking the word nature, and each element has four values, namely: word segment, word nature, the starting index corresponding to the word segment in "tok / fine", and the ending index corresponding to the word segment in "tok / fine".

[0057] Traverse the results after tagging the词性 of the segmented words, and respectively tag the span tags for the segmented words such as nt, orber, YeJi, HIT, WIN, etc., and assign the corresponding span attributes in the span tags. Table 1 shows the corresponding span attributes of each词性 in the span tag.

[0058] Table 1

[0059]

[0060] Then, according to the下标, combine the tagged text and the unprocessed segmented words in "tok / fine" into a new node text and assign it to the corresponding Element node. Add the attribute m='r' to each span, indicating that the segmented word corresponding to this span tag may be removed and is used for subsequent logical reasoning.

[0061] In addition, in the specific implementation process, there is additional processing for the ranking. Because as long as the top three are recognized, the above-mentioned segmentation model will separately tag "1", "2", "3", "一", "二", "三" as the ranking词性, but there are special cases, such as "一路", "二期", "三室", etc. Therefore, when marking the span tag with the ranking attribute, if the segmented word is "1", "2", "3", "一", "二", "三", and this segmented word is not the last character in the current text, judge the next character in the sentence. If the character is "名", then judge it as a ranking. Finally, convert the Element object with the replaced node text into a string, and then use html.unescape() to encode the string to obtain the tagged text.

[0062] It should be noted that although this application uses span tags for marking, it is only an example to illustrate this solution. However, it should be known that other types of tags can also be used for marking, and this application does not make specific limitations on this.

[0063] Since a sentence or a paragraph in the data to be annotated may contain the required data, but directly marking a sentence or a paragraph will make the marking object inaccurate. Therefore, it is necessary to segment the data to be annotated. In addition, it is also necessary to mark the attribute value of each segmented word, and judge whether it is a preset target phrase according to the attribute value. In this process, judge whether the data is the required data according to the attribute value of each segmented word, so as to剔除 the data that does not need to be analyzed. Therefore, the accuracy of the data to be analyzed is improved, and the efficiency of data annotation is further improved.

[0064] In some embodiments, the preset attribute values ​​include company name attribute and ranking attribute; the preset target phrase is matched for relation based on the offset and the contextual semantic relationship of the data to be labeled, and the labeling result of relation matching is obtained, including: traversing entity tags; if the entity tag contains preset key attribute values, and the entity tag contains ranking attribute and company name attribute, then the target ranking corresponding to the ranking attribute and the target company corresponding to the company name attribute are extracted; the distance between the target ranking and the target company is determined according to the offset of the entity tag; if the target ranking and the target company satisfy the contextual semantic relationship, then the target ranking and the target company with the closest distance are taken as the labeling result of relation matching.

[0065] In the specific implementation process, the main focus is on obtaining the correspondence between rankings and companies. Therefore, it is necessary to extract the ranking from the span tags corresponding to the ranking attribute in the entity tags and the company from the span tags corresponding to the company name attribute. Since there is a lot of unnecessary data in the entity tags, preset key attribute values ​​are used for filtering. These preset key attribute values ​​are included in the preset attribute values ​​and mainly refer to clue words that can express the semantic relationship of the context. These need to be pointed out manually, such as HIT and WIN.

[0066] When traversing entity tags, if the clue words "HIT" and "WIN" are encountered, the focus is on traversing the entity tags following those clue words. If the entity tag contains ranking and company name attributes, the target ranking corresponding to the ranking attribute and the target company corresponding to the company name attribute are extracted. It should be noted that if, after traversing entity tags containing ranking and company name attributes, a span tag containing a performance (YeJi) attribute is subsequently encountered, the entity tags containing ranking and company name attributes preceding the span tag containing the performance attribute need to be removed.

[0067] To facilitate understanding, the specific process of matching target rankings and target companies is as follows: Obtain the entity tags of the span tags following the preset key attribute values ​​HIT and WIN. If there are span tag entity tags containing both ranking and company name attributes, record the span tag containing the preset keyword attribute value as the index of the first span tag. Starting from tag index -1, traverse the subsequent span tags, determining whether the off value of the obtained span tag is less than the off value of the span tag containing the performance attribute value. If the determination is false, and the span tag contains either a ranking or company name attribute, save the target ranking corresponding to the ranking attribute and the target company corresponding to the company name attribute. Finally, determine the distance between the target ranking and the target company based on the offset of the saved entity tags, and use the closest target ranking and target company as the relationship matching label result. In this process, since the positional relationship between the target ranking and the target company may be that the target ranking is on the left and the target company is on the right, or vice versa, the distance determined based on the offset of the target ranking and the target company obtained above is an absolute value. According to Chinese writing conventions, two entities that are relatively close to each other are most likely to be related. Therefore, the target ranking and target company that are closest to each other are taken as the labeling results of the relationship matching.

[0068] The system identifies the company corresponding to the "company name" attribute and the ranking corresponding to the "rank" attribute. When traversing the marked entities, a further filtering is performed based on preset key attribute values. Only entities exhibiting these preset key attribute values ​​are traversed, and only entities exhibiting both the "rank" and "company name" attributes are marked and extracted. Finally, the distance between the extracted target ranking and target company is calculated based on their offsets, and the closest target ranking and target company name are used as the relationship matching result. Therefore, by using preset key attribute values ​​to further filter the marked entity data, invalid data is eliminated, effectively reducing noise and thus improving the efficiency of data annotation.

[0069] In some embodiments, the contextual semantic relationship includes the table semantic relationship; the distance between the target ranking and the target company is determined based on the offset of the entity tag; if the target ranking and the target company satisfy the contextual semantic relationship, the target ranking and the target company with the closest distance are used as the labeling result of the relationship matching, including: if the entity tag exists in the table, the target ranking and the target company in the entity tag are extracted based on the relative position of the entity tag in the table; the labeling result of the relationship matching is obtained based on the minimum distance between the table semantic relationship and the relative position.

[0070] In practical implementation, table semantic relationships refer to the relationships between data in a table, whether it is a horizontal or vertical table. A table orientation detection algorithm can be used to determine whether it is a horizontal or vertical table. If labeled data exists in the table, the labeling results need to be obtained based on the semantic relationships and relative positional distances of the tables. For ease of understanding, horizontal and vertical tables are provided for illustration. As shown in Table 2, to demonstrate the completeness of the project bidding table, in addition to providing information on the winning candidates, the table also provides information such as the project name, bid section number, name of the tendering entity, name of the tendering agency, contact person, and bid opening time. Table 2 is in horizontal table format, meaning the target ranking and target company are arranged left and right. It should be noted that in a horizontal table, the target ranking can be to the left or right of the target company; this application does not impose specific limitations. During traversal, after determining the target ranking, the target companies can be obtained by directly traversing the span tags immediately adjacent to and to the left and right of the target ranking.

[0071] Since Table 2 already demonstrates the completeness of the table, Table 3 only provides information on the winning candidates. Table 3 is in a vertical format, meaning the target ranking and target company are arranged vertically. Similarly, in a vertical table, the target ranking can be above or below the target company; this application does not impose any specific restrictions. When iterating, after determining the target ranking, the target companies can be obtained by directly iterating through the span tags immediately above and below the target ranking.

[0072] It should be noted that, for illustrative purposes, this application only shows some types of tables and some fields and key information in the tables. Similarly, new table types, fields and keyword information can be added, and this application does not specifically limit this.

[0073] Table 2

[0074]

[0075]

[0076] Table 3

[0077]

[0078] Since the data objects for entity labeling may also exist in tables, entity labels are extracted from tables based on their unique characteristics. In this process, corresponding data extraction is performed according to different data formats, further improving the efficiency of data labeling.

[0079] In some embodiments, after performing relation matching on the preset target phrases based on the offset to obtain the annotation results of the relation matching, the method further includes: determining whether the annotation results need to be deduplicated based on the number of sections corresponding to the target project, and if so, deduplicating the annotation results; wherein: the preset conditions include: the number of target companies corresponding to the target ranking in the annotation results is greater than the number of sections.

[0080] In the specific implementation process, the number of bid sections corresponding to the target project refers to the bid section type of the target project. In this embodiment, regular expressions are used to extract bid sections from the entire text. The bid section type of the target project refers to whether the target project contains only one bid section, then the target project is a single bid section; if the target project contains multiple bid sections, then the target project is a multi-bid section.

[0081] Single-section projects typically have only one winning bid. In the bidding results, the target ranking corresponds one-to-one with the target company, i.e., first place - Company A, second place - Company B, third place - Company C. Multi-section projects usually have multiple winning bids. In the bidding results, the target ranking may correspond to multiple target companies, or target companies may correspond to multiple target rankings, i.e., first place - Company A, Company B, Company C. This could mean that in a multi-section project, the first bidder for the first section is Company A, the first bidder for the second section is Company B, and the first bidder for the third section is Company C; or the first bidder for the first section is Company B, the first bidder for the second section is Company A, and the first bidder for the third section is Company C, or any other combination. This application does not impose specific limitations on these combinations. It should be noted that multi-section projects can also be viewed as a single target project containing multiple sub-sections, and each sub-section can be considered a single section. In this case, the target ranking and target company still have a one-to-one correspondence. Therefore, regardless of whether it is a single-section or multi-section project, the number of target companies corresponding to the target ranking is the same as the number of sections.

[0082] In the actual annotation process, all suspicious entities will be annotated. Therefore, there may be a situation where the number of target companies corresponding to the target ranking is greater than the number of segments. In this case, it is necessary to deduplicate the annotation results.

[0083] After obtaining the labeled data, the final bidding results differ depending on whether the project involves multiple bidding sections or single bidding sections. For example, in a single bidding section project, each ranking corresponds to only one company, while in a multi-bidding section project, which includes multiple sub-projects, a single ranking may correspond to multiple companies. Therefore, it is necessary to determine the number of bidding sections in the target project based on the labeled data to decide whether deduplication is required. In this process, deduplication methods are used for different project types to obtain the final labeled results, improving the accuracy of the labeled data and further enhancing the efficiency of data labeling.

[0084] In some embodiments, deduplication of the annotation results includes: calculating the absolute distance between the target ranking and each target company based on the offset of the entity tag corresponding to the target ranking and the offset of the entity tag corresponding to the target company; obtaining the minimum value of the absolute distance; and deduplicating the annotation results that do not meet the relationship matching corresponding to the minimum value.

[0085] In the specific implementation process, the absolute distance between the target ranking and each target company is calculated based on the offset between the target ranking and the entity tag corresponding to the target company, and deduplication is performed based on the magnitude of the absolute distance value.

[0086] In this embodiment, deduplication is mainly achieved through the following methods: The offsets of the successfully matched target rankings and target companies are obtained. If a target ranking matches two target companies simultaneously (e.g., the first-ranked company matches both Company A and Company B), the distance between the target ranking and the two target companies is calculated based on the offsets, and the target company with the smaller distance is selected as the target company matching the target ranking. If a target company matches two target rankings simultaneously (e.g., Company A matches both the first and second-ranked companies), the distance between the target company and the two target rankings is also calculated based on the offsets, and the target ranking with the smaller distance is selected as the target ranking matching the target company. If the target ranking is at the same distance as multiple matched target companies, the target company with the smaller offset is selected as the target company matching the target ranking. Similarly, if the target company is at the same distance as multiple matched target rankings, the target ranking with the smaller offset is selected as the target ranking matching the target company.

[0087] It should be noted that there can be three or more target companies that match the target ranking, or three or more target rankings that match the target company. This application does not make any specific restrictions on this. In addition, the deduplication operation is only performed when the number of target companies corresponding to the target ranking is greater than the number of segments.

[0088] Following Chinese writing conventions, two related entities are likely to be located close to each other in the text. Therefore, by calculating the absolute distance between the target ranking and each target company, the target companies ultimately related to the target ranking are determined. The deduplication process improves the accuracy of the annotation results, thereby further improving the efficiency of data annotation.

[0089] In some embodiments, before obtaining the data to be labeled, performing named entity recognition tagging on the preset target phrase of the target project contained in the data to be labeled, and obtaining the entity tag corresponding to the preset target phrase, the method further includes: obtaining web data, removing the styles, scripts, and annotations of the web data, and merging inline tags and synonyms to obtain the data to be labeled.

[0090] In the specific implementation process, web data published on the internet differs from plain text data. Web data contains a large amount of interfering data, such as JavaScript scripts, style scripts, document source code comments, and lists, tables, and form formats with inference or hint relationships. Therefore, it is necessary to remove invalid interfering data. The specific implementation process is as follows: Read the HTML (XML) data, remove JavaScript scripts, CSS style scripts, and web page comments from the HTML (XML) code, and merge inline tags and synonyms, as these can cause unnecessary interference to subsequent data processing. For data of the form of... <strong> <em> China Railway X Bureau< / em> XXXXX Co., Ltd.< / strong> Embedded inline tags are used to recursively remove them, yielding the text: China Railway X Bureau XXXXX Co., Ltd. For synonym phrases... First winning bid candidate Remove the tags to get the text result: First successful candidate. It should be noted that inline tags are special tags defined in HTML; the space these tags occupy on the page is determined solely by the content. Synonyms refer to complete phrases that are separated by tags.

[0091] Web data is processed to remove meaningless and distracting data, such as styles, scripts, and documentation source code annotations. By processing the raw web data, meaningful data to be labeled is retained, reducing the time spent traversing invalid data and thus further improving the efficiency of data labeling.

[0092] Figure 2 An overview diagram of data processing provided for embodiments of this application; as shown Figure 2 As shown, step 201 is to preprocess the data; step 202 is to perform pre-defined target phrase named entity tagging on the preprocessed data; step 203 is to calculate the entity tag offset of the entity tags obtained in step 202 to obtain the offset of each entity tag; step 204 is to calculate entity relationships based on the offset obtained in step 203 and the contextual semantic relationship; step 205 is to delete the entity relationships that are not valid in step 204 to obtain the final labeled data; step 206 is to export the labeled data obtained in step 205 for model training.

[0093] It should be noted that the specific implementation process of steps 201 to 204 is the same as in the above embodiment, and will not be repeated here. Step 205, deleting invalid entity relationships, is mainly achieved as follows: First, retain the off value of the span tag corresponding to each entity tag and store it in dictionary 1. Then, iterate through the successfully matched tagging results, comparing the off value of the span tag corresponding to each successfully matched entity tag with the off value in dictionary 1. If the comparison is successful, remove the m attribute of the span tag corresponding to that off value.

[0094] Then, iterate through all the `span` tags in the text, checking if each `span` tag has an `m` attribute. If the `m` attribute is missing, remove the `span` tag, keeping only the text corresponding to the `span` tag. After performing this operation on all `span` tags, the final annotation result is obtained, containing only the annotations of the matched targets, and retaining the annotation results that show a one-to-one correspondence between the target ranking and the target company. For example, the first place corresponds to the name of the first-place company.

[0095] Figure 3 An overview diagram of relationship matching for labeled data provided in this application embodiment, such as Figure 3 As shown,

[0096] Step 301: Obtain entity tag data with added span tags. The acquisition process is the same as in the above embodiment and will not be repeated here.

[0097] Step 302 adds the corresponding p and off attributes to the span tags corresponding to the marked data obtained in step 301. Here, p is the table identifier. If p = t, it means that the span tag is in a table. If p = p, it means that the span tag is in a paragraph. The off attribute represents the offset of the text corresponding to the span tag.

[0098] Step 303 iterates through all entity tag data to perform relationship matching, including: step 304 performing relationship matching on entity tags following span tags containing the preset key attribute value HIT; step 305 performing relationship matching on entity tags following span tags containing the preset key attribute value WIN; and step 306 performing relationship matching on entity tags in the table. It should be noted that steps 304, 305, and 306 can be performed simultaneously or sequentially according to a certain priority order; this embodiment does not specifically limit this. The specific relationship matching process is described in the above embodiment and will not be repeated here.

[0099] After the relationship matching steps described above, step 307 obtains the successfully matched annotation results. Step 308 determines whether the number of target companies corresponding to the target ranking in the successfully matched annotation results is greater than the number of segments. If it is less than or equal to the number of segments, proceed to step 309 to save all annotation results; if it is greater, proceed to step 310 to deduplicate the annotation results. The deduplication process is the same as described in the above embodiment and will not be repeated here. Finally, step 311 ends this process. It should be noted that when obtaining the successfully matched annotation results, it is necessary to delete the entity relationships that failed to match. The specific deletion process is the same as described in the above embodiment and will not be repeated here.

[0100] Figure 4 This is a schematic diagram of a data processing device structure provided in an embodiment of this application, such as... Figure 4 As shown, the device includes a marking module 401, a calculation module 402, and a matching module 403, wherein...

[0101] The tagging module 401 is used to acquire data to be labeled, and to perform named entity recognition tagging on the preset target phrases of the target project contained in the data to be labeled, thereby obtaining entity tags corresponding to the preset target phrases; wherein, the data to be labeled is HTML document data; the calculation module 402 is used to perform offset calculation based on the entity tags, thereby obtaining the offset corresponding to the preset target phrase; wherein, the offset refers to the relative position distance of the entity tag relative to the data to be labeled; the matching module 403 is used to perform relationship matching on the preset target phrases based on the offset and the contextual semantic relationship of the data to be labeled, thereby obtaining the labeling result of the relationship matching.

[0102] Based on the above embodiments, the marking module 401 is specifically used for:

[0103] The data to be labeled is segmented into words to obtain multiple words, and the attribute value of each word is identified.

[0104] If the attribute value is a preset attribute value, then the word segmentation is determined to be the preset target phrase;

[0105] Generate entity tags for the preset target phrase.

[0106] Based on the above embodiments, the preset attribute values ​​include company name attributes and ranking attributes, and the matching module 403 is specifically used for:

[0107] Traverse the entity markers;

[0108] If the entity tag contains a preset key attribute value, and the entity tag contains a ranking attribute and a company name attribute, then the target ranking corresponding to the ranking attribute and the target company corresponding to the company name attribute are extracted.

[0109] The distance between the target ranking and the target company is determined based on the offset of the entity tag; if the target ranking and the target company satisfy the contextual semantic relationship, then the target ranking and the target company with the closest distance are taken as the labeling result of the relationship matching.

[0110] Based on the above embodiments, the contextual semantic relationship includes a table semantic relationship; the matching module 403 is specifically used for:

[0111] If the entity tag exists in the table, then extract the target ranking and the target company from the entity tag based on the relative position of the entity tag in the table;

[0112] The labeling result of the relationship matching is obtained based on the minimum distance between the semantic relationship in the table and the relative position.

[0113] Based on the above embodiments, the device further includes:

[0114] The deduplication module is used to determine whether the annotation result needs to be deduplicated based on the number of bidding sections corresponding to the target project and using preset conditions. If it is necessary, the annotation result is deduplicated. The preset conditions include: the number of target companies in the annotation result that correspond to the target ranking is greater than the number of bidding sections.

[0115] Based on the above embodiments, the deduplication module is specifically used for:

[0116] Calculate the absolute distance between the target ranking and each of the target companies based on the offset of the entity tag corresponding to the target ranking and the offset of the entity tag corresponding to the target company;

[0117] Obtain the minimum absolute value of the distance, and remove duplicate annotation results that do not match the relationship corresponding to the minimum value.

[0118] Based on the above embodiments, the device further includes:

[0119] The acquisition module is used to acquire web data, remove styles, scripts, and annotations from the web data, and merge inline tags and synonyms to obtain the data to be labeled.

[0120] The tagging module marks predefined target phrases in the data to be labeled with named entities. The calculation module calculates the offset of each entity tag, and the matching module performs relationship matching of the predefined target phrases based on the offsets. In this process, since the acquired data to be labeled may contain a lot of invalid data in addition to the required data, the necessary data entities are marked first, and then the positional relationships between the marked entities are obtained based on the offsets. This allows for targeted extraction of relationships between entities, thus improving the efficiency of data labeling.

[0121] Figure 5 This is a schematic diagram of the electronic device structure provided in the embodiments of this application, such as... Figure 5 As shown, the electronic device includes a processor 501, a memory 502, and a bus 503; wherein the processor 501 and the memory 502 communicate with each other via the bus 503. The processor 501 is used to call program instructions in the memory 502 to execute the methods provided in the above-described method embodiments.

[0122] Processor 501 can be an integrated circuit chip with signal processing capabilities. The processor 501 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0123] The memory 502 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0124] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the methods provided in the above-described method embodiments, such as: first, acquiring data to be labeled, performing named entity recognition tagging on a preset target phrase of the target project contained in the data to be labeled, and obtaining entity tags corresponding to the preset target phrases; wherein, the data to be labeled is HTML document data; then, performing offset calculation based on the entity tags to obtain the offset corresponding to the preset target phrases; wherein, the offset refers to the relative position distance of the entity tags relative to the data to be labeled; finally, performing relation matching on the preset target phrases based on the offset and the contextual semantic relationship of the data to be labeled, and obtaining the relation matching labeling result.

[0125] This embodiment provides a non-transitory computer-readable storage medium storing computer instructions that cause the computer to execute the methods provided in the above-described method embodiments. For example, the methods include: first, acquiring data to be labeled; performing named entity recognition (NAME) tagging on a preset target phrase of a target project contained in the data to be labeled to obtain entity tags corresponding to the preset target phrases; wherein the data to be labeled is HTML document data; then, performing offset calculation based on the entity tags to obtain the offset corresponding to the preset target phrases; wherein the offset refers to the relative positional distance of the entity tag relative to the data to be labeled; finally, performing relation matching on the preset target phrases based on the offset and the contextual semantic relationship of the data to be labeled to obtain the relation-matched labeling result.

[0126] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0127] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0128] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0129] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0130] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data annotation method for HTML documents in the engineering field, characterized in that, The method includes: Obtain the data to be labeled, and perform named entity recognition tagging on the preset target phrases of the target project contained in the data to be labeled to obtain the entity tags corresponding to the preset target phrases; wherein, the data to be labeled is HTML document data; Offset calculation is performed based on the entity marker to obtain the offset amount corresponding to the preset target phrase; wherein, the offset amount refers to the relative position distance of the entity marker relative to the data to be labeled; Based on the offset and the contextual semantic relationship of the data to be labeled, the preset target phrase is matched for relationship to obtain the labeling result of the relationship matching; The step of performing named entity recognition tagging on the preset target phrases of the target project contained in the data to be labeled, and obtaining the entity tags corresponding to the preset target phrases, includes: The data to be labeled is segmented into words to obtain multiple words, and the attribute value of each word is identified. If the attribute value is a preset attribute value, then the word segmentation is determined to be the preset target phrase; Generate entity tags for the preset target phrase; The preset attribute values ​​include company name attributes and ranking attributes; the step of performing relation matching on the preset target phrase based on the offset and the contextual semantic relationship of the data to be labeled, to obtain the labeling result of the relation matching, includes: Traverse the entity markers; If the entity tag contains a preset key attribute value, and the entity tag contains a ranking attribute and a company name attribute, then the target ranking corresponding to the ranking attribute and the target company corresponding to the company name attribute are extracted. The distance between the target ranking and the target company is determined based on the offset of the entity marker; If the target ranking and the target company satisfy the contextual semantic relationship, then the closest target ranking and the target company are used as the labeling result of the relationship matching.

2. The method according to claim 1, characterized in that, The contextual semantic relationship includes a table semantic relationship; determining the distance between the target ranking and the target company based on the offset of the entity tag; if the target ranking and the target company satisfy the contextual semantic relationship, then using the closest target ranking and the target company as the labeling result of the relationship matching includes: If the entity tag exists in the table, then extract the target ranking and the target company from the entity tag based on the relative position of the entity tag in the table; The labeling result of the relationship matching is obtained based on the minimum distance between the semantic relationship in the table and the relative position.

3. The method according to claim 1, characterized in that, After performing relation matching on the preset target phrase based on the offset to obtain the annotation result of the relation matching, the method further includes: Based on the number of sections corresponding to the target project, a preset condition is used to determine whether the annotation results need to be deduplicated. If so, the annotation results are deduplicated; wherein: The preset conditions include: the number of target companies corresponding to the target ranking in the annotation results is greater than the number of segments.

4. The method according to claim 3, characterized in that, The process of removing duplicates from the labeled results includes: Calculate the absolute distance between the target ranking and each of the target companies based on the offset of the entity tag corresponding to the target ranking and the offset of the entity tag corresponding to the target company; Obtain the minimum absolute value of the distance, and remove duplicate annotation results that do not match the relationship corresponding to the minimum value.

5. The method according to claim 1, characterized in that, Before obtaining the data to be labeled, performing named entity recognition tagging on the preset target phrases of the target project contained in the data to be labeled, and obtaining the entity tags corresponding to the preset target phrases, the method further includes: Obtain web data, remove styles, scripts, and annotations from the web data, and merge inline tags and synonyms to obtain the data to be annotated.

6. A data annotation device for HTML documents in the engineering field, characterized in that, include: The tagging module is used to acquire data to be tagged, perform named entity recognition tagging on the preset target phrases of the target project contained in the data to be tagged, and obtain the entity tags corresponding to the preset target phrases; wherein, the data to be tagged is HTML document data; The calculation module is used to perform offset calculation based on the entity marker to obtain the offset corresponding to the preset target phrase; wherein, the offset refers to the relative position distance of the entity marker relative to the data to be labeled; The matching module is used to perform relation matching on the preset target phrase based on the offset and the contextual semantic relationship of the data to be labeled, and to obtain the labeling result of the relation matching; Specifically, the marking module is used for: The data to be labeled is segmented into words to obtain multiple words, and the attribute value of each word is identified. If the attribute value is a preset attribute value, then the word segmentation is determined to be the preset target phrase; Generate entity tags for the preset target phrase; The preset attribute values ​​include company name attribute and ranking attribute; the matching module is specifically used for: Traverse the entity markers; If the entity tag contains a preset key attribute value, and the entity tag contains a ranking attribute and a company name attribute, then the target ranking corresponding to the ranking attribute and the target company corresponding to the company name attribute are extracted. The distance between the target ranking and the target company is determined based on the offset of the entity marker; If the target ranking and the target company satisfy the contextual semantic relationship, then the closest target ranking and the target company are used as the labeling result of the relationship matching.

7. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the data annotation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the data annotation method as described in any one of claims 1 to 5.

Images