Webpage embedded text labeling method

Abstract

The invention discloses a webpage embedded text labeling method, which comprises the following steps that: the front end of a webpage labeling plug-in uses Javascript and Chrome browser native API (Application Program Interface) to respond to a labeling instruction of a labeling user on an original text through a document object model element and a browser object model element; a webpage labeling plug-in background identifies and labels original text corpora obtained by a front end by using a pre-preinstalled labeling pre-training model in a pre-labeling algorithm engine library, performs data storage by using a distributed storage engine Elasticsearch, and combines and stores a model pre-labeling result and a manual labeling result. According to the method, the input cost of text collection and text cleaning work in the aspects of manpower and time at the present stage is reduced, the text labeling process is simplified, the use threshold of a traditional labeling platform or tool is reduced, and a user can realize data screening and labeling while browsing a webpage.

Description

Technical Field

[0001] The invention relates to the field of artificial intelligence, and in particular to a web page embedded text annotation method. Background Art

[0002] In the current era of rapid development of artificial intelligence and machine learning technology, text annotation plays the most basic and crucial role. Among the three elements of artificial intelligence, data, computing power and algorithms, data is equivalent to the fuel of AI algorithms; and text annotation is equivalent to "feeding" training data to the natural language processing (NLP) deep learning model. By marking the text with features, that is, manually labeling words with parts of speech, purpose, context, emotion and other labels, a standard training data set is formed and injected into the deep learning model to complete NLP tasks such as text classification, entity recognition, relationship extraction, sentiment classification, and event element extraction. At present, for vertical fields, it is often necessary to build professional corpora and NLP deep learning models in their own fields. Therefore, domain-specific text data annotation work is indispensable.

[0003] According to different NLP tasks, text annotation tasks can be divided into entity annotation, relationship annotation, event extraction, text classification, sentiment recognition, etc. Although there are some differences between different annotation tasks, they all roughly follow the process of data collection -> data cleaning -> data annotation -> data quality inspection -> data delivery. Usually, before using the annotation platform to carry out annotation, users need to collect and build a batch of relevant and neat and standardized texts in advance, involving two links: data collection and data cleaning: the former is to use crawler technology to filter and parse the web page text content of interest; the latter is to further organize the collection results, such as removing HTML tags and special characters, deleting stop words, processing punctuation marks, etc. The processed text data can be further imported into the text annotation tool for annotation. Since the quality of the annotated corpus is directly determined by the quality of the data collection and cleaning, the above two links require programmers to spend a lot of time and energy to process them carefully.

[0004] In addition, a good annotation tool can reduce labor costs and improve annotation efficiency for large-scale text annotation tasks. At present, the more common open source text annotation tools include: YEDDA, ​​Brat, Doccano, etc. Although these platforms are aimed at different annotation tasks, they all have certain intelligent features. For example: YEDDA is mainly used to annotate named entities. In addition to supporting shortcuts and command line mode, it also provides suggestions to users through intelligent annotation algorithms; Chinese-Annotator is mainly used for text classification. It carries its own active learning framework and can complete pre-recognition work through online learning; Marktool supports large-scale entity annotation, relationship annotation, event annotation, and text classification. It introduces dictionary matching and regular matching in the annotation process to achieve automatic annotation, and also supports iterative annotation of text. It can be seen that an efficient and intelligent text annotation tool, in addition to having a simple and friendly operation interface, also needs to use intelligent algorithms to reduce repetitive manual annotation operations as much as possible. Summary of the invention

[0005] The purpose of the present invention is to propose a web page embedded text annotation method, aiming to reduce the current investment cost of text collection and text cleaning in terms of manpower and time, simplify the text annotation process, lower the threshold for using traditional annotation platforms or tools, and support users to screen and annotate data while browsing web pages.

[0006] The technical solution to achieve the purpose of the present invention is: a web page embedded text annotation method, comprising the following steps:

[0007] Step 1: Design and implement the front-end function of the webpage annotation plug-in

[0008] The front end of the web page annotation plug-in uses Javascript and Chrome browser native API to respond to the annotation user's annotation instructions on the original text through document object model elements and browser object model elements;

[0009] Step 2: Design and implementation of web page annotation plug-in background

[0010] The web page annotation plug-in backend uses the pre-installed annotation pre-trained model in the pre-annotation algorithm engine library to identify and annotate the original text corpus obtained by the front end, uses the distributed storage engine Elasticsearch to store data, and merges the model pre-annotation results and manual annotation results for storage.

[0011] Further, step 1: design and implement the front-end function of the web page annotation plug-in. The specific method is:

[0012] Step 1.1: Create a labeling device project

[0013] Create a folder and name it after the plugin as the root directory of the web page annotation plugin front-end project. Use a text editor to create a .json file and three .js files, and name them manifest.json, background.js, contentScript.js, and popup.js. Their uses are described as follows:

[0014] manifest.json manifest file: used to configure the plug-in name, version number, permission application, and script file path;

[0015] background.js logic script file: used to process the core logic of the plug-in, including responding to browser events and communicating with backend algorithm services;

[0016] contentScript.js webpage injection file: used to inject custom functions and styles into the current page;

[0017] popup.js plug-in window UI file: used to define the page elements that users see in the plug-in window and call API operations;

[0018] Step 1.2: Obtain and display the web page annotation corpus module

[0019] The web page annotation corpus module provides users with a mouse word marking function to help users directly obtain news corpus content of interest from web pages, making it easier for users to carry out annotation work in the web page annotation plug-in. The specific implementation is as follows:

[0020] Step 1.2.1: Respond to the user's mouse click operation

[0021] In the contentScript.js file, design and bind a "mouseup" event to the document object of the current web page, and then bind two event processing functions, postCorpus() and showCorpus(), to the event:

[0022] postCorpus(): a function to obtain the content of the user's mark;

[0023] showCorpus(): a function to display the content of the user's markup;

[0024] Step 1.2.2: Get the user's word-marking content

[0025] In the postCorpus() function, the browser's native BOMAPI is called, and window.getSelection() obtains the text selected by the user as the original annotated corpus; then, the obtained word-marking content is passed from the contentScript.js script file to the background.js file through the message passing mechanism and saved as the variable selectedStr;

[0026] Step 1.2.3: Display the user-marked content in the plug-in window

[0027] In the function showCorpus(), the original corpus saved by background.js, i.e. variable selectedStr, is obtained through the message passing mechanism and displayed in the plugin window;

[0028] Step 1.3: Start, obtain and render the machine pre-labeling result module

[0029] The machine pre-labeling result module is used to collect the machine pre-labeling results obtained by the backend and render and label them on the frontend. After the user clicks the "Machine Pre-labeling" button in the plug-in window, the backend receives the original labeled corpus sent by the frontend and starts the machine pre-labeling function, calling the pre-installed labeling pre-training model to perform machine pre-labeling on the original text corpus. The specific implementation is as follows:

[0030] Step 1.3.1: Obtain machine pre-labeling results

[0031] Define the "Machine Pre-labeling" button in the plug-in window and bind the click event. In the click event processing function clickPreLabel(), popup.js will send a message to the background.js file. The background.js file initiates an HTTP request and sends the original text corpus as a request parameter to the background. After the background calls the pre-labeling algorithm engine library to complete the machine pre-labeling, the machine pre-labeling results, including the starting position of the entity word and the category label, are returned to the popup.js file.

[0032] Step 1.3.2: Rendering machine pre-marking results

[0033] In the popup.js file, add a span tag to wrap the text in the middle of the starting position of each entity word, add different highlight styles to the span tag of the annotation result according to the annotation category, give the starting position of the entity word and display the tag name above it;

[0034] Step 1.4: Manual annotation module

[0035] When the user disagrees with the machine pre-labeling results, the manual labeling module provides a manual labeling function, which allows the user to correct the labeling results. The correction operation includes the following two operations:

[0036] Step 1.4.1: Delete the error label operation

[0037] The user can delete the wrong label directly in the plug-in window by right-clicking the mouse. This operation will trigger the "mouseup" event defined in the plug-in code and execute the event processing function deleteLabel. In the plug-in window, the highlight style of the annotation result will be cancelled and the latest annotation result will be saved to the server. The specific implementation is as follows:

[0038] Respond to the user's right-click operation: In popup.js, a contextmenu event is bound to the DOM element corresponding to each annotation entity, and the pre-defined event processing function deleteLabel() is bound to the event. When the user moves the mouse to the annotation entity and right-clicks the mouse, the event processing function deleteLabel() is triggered and executed;

[0039] Render and save the latest annotation result: In deleteLabel(), use document.getElementsByClassName to get the current annotation entity, cancel the highlight style of the current annotation result, delete the current annotation result from the annotation result JSON text, and call the background interface to save the latest annotation result in the database;

[0040] Step 1.4.2: Add manual annotation results

[0041] The user needs to mark the word with the mouse in the plug-in window and define the starting position and label type of the marked word. The user needs to hold down the left mouse button and drag the cursor to the end position of the marked word and then release the left mouse button. This operation will trigger the "mouseup" event defined in the plug-in code and execute the event processing function addLabel(). The marking result is rendered in the plug-in window and the latest marking result is saved to the server. The specific implementation is as follows:

[0042] Respond to the user's word-marking operation: In popup.js, bind a mouseup event to the DOM element in popup.html, and bind a predefined event processing function addLabel() to the event;

[0043] Get the label category: In the addLabel() function, control the pop-up of the entity category selection box. When the user selects a category in the entity category selection box and clicks "OK", the label category is saved;

[0044] Render and save the latest annotation results: In the addLabel() function, use the native BOM API, window.getSelection() to obtain the entity word selected by the user and its starting position; add a span tag to wrap the annotation text in the middle of the starting position, and add different highlight styles to the span tag of the annotation result according to the annotation category; at the same time, save the entity word, category information and its starting position in the annotation result JSON file; finally, the front end calls the backend interface to save the annotation result JSON file in the database.

[0045] Further, step 2: design and implementation of the web page annotation plug-in background, specifically:

[0046] (1) Building a data storage module

[0047] The data storage module uses the distributed storage engine Elasticsearch to store data, and merges the model pre-labeling results and the manual labeling results for storage. The results are stored in the mapping table text_label of Elasticsearch. The jsonStr field in the table stores the model prediction results or the manual labeling results. The isMarked field is used to distinguish whether it is the model prediction result or the manual labeling result. If the field is true, it means that it is the manual labeling result, otherwise it is the model prediction result.

[0048] (2) Build a machine pre-marking module

[0049] The machine pre-labeling module uses the pre-installed labeling pre-trained models in the pre-labeling algorithm engine library, including Google's BERT and Baidu's UIE model, to recognize and label the original text corpus obtained by the front end, and identify and label the corresponding named entities;

[0050] (3) Constructing model iteration module

[0051] The model iteration module uses manually annotated data to form a new training data set to iteratively train the pre-annotated pre-trained model pre-installed in the background pre-annotation algorithm engine library. There are two ways to iterate the pre-annotated pre-trained model: one is to iterate the model according to the "number of annotated corpus", and the other is to iterate according to the "user set time".

[0052] Furthermore, when the user requests to call the annotation pre-training model on the front end to pre-mark the web page text, the front-end service will initiate an HTTP request and send the original corpus to the back end through the HTTP request. At this time, the back end calls the standard pre-training model service to perform machine pre-marking on the original text corpus. The results of the machine annotation will be stored in the jsonStr field of the text_label index in the Elasticsearch database, and the isMarked field will be marked as false; when the user finds that the results extracted by the annotation pre-training model do not meet the user's expectations, manual annotation will be initiated, and the results of manual annotation will be passed to the back end through the front end. The back end will update the jsonStr field of the record, set the isMarked flag to true, and update the manual review table, and update the latest record and time to the manual review table.

[0053] Furthermore, when a certain number of manual labeling results have accumulated in the data storage module, the training of the labeling pre-training model is restarted, and the parameters of the labeling pre-training model are updated, and a prompt message is sent to the backend. After receiving the prompt message, the backend clears the records in the manual review table, and re-machine-pre-labels the text data marked as isMarked as false in the text-label index in the Elasticsearch database, and updates the newly generated text labeling results to the jsonStr field of the text. In this way, the background completes a new round of automatic label updates for the corpus in the library based on the labeling pre-training model of a newly formed batch of training data sets; thereafter, when the front-end receives a request from the user to call the machine pre-labeling function, the backend will start the recently updated labeling pre-training model in the pre-labeling algorithm engine library, perform entity recognition on the original text corpus passed by the front end, and return the labeling results to the front end, so that the front end can display the labeling results.

[0054] A webpage embedded text annotation plug-in device implements the webpage embedded text annotation method to achieve webpage embedded text annotation.

[0055] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the web page embedded text annotation method is implemented to realize web page embedded text annotation.

[0056] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the web page embedded text annotation method is implemented to achieve web page embedded text annotation.

[0057] Compared with the prior art, the present invention has the following significant advantages: (1) It supports users to directly annotate text on the front end of a web page, greatly simplifying the annotation process and saving manpower and time for data collection and cleaning during the annotation process; (2) There is no need to reorganize the annotated corpus for the second time, and the annotation results can be directly used to construct or expand the training corpus of the NLP model to participate in tasks such as entity recognition, relationship extraction, and event classification; (3) The device is plug-and-play and has the advantage of easy transplantation; (4) In addition to being used to construct text annotation corpus, the ideas and methods of the device can also be extended to the field of image annotation, and can be used to construct image annotation tools and training data sets for web pages. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 Design schematic diagrams for the front-end and back-end functional modules of the plug-in;

[0059] Figure 2 A flow chart for using the plug-in device;

[0060] Figure 3 A schematic diagram of the results obtained for the original text corpus;

[0061] Figure 4 This is a schematic diagram of the machine pre-marking results;

[0062] Figure 5 This is a schematic diagram of the annotation results after manual review and correction. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] The present invention provides a webpage embedded text annotation method, which is used to identify three major categories (entity category, time category, and number category) and seven minor categories (names, organizations, place names, time, date, currency, and percentage) of named entities in a text to be processed. The specific steps are as follows:

[0065] Step 1: Design and implement the front-end function of the webpage annotation plug-in

[0066] The front end of the web page annotation plug-in mainly uses the methods provided in Javascript and the native API of the Chrome browser to manipulate DOM (Document Object Model) elements and BOM elements (Browser Object Model) to respond to the annotation user's annotation instructions for the original text. The plug-in device not only supports the Chrome browser, but also supports other browsers with Chromium as the core, such as Edge browser and Firefox browser. The implementation of the front-end interactive function of the device is mainly divided into the following four parts:

[0067] Step 1.1: Create a labeling device project. First, create a folder and name it after the plugin as the root directory of the plugin front-end project. In this directory, use a text editor to create a .json file and three .js files, and name them manifest.json, background.js, contentScript.js, and popup.js. The main uses of the four files are described as follows:

[0068] manifest.json manifest file: This file is mainly used to configure the plug-in name, version number, permission application, and script file path;

[0069] background.js logic script file: This file is mainly used to process the core logic of the plug-in, including responding to browser events and communicating with the backend algorithm service;

[0070] contentScript.js webpage injection file: This file is mainly used to inject custom functions and styles into the current page;

[0071] popup.js plug-in window UI file: This file is mainly used to define the page elements that users see in the plug-in window and call API operations. Popup.js is actually the core of the entire plug-in front-end implementation and display.

[0072] Step 1.2: Get and display the webpage annotation corpus module. The webpage annotation corpus module provides users with a mouse word-marking function to help users directly obtain news corpus content of interest from web pages, making it easier for users to further perform annotation work in the annotation plug-in. The specific implementation is as follows:

[0073] Step 1.2.1: Respond to the user's mouse click operation. In the contentScript.js file, first design and bind a "mouseup" event to the document object of the current web page, and then bind two event processing functions, postCorpus() and showCorpus(), to the event:

[0074] postCorpus(): a function to obtain the content of the user's mark;

[0075] showCorpus(): A function that displays the content of the user's highlighted words.

[0076] Step 1.2.2: Get the user's highlighted content. In the postCorpus() function, call the browser's native BOMAPI, window.getSelection() to get the text selected by the user as the original annotated corpus; then, pass the obtained highlighted content (i.e., original text corpus) from the contentScript.js script file to the background.js file through the message passing mechanism and save it as the variable selectedStr.

[0077] Step 1.2.3: Display the user-marked content in the plug-in window. In the function showCorpus(), the original corpus saved in background.js, i.e. the variable selectedStr, is obtained through the message passing mechanism and displayed in the plug-in window.

[0078] Step 1.3: Start, obtain and render the machine pre-marking result module. The machine pre-marking result module is used to collect the machine pre-marking results obtained in the background and render and mark the results in the front end. After the user clicks the "Machine Pre-marking" button in the plug-in window (see Appendix Figure 3 ), the backend receives the original annotated corpus sent by the frontend and starts the machine pre-annotation function, calling the pre-installed annotation pre-training model in the algorithm library to perform machine pre-annotation on the original text corpus to reduce the workload of manual annotation. The specific implementation is as follows:

[0079] Step 1.3.1: Get the machine pre-labeling results. Define the "Machine Pre-labeling" button in the plug-in window and bind the click event. In the click event processing function clickPreLabel(), popup.js will send a message to the background.js file. The background.js file initiates an HTTP request and sends the original text corpus as a request parameter to the background. After the background calls the pre-labeling algorithm engine library to complete the machine pre-labeling, the machine pre-labeling results (for example: the starting position of the entity word, the category label) are returned to the popup.js file.

[0080] Step 1.3.2: Render the machine pre-labeled results. In the popup.js file, add a span tag to wrap the text in the middle of the starting position of each entity word, add different highlight styles to the span tags of the labeling results according to the labeling category, give the starting position of the entity word and display the label name above it.

[0081] Step 1.4: Manual labeling module. When the user disagrees with the machine pre-labeling results, the manual labeling module provides a manual labeling function, which allows the user to correct the labeling results. The correction operation mainly includes the following two operations:

[0082] Step 1.4.1: Delete the error label operation. The user can delete the error label directly in the plug-in window by right-clicking the mouse. This operation will trigger the "mouseup" event defined in the plug-in code and execute the event processing function deleteLabel. In the plug-in window, the highlight style of the annotation result will be cancelled and the latest annotation result will be saved to the server. The specific implementation of this operation is as follows:

[0083] Respond to the user's right-click operation: In popup.js, bind a contextmenu event to the DOM element corresponding to each annotation entity, and bind the pre-defined event processing function deleteLabel() to the event. When the user moves the mouse to the annotation entity and right-clicks the mouse, the event processing function deleteLabel() can be triggered and executed.

[0084] Render and save the latest annotation result: In deleteLabel(), use document.getElementsByClassName to get the current annotation entity, cancel the highlight style of the current annotation result, then delete the current annotation result from the annotation result JSON text, and call the background interface to save the latest annotation result in the database.

[0085] Step 1.4.2: Add manual annotation results. The user needs to use the mouse to mark the word in the plug-in window and define the starting position and label type of the annotated word. The user needs to hold down the left mouse button and drag the cursor to the end position of the annotated word and then release the left mouse button. This operation will trigger the "mouseup" event defined in the plug-in code and execute the event processing function addLabel(), rendering the annotation result in the plug-in window and saving the latest annotation result to the server. The specific implementation of this operation is as follows:

[0086] Respond to the user's word marking operation: In popup.js, bind a mouseup event to the DOM element in popup.html, and bind a pre-defined event processing function addLabel() to the event.

[0087] Get the label category: In the addLabel() function, control the pop-up of the entity category selection box. When the user selects a category in the entity category selection box and clicks "OK", the label category is saved.

[0088] Render and save the latest annotation results: In the addLabel() function, use the native BOM API, window.getSelection() to obtain the entity word selected by the user and its starting position; add a span tag to wrap the annotation text in the middle of the starting position, and add different highlight styles to the span tag of the annotation result according to the annotation category. At the same time, save the entity word, category information and its starting position in the annotation result JSON file. Finally, the front-end calls the back-end interface to save the annotation result JSON file in the database.

[0089] Step 2: Design and implementation of the web page annotation plug-in background. The web page annotation plug-in background mainly provides corresponding algorithm and data storage support, including data storage module and extraction algorithm module.

[0090] Step 2.1: Data storage module. The main function of the data storage module is to store the process data generated before and after the labeling task is executed: one type is the model pre-labeling result (see step 1.3), and the other type is the manual labeling result (see step 1.4). The underlying layer of the data storage module uses the distributed storage engine Elasticsearch to store data, and merges the model pre-labeling results and the manual labeling results for storage. The results are stored in the mapping table text_label of Elasticsearch. The jsonStr field in the table stores the results predicted by the model or the results of manual labeling. The isMarked field is used to distinguish whether it is the result predicted by the model or the result of manual labeling. If this field is true, it means that it is the result of manual labeling, otherwise it is the result predicted by the model.

[0091] Step 2.1.1: Storage of machine pre-labeling results. When the user requests to call the pre-labeling training model to pre-label the webpage text on the front-end, the front-end service will initiate an HTTP request and send the original corpus to the back-end server through the HTTP request. At this time, the back-end service will automatically call the standard pre-training model service to perform machine pre-labeling on the original text corpus. The machine labeling results will be stored in the jsonStr field of the text_label index in the Elasticsearch database, and the isMarked field will be marked as false.

[0092] Step 2.1.2: Manual labeling result storage. When the user finds that the results extracted by the labeling pre-training model do not meet the user's expectations, manual labeling will be initiated (step 1.4). The results of manual labeling are passed to the backend through the frontend, and the backend will update the jsonStr field of the record, set the isMarked flag to true, and update the manual review table, and update the latest record and time to the manual review table.

[0093] Step 2.2: Machine pre-labeling module. The main function of the machine pre-labeling module is to use the pre-installed pre-labeling pre-trained model in the pre-labeling algorithm engine library to identify and label the original text corpus obtained by the front end. The pre-labeling pre-training model is the core of this module. In order to correspond to the named entity recognition labeling task, some pre-labeling pre-training models need to be pre-loaded in the algorithm engine library. At present, the more common domain labeling pre-training models include Google's BERT and Baidu's UIE (Universal Information Extraction) model, which can be used for named entity recognition, text classification, relationship extraction or event element extraction. Users can adjust them according to their actual labeling needs.

[0094] Step 2.3: Model iteration module. The main function of the model iteration module is to use the manually labeled data to form a new training data set to iteratively train the pre-installed labeling pre-training model in the background pre-labeling algorithm engine library. There are two ways to iterate the labeling pre-training model:

[0095] One is to iterate the model according to the "number of annotated corpora", and the other is to iterate according to the "user-set time". The process of the first one is as follows: the backend server will scan the manual review table regularly (such as 24:00). When the number of annotations in the manual review table reaches the set number (such as 200), the background program will push the batch of annotated data to the algorithm service and start the training of the annotated pre-trained model. The second is to iterate the algorithm model regularly according to time, that is, when the user sets the time for algorithm iteration, such as once a month, when the specified time comes, the background program will push the batch of annotated data in the manual review table to the algorithm service and start the training of the annotated pre-trained model. After the above two modes complete the iterative update of the annotation pre-training model and its parameters, a prompt message will be sent to the backend service. After receiving the prompt message, the backend service will clear the records in the manual review table, and re-machine pre-mark the text data marked as isMarked as false in the text-label index in the Elasticsearch database, and update the newly generated text annotation results to the jsonStr field of the text. In this way, the background completes a new round of automatic label updates for the corpus in the library based on the annotation pre-training model of the newly formed batch of training data sets. Afterwards, when the front-end receives a request from the user to call the machine pre-labeling function, the back-end will start the recently updated annotation pre-training model in the pre-labeling algorithm engine library, perform entity recognition on the original text corpus passed by the front-end, and return the annotation results to the front-end, allowing the front-end to display the annotation results.

[0096] In summary, an active learning framework is introduced on the basis of manual labeling, and intelligent algorithms are used to assist manual discrimination, converting users from labelers to label reviewers to speed up the labeling process. The results of labeling can expand the training corpus of the model, so that it can continuously optimize the training parameters of the model driven by high-quality data, thereby continuously improving the model's discrimination accuracy and effectively reducing manual input.

[0097] Example

[0098] Taking the content mentioned in a science and technology news published by Xinhuanet in November 2024, "Recently, Company C obtained a credit line of US$650 million from several investment banks. Prior to the credit arrangement, Company C raised US$12.7 billion in equity and debt financing in the past 12 months, including US$7.5 billion in debt financing in May. It also raised a US$2.3 billion loan from some investors in August 2023 - using Company M's artificial intelligence chip as collateral. Company C, headquartered in New Jersey, Country X, was founded in 2017. The company plans to have 28 data centers by the end of the year." as an example, the entity recognition and labeling steps of the embodiment of the present invention are described in detail. Entity recognition is to identify and classify the named entities in the labeled sample sentences according to proper nouns such as personnel, organizations, place names, time expressions, and quantifiers.

[0099] Step 1: Install and start the plug-in. Before using this plug-in device for text annotation, users need to start the plug-in first. The specific steps are as follows: First, open the Edge or Chrome browser and enter edge: / / extensions in the address bar to enter the extension page; then, find the "Developer Mode" button on the page and turn it on; then, find the "Load unpacked extension" button on the page and click it. In the pop-up file selection window, navigate to the plug-in project folder.

[0100] Step 2: Obtain web page annotated corpus. When a user finds a news topic or content that he is interested in on a news web page, he can use the mouse to open the news link and view the news details, and use the mouse to mark the part of the news that he is interested in. The front end will display the captured text content in the plug-in window through the message transmission mechanism.

[0101] Step 3: Machine pre-marking. The user clicks the "Machine pre-marking" button in the plug-in window to obtain the machine pre-marking results and display them in the plug-in window.

[0102] like Figure 4As shown in the figure, the intelligent algorithm only recognizes two types of entities from the original corpus, namely: "Bank" (organization), "M Company" (organization) and "X Country" (place name). The front end highlights the two entities in different colors and displays the starting position and label name of the entity word in the form of "brackets + labels" above the entity word.

[0103] Step 4: Manual labeling: Obviously, the labeling pre-training model is not accurate enough for the recognition of certain proper nouns in the above examples, and needs to be manually reviewed and corrected. Figure 5 The results after manual annotation are shown in Figure 2. The results of machine pre-annotation can generally be summarized into the following three problems:

[0104] Annotation errors: For example, the machine pre-annotation annotates the word "bank" in the original text corpus as "organization", but this word is a general entity rather than a proper noun entity and does not need to be specially annotated in the text;

[0105] Incomplete annotation: For example, the machine pre-annotates "X country" in the original text corpus as "place", but does not annotate "Xinzhou" which is also a "place", which needs to be corrected manually; the user can first place the mouse on the original annotated word and right-click to delete the annotated word, and then use the mouse word-tracing function to determine the start and end positions of the annotated entity and the corresponding entity type;

[0106] Missing annotation: The quality of machine pre-annotation results depends on the quality and type of data used to train the pre-annotation model, such as Figure 5 As shown in the figure, the machine pre-labeling failed to recognize the English organization "Company C", time (such as "May", "2017") and quantifiers (such as "650 million / 127 / 750 million", "28"), and they must be supplemented by manual labeling; users can use the mouse word marking function to determine the start and end positions of the labeled entity and the corresponding entity type; if the corresponding entity type is not included in the "Select Category", users can click the "+" button in the lower right corner of the plug-in window to add it.

[0107] Step 5: Store manual annotation results. When manual annotation is completed, the front-end will call the storage interface of the back-end to store the manual annotation results in the Elasticsearch storage engine of the data storage module, overwriting the previous machine pre-annotation results, and set the data flag position to true, indicating that the data has been manually corrected and can be used as standard corpus for iterative training of algorithm models. All annotation results will be exported in JSON format and saved to the training data folder "… / Trainingdata / sample.json" under the installation path. The sample format of the exported JSON file is as follows:

[0108]

[0109]

[0110] Among them, the 'text' field represents the annotated text content; the 'category' field corresponds to the category to which the current entity belongs; the 'startindex' field corresponds to the starting position of the annotated entity in the current sentence, counting from 0; the 'endindex' field corresponds to the ending position of the annotated entity in the current sentence.

[0111] Step 6: Model iteration. When the number of labeled data reaches a certain amount (such as 200; the number can be set in the background program), the user can restart the model training program; the user can also directly set the model retraining time in the background program (such as: December 31, 2024 00:00:00). The model parameter results after training will be saved in the background along with the new training corpus.

[0112] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0113] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A web page embedded text annotation method, characterized in that: The steps include: Step 1: Design and implement the front-end function of the web page annotation plug-in The front end of the web page annotation plug-in uses Javascript and Chrome browser native API to respond to the annotation user's annotation instructions on the original text through document object model elements and browser object model elements; Step 2: Design and implementation of web page annotation plug-in background The web page annotation plug-in backend uses the pre-installed annotation pre-trained model in the pre-annotation algorithm engine library to identify and annotate the original text corpus obtained by the front end, uses the distributed storage engine Elasticsearch to store data, and merges the model pre-annotation results and manual annotation results for storage.

2. The webpage embedded text annotation method according to claim 1, characterized in that: Identify and annotate named entities, including people, organizations, place names, time expressions, and quantifiers.

3. The webpage embedded text annotation method according to claim 1, characterized in that: Step 1: Design and implement the front-end function of the web page annotation plug-in. The specific method is as follows: Step 1.1: Create a labeling device project Create a folder and name it after the plugin as the root directory of the web page annotation plugin front-end project. Use a text editor to create a .json file and three .js files, and name them manifest.json, background.js, contentScript.js, and popup.js. Their uses are described as follows: manifest.json manifest file: used to configure the plug-in name, version number, permission application, and script file path; background.js logic script file: used to process the core logic of the plug-in, including responding to browser events and communicating with the backend algorithm service; contentScript.js webpage injection file: used to inject custom functions and styles into the current page; popup.js plug-in window UI file: used to define the page elements that users see in the plug-in window and call API operations; Step 1.2: Obtain and display the web page annotation corpus module The web page annotation corpus module provides users with a mouse word marking function to help users directly obtain news corpus content of interest from web pages, making it easier for users to carry out annotation work in the web page annotation plug-in. The specific implementation is as follows: Step 1.2.1: Respond to the user's mouse click operation In the contentScript.js file, design and bind a "mouseup" event to the document object of the current web page, and then bind two event processing functions, postCorpus() and showCorpus(), to the event: postCorpus(): a function to obtain the content of the user's mark; showCorpus(): a function to display the content of the user's markup; Step 1.2.2: Get the user's word-marking content In the postCorpus() function, the browser's native BOMAPI is called, and window.getSelection() obtains the text selected by the user as the original annotated corpus; then, the obtained word-marking content is passed from the contentScript.js script file to the background.js file through the message passing mechanism and saved as the variable selectedStr; Step 1.2.3: Display the user-marked content in the plug-in window In the function showCorpus(), the original corpus saved by background.js, i.e. variable selectedStr, is obtained through the message passing mechanism and displayed in the plug-in window; Step 1.3: Start, obtain and render the machine pre-labeling result module The machine pre-labeling result module is used to collect the machine pre-labeling results obtained by the backend and render and label them on the frontend. After the user clicks the "Machine Pre-labeling" button in the plug-in window, the backend receives the original labeled corpus sent by the frontend and starts the machine pre-labeling function, calling the pre-installed labeling pre-training model to perform machine pre-labeling on the original text corpus. The specific implementation is as follows: Step 1.3.1: Obtain machine pre-labeling results Define the "Machine Pre-labeling" button in the plug-in window and bind the click event. In the click event processing function clickPreLabel(), popup.js will send a message to the background.js file. The background.js file initiates an HTTP request and sends the original text corpus as a request parameter to the background. After the background calls the pre-labeling algorithm engine library to complete the machine pre-labeling, the machine pre-labeling results, including the starting position of the entity word and the category label, are returned to the popup.js file. Step 1.3.2: Rendering machine pre-marking results In the popup.js file, add a span tag to wrap the text in the middle of the starting position of each entity word, add different highlight styles to the span tag of the annotation result according to the annotation category, give the starting position of the entity word and display the tag name above it; Step 1.4: Manual annotation module When the user disagrees with the machine pre-labeling results, the manual labeling module provides a manual labeling function, which allows the user to correct the labeling results. The correction operation includes the following two operations: Step 1.4.1: Delete the error label operation The user can delete the wrong label directly in the plug-in window by right-clicking the mouse. This operation will trigger the "mouseup" event defined in the plug-in code and execute the event processing function deleteLabel. The highlight style of the annotation result will be cancelled in the plug-in window, and the latest annotation result will be saved to the server. The specific implementation is as follows: Respond to the user's right-click operation: In popup.js, a contextmenu event is bound to the DOM element corresponding to each annotation entity, and the pre-defined event processing function deleteLabel() is bound to the event. When the user moves the mouse to the annotation entity and right-clicks the mouse, the event processing function deleteLabel() is triggered and executed; Render and save the latest annotation result: In deleteLabel(), use document.getElementsByClassName to get the current annotation entity, cancel the highlight style of the current annotation result, delete the current annotation result from the annotation result JSON text, and call the background interface to save the latest annotation result in the database; Step 1.4.2: Add manual annotation results The user needs to mark words with the mouse in the plug-in window and define the starting position and label type of the marked word. The user needs to hold down the left mouse button and drag the cursor to the end position of the marked word and then release the left mouse button. This operation will trigger the "mouseup" event defined in the plug-in code and execute the event processing function addLabel(). The marking result is rendered in the plug-in window and the latest marking result is saved to the server. The specific implementation is as follows: Respond to the user's word-marking operation: In popup.js, bind a mouseup event to the DOM element in popup.html, and bind a predefined event processing function addLabel() to the event; Get the label category: In the addLabel() function, control the pop-up of the entity category selection box. When the user selects a category in the entity category selection box and clicks "OK", the label category is saved; Render and save the latest annotation results: In the addLabel() function, use the native BOM API, window.getSelection() to obtain the entity word selected by the user and its starting position; add a span tag to wrap the annotation text in the middle of the starting position, and add different highlight styles to the span tag of the annotation result according to the annotation category; at the same time, save the entity word, category information and its starting position in the annotation result JSON file; finally, the front end calls the backend interface to save the annotation result JSON file in the database.

4. The webpage embedded text annotation method according to claim 3, characterized in that: Step 2: Design and implementation of the web page annotation plug-in background, specifically: (1) Building a data storage module The data storage module uses the distributed storage engine Elasticsearch to store data, and merges the model pre-labeling results and the manual labeling results for storage. The results are stored in the mapping table text_label of Elasticsearch. The jsonStr field in the table stores the model prediction results or the manual labeling results. The isMarked field is used to distinguish whether it is the model prediction result or the manual labeling result. If the field is true, it means that it is the manual labeling result, otherwise it is the model prediction result. (2) Build a machine pre-marking module The machine pre-labeling module uses the pre-installed labeling pre-trained models in the pre-labeling algorithm engine library, including Google's BERT and Baidu's UIE model, to recognize and label the original text corpus obtained by the front end, and identify and label the corresponding named entities; (3) Constructing model iteration module The model iteration module uses manually annotated data to form a new training data set to iteratively train the pre-annotated pre-trained model pre-installed in the background pre-annotation algorithm engine library. There are two ways to iterate the pre-annotated pre-trained model: one is to iterate the model according to the "number of annotated corpora", and the other is to iterate according to the "user-set time".

5. The webpage embedded text annotation method according to claim 4, characterized in that: When the user requests to call the annotation pre-training model to pre-label the web page text on the front-end, the front-end service will initiate an HTTP request and send the original corpus to the back-end through the HTTP request. At this time, the back-end calls the standard pre-training model service to perform machine pre-labeling on the original text corpus. The machine annotation results will be stored in the jsonStr field of the text_label index in the Elasticsearch database, and the isMarked field will be marked as false; when the user finds that the results extracted by the annotation pre-training model do not meet the user's expectations, manual annotation will be initiated. The results of manual annotation will be passed to the back-end through the front-end. The back-end will update the jsonStr field of the record, set the isMarked flag to true, and update the manual review table, and update the latest record and time to the manual review table.

6. The webpage embedded text annotation method according to claim 4, characterized in that: When a certain number of manual labeling results have accumulated in the data storage module, the training of the labeling pre-training model is restarted, and the parameters of the labeling pre-training model are updated, and a prompt message is sent to the backend. After receiving the prompt message, the backend clears the records in the manual review table, and re-machine pre-labels the text data marked as isMarked as false in the text-label index in the Elasticsearch database, and updates the newly generated text labeling results to the jsonStr field of the text. In this way, the backend completes a new round of automatic label updates for the corpus in the library based on the labeling pre-training model of a newly formed batch of training data sets; afterwards, when the front-end receives a request from the user to call the machine pre-labeling function, the backend will start the recently updated labeling pre-training model in the pre-labeling algorithm engine library, perform entity recognition on the original text corpus passed by the front end, and return the labeling results to the front end, allowing the front end to display the labeling results.

7. A web page embedded text annotation plug-in device, characterized in that: Implement the web page embedded text annotation method described in any one of claims 1-6 to achieve web page embedded text annotation.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the web page embedded text annotation method according to any one of claims 1 to 6 is implemented to realize web page embedded text annotation.

9. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the web page embedded text annotation method according to any one of claims 1 to 6 is implemented to achieve web page embedded text annotation.

Images