Static webpage labeling method and device and storage medium
By separating the main process from the rendering process and using an inter-process communication mechanism, the version control conflict problem of static web page annotation in a collaborative work environment is solved, achieving zero-pollution annotation and accurate positioning, and improving code maintainability and collaborative compatibility.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XIAMEN MEIYABAIKE INFORMATION SECURITY RES INST CO LTD
- Filing Date
- 2025-12-04
- Publication Date
- 2026-05-05
AI Technical Summary
Existing static webpage annotation technology is prone to causing conflicts with version control systems in collaborative work environments, damaging document structure, affecting code purity and maintainability, and resulting in poor collaborative compatibility.
The main process and rendering process are separated. The annotation elements generated by user action events are serialized into structured data and persistently stored through inter-process communication mechanism, maintaining physical isolation from the source file. The annotation elements are displayed when the webpage is loaded, and the position of the annotation elements is dynamically calculated to adapt to layout changes.
It enables annotation without polluting static webpage source files, reduces version control system conflicts, improves code maintainability and collaborative compatibility, and ensures that annotated elements maintain accurate positioning when the layout changes.
Smart Images
Figure CN121979423A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer web front-end engineering technology, specifically to a method, apparatus, and storage medium for annotating static web pages. Background Technology
[0002] With the rapid development of web technology and the widespread adoption of digital content, static web pages have been widely used in various fields such as education, design, online review, software testing, and electronic data forensics. In these application scenarios, users often need to interactively annotate web page content to complete operations such as commenting, annotation, highlighting key points, or providing feedback.
[0003] Currently, the mainstream implementation of static webpage annotation is inline annotation. This technology dynamically inserts DOM elements or directly modifies the existing node structure using JavaScript to display annotations. Its core idea is: when a user performs an annotation operation, JavaScript locates the target position in the webpage's HTML structure and inserts a new annotation element (such as a highlighting element). This involves adding tags or directly modifying the style attributes of existing elements. While simple to implement, this method has a fundamental flaw: it directly alters the original HTML document structure, leading to code redundancy and maintenance difficulties. Especially in collaborative work environments, DOM modifications caused by annotations are highly likely to conflict with business code changes in the version control system, increasing the complexity of merging and maintenance. Imagine two developers, A and B, working on the same original HTML file. A adds some annotations to the file, while B simultaneously modifies some business logic. When they attempt to merge the code, the version control system will detect modifications in different parts of the same file, easily causing conflicts. These conflicts arise from annotation modifications unrelated to core functionality, making resolution tedious and error-prone.
[0004] In summary, existing annotation technologies have the following limitations: they disrupt document structure; inline annotations pollute the original HTML, affecting code purity and maintainability; and they have poor collaborative compatibility: annotation data is coupled with the page structure, easily causing conflicts in version control and team collaboration. Therefore, there is an urgent need for a static webpage annotation solution that can maintain zero pollution of the original webpage. Summary of the Invention
[0005] To perform static webpage annotation while maintaining zero contamination of the original webpage, in a first aspect, embodiments of this application provide a method for annotating static webpages, the method comprising: A browser window is created by the main process, and a rendering process loads and displays the static webpage to be annotated in the browser window. The static webpage to be annotated includes annotation tools. The rendering process listens for user action events, which are interactive behaviors on the original DOM elements using the annotation tool. The annotation elements generated by the user action event are sent to the main process through an inter-process communication mechanism, so that the main process can serialize the annotation elements to obtain structured annotation data and persistently store the structured annotation data locally. The structured annotation data is physically isolated from the source file of the static webpage to be annotated. When the main process listens to the rendering process's webpage loading completion event, it reads the annotation data from the local machine and sends the annotation data to the rendering process through the inter-process communication mechanism, so that the rendering process can display the annotation elements at the corresponding positions on the static webpage according to the received annotation data.
[0006] In one possible implementation, when the main process listens to the rendering process's webpage loading completion event, it reads annotation data from the local machine and sends the annotation data to the rendering process via an inter-process communication mechanism, so that the rendering process displays annotation elements at the corresponding positions on the static webpage based on the received annotation data, including: When the main process listens to the rendering process's webpage loading completion event, it reads the annotation data from the local machine and sends the annotation data to the rendering process through the inter-process communication mechanism; When the rendering process listens to the annotation data sent by the main process, it parses the annotation data, determines the original DOM element to be annotated through the CSS selector, obtains the position and size of the original DOM element to be annotated, creates an annotation element corresponding to the annotation data, adds the annotation element to the transparent annotation layer overlaid on the static webpage, calculates the display position of the annotation element based on the position and size of the original DOM element to be annotated, and displays the annotation element at the display position.
[0007] In one possible implementation, the method further includes: The rendering process maintains a mapping relationship including annotation elements, CSS selectors, and annotation data IDs. When the rendering process detects a layout change event, it obtains all the annotation data of the current webpage, sorts all the annotation data according to spatial location, and obtains a queue to be scanned. The queue to be scanned prioritizes the annotation data within the current viewport. The original DOM elements corresponding to the labeled data in the queue to be scanned are determined by using CSS selectors; Remeasure the position and size information of the original DOM element in the current viewport; Find the corresponding annotation element using the annotation data ID; Based on the position and size information, the updated position information of the labeled element is dynamically calculated, and the labeled element is overlaid on the original DOM element according to the updated position information.
[0008] In one possible implementation, the step of dynamically calculating the updated position information of the labeled element based on position information and size information, and then overlaying the labeled element onto the original DOM element according to the updated position information, includes: If the annotation element is a highlighted text element, it will completely cover the original DOM element; if the annotation element is a comment bubble for text, it will maintain the relative position of the annotation element and the original DOM element; if the annotation element is an arrow for text, it will point from the updated starting point to the original DOM element.
[0009] In one possible implementation, the method further includes: Determine whether the updated position information of the labeled element is within the range of change; If not, obtain the update location information and annotation data ID of the labeled element, and generate an update queue; The update queue is sent to the main process via an inter-process communication mechanism; The main process reads the currently stored annotation data file and updates the location and version information of the corresponding annotation data according to the annotation data ID in the update queue using an incremental update strategy.
[0010] Secondly, embodiments of this application provide a labeling device for static web pages, the device comprising: A static webpage loading and display module is used to create a browser window through the main process, and load and display the static webpage to be annotated on the browser window through the rendering process. The static webpage to be annotated includes annotation tools. The user action event listening module is used to listen for user action events through the rendering process. The user action events are the interactive behaviors of the original DOM elements using the annotation tool. The annotation information persistence module is used to send the annotation elements generated by the user action event to the main process through the inter-process communication mechanism, so that the main process can serialize the annotation elements to obtain structured annotation data and persistently store the structured annotation data locally. The structured annotation data is physically isolated from the source file of the static webpage to be annotated. The annotation information display module is used to read annotation data from the local machine and send the annotation data to the rendering process through an inter-process communication mechanism when the main process listens to the webpage loading completion event of the rendering process, so that the rendering process can display annotation elements at the corresponding positions on the static webpage according to the received annotation data.
[0011] In one possible implementation, the annotation information display module is specifically used for: When the main process listens to the rendering process's webpage loading completion event, it reads the annotation data from the local machine and sends the annotation data to the rendering process through the inter-process communication mechanism; When the rendering process listens to the annotation data sent by the main process, it parses the annotation data, determines the original DOM element to be annotated through the CSS selector, obtains the position and size of the original DOM element to be annotated, creates an annotation element corresponding to the annotation data, adds the annotation element to the transparent annotation layer overlaid on the static webpage, calculates the display position of the annotation element based on the position and size of the original DOM element to be annotated, and displays the annotation element at the display position.
[0012] In one possible implementation, the device further includes: The mapping module is used to maintain the mapping relationship between annotation elements, CSS selectors, and annotation data IDs through the rendering process; The sorting module is used to obtain all the annotation data of the current webpage when the rendering process listens for layout change events, sort all the annotation data according to spatial position, and obtain a queue to be scanned. The queue to be scanned prioritizes the annotation data in the current viewport. The original DOM element determination module is used to determine the original DOM elements corresponding to the labeled data in the queue to be scanned by using CSS selectors; The retest module is used to remeasure the position and size information of the original DOM element in the current viewport; The annotation element determination module is used to find the corresponding annotation element through the annotation data ID; The update position information determination module is used to dynamically calculate the update position information of the labeled element based on the position information and size information, and then overlay the labeled element onto the original DOM element according to the update position information.
[0013] In one possible implementation, the device further includes: The judgment module is used to determine whether the updated position information of the labeled element is within the range of change; The update queue generation module is used to, if not, obtain the update position information and annotation data ID of the labeled element and generate an update queue; An update queue sending module is used to send the update queue to the main process via an inter-process communication mechanism. The update module is used to read the currently stored annotation data file through the main process, and update the position information and version information of the corresponding annotation data according to the annotation data ID in the update queue using an incremental update strategy.
[0014] Thirdly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned methods for annotating static web pages.
[0015] The static webpage annotation method provided in this application involves creating a browser window through a main process, loading and displaying the static webpage to be annotated in the browser window through a rendering process, and the static webpage to be annotated including annotation tools. The rendering process listens for user action events, which are interactions with the original DOM elements using the annotation tools. Annotation elements generated by the user action events are sent to the main process via an inter-process communication mechanism, allowing the main process to serialize the annotation elements to obtain structured annotation data and persistently store the structured annotation data locally. This structured annotation data is physically isolated from the source file of the static webpage to be annotated. When the main process detects a webpage loading completion event from the rendering process, it reads the annotation data from the local machine and sends it to the rendering process via the inter-process communication mechanism, enabling the rendering process to display the annotation elements at the corresponding positions on the static webpage based on the received annotation data. This method enables annotation and display of annotations without any contamination of the static webpage source file. Attached Figure Description
[0016] Figure 1 This is a schematic diagram of the first flowchart of the static webpage annotation method provided in the embodiments of this application; Figure 2 This is a schematic diagram of a second process for the static webpage annotation method provided in the embodiments of this application; Figure 3 This application provides a schematic diagram of a process for displaying historical annotations. Figure 4 This is a schematic diagram illustrating a result of applying the static webpage annotation method provided in an embodiment of this application. Detailed Implementation
[0017] The present invention will be described in detail below through embodiments.
[0018] With the rapid development of web technology and the widespread adoption of digital content, static web pages have been widely used in various fields such as education, design, online review, software testing, and electronic data forensics. In these application scenarios, users often need to interactively annotate web page content to complete operations such as commenting, annotation, highlighting key points, or providing feedback.
[0019] Currently, the mainstream implementation of static webpage annotation is inline annotation. This technology dynamically inserts DOM elements or directly modifies the existing node structure using JavaScript to display annotations. Its core idea is: when a user performs an annotation operation, JavaScript locates the target position in the webpage's HTML structure and inserts a new annotation element (such as a highlighting element). This involves adding tags or directly modifying the style attributes of existing elements. While simple to implement, this method has a fundamental flaw: it directly alters the original HTML document structure, leading to code redundancy and maintenance difficulties. Especially in collaborative work environments, DOM modifications caused by annotations are highly likely to conflict with business code changes in the version control system, increasing the complexity of merging and maintenance. Imagine two developers, A and B, working on the same original HTML file. A adds some annotations to the file, while B simultaneously modifies some business logic. When they attempt to merge the code, the version control system will detect modifications in different parts of the same file, easily causing conflicts. These conflicts arise from annotation modifications unrelated to core functionality, making resolution tedious and error-prone.
[0020] In summary, existing annotation technologies have the following limitations: they disrupt document structure; inline annotations pollute the original HTML, affecting code purity and maintainability; and they have poor collaborative compatibility: annotation data is coupled with the page structure, easily causing conflicts in version control and team collaboration. Therefore, there is an urgent need for a static webpage annotation solution that can maintain zero pollution of the original webpage.
[0021] Firstly, see [the following] Figure 1 This application provides a method for annotating static web pages, the method comprising: S101, a browser window is created by the main process, and the static webpage to be annotated is loaded and displayed on the browser window by the rendering process. The static webpage to be annotated includes annotation tools.
[0022] This application employs an Electron-based model that separates the main process from the rendering process. The main process runs in a Node.js environment, responsible for scheduling, serializing, and persistently storing annotation data, and exchanging data with the rendering process. This ensures asynchronous separation between interface operations and data read / write, improving response speed and system smoothness. The rendering process runs in the Chromium rendering engine, handling webpage content rendering and user interaction event responses, and is responsible for drawing the annotation UI and providing visual feedback. Each browser window corresponds to an independent rendering process, which can be understood as an independent environment similar to a browser tab. The key difference is that each window runs in its own separate process, rather than sharing the same browser process. Annotation tools include highlight layers, arrows, comment bubbles, and underlines. The rendering process is responsible for HTML (Hypertext Markup Language) parsing and DOM (Document Object Model) tree construction, CSS (Cascading Style Sheets) parsing and CSSOM (Cascading Style Sheets Object Model) tree construction, and JavaScript parsing and execution.
[0023] S102, listen for user action events through the rendering process, wherein the user action events are the interaction behaviors of the original DOM elements using the annotation tool.
[0024] The raw DOM elements refer to each basic unit directly defined by the HTML source code of a webpage and constituting the initial structure and content of the webpage. The rendering process also captures interactive behaviors such as mouse clicks, drags, and keyboard input. For example, when a user draws a label or enters comment text, these operations are first captured and processed by the JavaScript logic in the rendering process.
[0025] S103, the annotation elements generated by the user action event are sent to the main process through the inter-process communication mechanism, so that the main process can serialize the annotation elements to obtain structured annotation data and persistently store the structured annotation data locally. The structured annotation data is physically isolated from the source file of the static webpage to be annotated.
[0026] The annotation element is a separate layer floating on the original webpage. The main process serializes the annotation element, for example, to obtain the JSON string {selector: "#id", position: {x, y}, content: "annotation text"}.
[0027] S104, when the main process listens to the webpage loading completion event of the rendering process, it reads the annotation data from the local machine and sends the annotation data to the rendering process through the inter-process communication mechanism, so that the rendering process displays the annotation elements at the corresponding positions of the static webpage according to the received annotation data.
[0028] In one example, when the main process listens to the rendering process's webpage loading completion event, it reads the annotation data from the local machine and sends the annotation data to the rendering process through an inter-process communication mechanism; When the main process listens to the rendering process's webpage loading completion event, it reads the annotation data from the local machine and sends the annotation data to the rendering process through the inter-process communication mechanism; When the rendering process listens to the annotation data sent by the main process, it parses the annotation data, determines the original DOM element to be annotated through the CSS selector, obtains the position and size of the original DOM element to be annotated, creates an annotation element corresponding to the annotation data, adds the annotation element to the transparent annotation layer overlaid on the static webpage, calculates the display position of the annotation element based on the position and size of the original DOM element to be annotated, and displays the annotation element at the display position.
[0029] In conventional understanding, once a user adds a annotation, the added annotation will be displayed on the current page. However, in the case of reopening a webpage, it is necessary to trigger an event to display historical annotations. In this application, this triggering event is the webpage loading completion event.
[0030] In this embodiment of the application, the display of historical annotations will be automatically triggered after the webpage is loaded, which can display historical annotations in a timely and accurate manner.
[0031] The static webpage annotation method provided in this application involves creating a browser window through a main process, loading and displaying the static webpage to be annotated in the browser window through a rendering process, and the static webpage to be annotated including annotation tools. The rendering process listens for user action events, which are interactions with the original DOM elements using the annotation tools. Annotation elements generated by the user action events are sent to the main process via an inter-process communication mechanism, allowing the main process to serialize the annotation elements to obtain structured annotation data and persistently store the structured annotation data locally. This structured annotation data is physically isolated from the source file of the static webpage to be annotated. When the main process detects a webpage loading completion event from the rendering process, it reads the annotation data from the local machine and sends it to the rendering process via the inter-process communication mechanism, enabling the rendering process to display the annotation elements at the corresponding positions on the static webpage based on the received annotation data. This method enables annotation and display of annotations without any contamination of the static webpage source file.
[0032] In one example, the method further includes: S201 maintains the mapping relationship between annotation elements, CSS selectors, and annotation data IDs through the rendering process.
[0033] S202, when the rendering process listens to the layout change event, it obtains all the annotation data of the current webpage, sorts all the annotation data according to the spatial position, and obtains the scanning queue. The scanning queue prioritizes scanning the annotation data in the current viewport.
[0034] Layout change events include window resize and scroll events, as well as CSS media query change events (CSS media queries allow developers to apply different CSS styles based on different device characteristics such as screen size and resolution. Through media queries, page layouts can be customized for different devices, thus achieving responsive design). When the window is resized or scrolled, and during CSS media queries, all created annotations are traversed, and all annotation data is sorted according to the logic of processing annotation data within the current viewport first, followed by annotation data outside the current viewport, resulting in a queue to be scanned. A webpage can be very long, and only a portion of the webpage content is seen at a time; this portion is called the viewport.
[0035] S203, determine the original DOM element corresponding to the labeled data in the queue to be scanned by using a CSS selector.
[0036] By utilizing the mapping relationship between labeled elements, CSS selectors, and labeled data IDs, for each labeled data, its corresponding stored CSS selector can be obtained, and the original DOM element corresponding to the labeled data in the queue to be scanned can be determined through the CSS selector.
[0037] S204, Remeasure the position and size information of the original DOM element in the current viewport.
[0038] The exact position and size of the element in the current viewport can be remeasured by calling element.getBoundingClientRect(). This includes: const rect = document.getElementById('span').getBoundingClientRect(); console.log(rect.top); / / Distance from the top of the element to the top of the viewport console.log(rect.left); / / Distance from the left side of the element to the left side of the viewport. console.log(rect.width); / / The width of the element itself S205, find the corresponding annotation element through the annotation data ID.
[0039] Find the corresponding annotation element by utilizing the mapping relationship between annotation elements, CSS selectors, and annotation data IDs.
[0040] S206, Based on the position information and size information, dynamically calculate the updated position information of the labeled element, and overlay the labeled element on the original DOM element according to the updated position information.
[0041] Based on the new position data measured, dynamically calculate the top and left values that the label layer (such as an absolutely positioned div) should appear, or more modernly, use transform: translate(Xpx, Ypx) to correct its position in real time, ensuring that it always accurately covers the original DOM element.
[0042] In this application, when a layout change occurs, a dynamic position compensation method is used to adjust the display position of each annotation element in a timely manner, achieving pixel-level precise positioning so that the annotation offset is so small that it is almost imperceptible to the user. This application provides an effective dynamic adaptation mechanism for responsive layouts, preventing the annotation layer from failing to automatically perceive and synchronously update its position when the page layout is rearranged due to viewport changes, CSS media queries, or font adjustments, thus avoiding obvious misalignment between annotation elements and the underlying target content.
[0043] If the annotation element is a highlighted text element, it will completely cover the original DOM element; if the annotation element is a comment bubble for text, it will maintain the relative position of the annotation element and the original DOM element; if the annotation element is an arrow for text, it will point from the updated starting point to the original DOM element.
[0044] For highlighted annotations, the annotation will be placed wherever the original DOM element is. For annotations in comment bubbles, the annotation must maintain its relative position to the original DOM element while avoiding screen edges and other elements. For annotations indicated by arrows, the new endpoint is the nearest point on the new position of the original DOM element, and the new starting point is calculated by maintaining a relative vector relationship with the new starting point.
[0045] In this embodiment, different dynamic compensation strategies are adopted for different annotation elements, so that the adjusted annotation elements have better display effects.
[0046] In one example, the method further includes: S301, determine whether the updated position information of the labeled element is within the range of change.
[0047] If the new position of a labeled element is only offset by 1 pixel compared to its old position, for efficiency reasons, it is not necessary to update its information in storage for such a small shift. The range of change can be 2 pixels.
[0048] S302, if not, obtain the update position information and annotation data ID of the labeled element, and generate an update queue.
[0049] If the updated location information exceeds the range of changes, obtain the updated location information and annotation data ID of the qualified annotation elements, and generate an update queue.
[0050] S303, the update queue is sent to the main process through the inter-process communication mechanism.
[0051] S304, the main process reads the currently stored annotation data file and updates the position and version information of the corresponding annotation data according to the annotation data ID in the update queue using an incremental update strategy.
[0052] The main process compares the update queue with the currently stored annotation data file and uses an incremental update strategy to update only the parts that have changed. Specifically, for the annotation data that has changed, the currently stored location information is updated to the updated location information, and the version information is incremented to indicate that the current location information has been updated.
[0053] In this embodiment, after dynamic position compensation, the position information of the labeled elements stored in the main process is updated. In this way, when the rendering process reloads and displays the static webpage, it can obtain the latest position information to display the labeled elements.
[0054] In one example, see Figure 2 This is a flowchart illustrating an annotation method provided in an embodiment of this application. Each rendering process runs a complete Chromium rendering engine, with capabilities identical to a tab in the Chrome browser, specifically responsible for parsing, laying out, and rendering web page content (including HTML, CSS, and JavaScript). In the annotation system, the rendering process handles all visual and interactive tasks: it first loads and displays the static web page to be annotated; after the web page loads, it draws visual elements such as the annotation tool interface, highlight layers, arrows, and comment bubbles on the page. Simultaneously, the rendering process is also responsible for listening to user interactions to capture mouse clicks, drags, keyboard input, and other interactive behaviors. For example, when a user draws an annotation box or enters comment text, these operations are first captured and processed by the JavaScript logic in the rendering process. The annotation UI is drawn after receiving user annotation operations.
[0055] The main process has full access to the operating system and Node.js API. It runs in the Node runtime environment and can fully utilize all the capabilities provided by Node.js.
[0056] When the rendering process detects user annotation behavior, it can send the relevant annotation data to the main process via inter-process communication (IPC), where the main process is responsible for storage. In this scheme, data storage uses local files: the main process serializes the annotation data (e.g., converts it to a JSON string) using the FS module and writes it to a disk file to achieve annotation information persistence; similarly, it can also read stored annotation data from local files.
[0057] See Figure 3 This is a schematic diagram illustrating the process of displaying historical annotations according to an embodiment of this application. When it is necessary to display historical annotation data in a browser window, the window needs a time to trigger the loading of annotation data.
[0058] After the main process creates a browser window and opens the HTML page (after page elements, images, JS, CSS, and other dynamic effects have been rendered), it listens for the `load-finished` event of the rendering process. This event is responsible for reading historical annotation information through the FS module. After reading, the annotation data is sent to the rendering process via the IPC bridge. The rendering process listens for the annotation data sent by the main process through a message listener and uses the DOM API and the CSSOM View module to display the annotation data in the corresponding positions.
[0059] See Figure 4 This diagram illustrates one result of the static webpage annotation method provided in this application, showing the highlighted annotation effect. In tests on a 1080P resolution device, the annotation offset is ≤2px (compared to an average offset of 32px for traditional methods). This solution supports HTML5 / CSS3 standard pages and is based on the Chromium kernel. It requires Electron version 28 or higher.
[0060] Secondly, embodiments of this application provide a labeling device for static web pages, the device comprising: A static webpage loading and display module is used to create a browser window through the main process, and load and display the static webpage to be annotated on the browser window through the rendering process. The static webpage to be annotated includes annotation tools. The user action event listening module is used to listen for user action events through the rendering process. The user action events are the interactive behaviors of the original DOM elements using the annotation tool. The annotation information persistence module is used to send the annotation elements generated by the user action event to the main process through the inter-process communication mechanism, so that the main process can serialize the annotation elements to obtain structured annotation data and persistently store the structured annotation data locally. The structured annotation data is physically isolated from the source file of the static webpage to be annotated. The annotation information display module is used to read annotation data from the local machine and send the annotation data to the rendering process through an inter-process communication mechanism when the main process listens to the webpage loading completion event of the rendering process, so that the rendering process can display annotation elements at the corresponding positions on the static webpage according to the received annotation data.
[0061] In one possible implementation, the annotation information display module is specifically used for: When the main process listens to the rendering process's webpage loading completion event, it reads the annotation data from the local machine and sends the annotation data to the rendering process through the inter-process communication mechanism; When the rendering process listens to the annotation data sent by the main process, it parses the annotation data, determines the original DOM element to be annotated through the CSS selector, obtains the position and size of the original DOM element to be annotated, creates an annotation element corresponding to the annotation data, adds the annotation element to the transparent annotation layer overlaid on the static webpage, calculates the display position of the annotation element based on the position and size of the original DOM element to be annotated, and displays the annotation element at the display position.
[0062] In one possible implementation, the device further includes: The mapping module is used to maintain the mapping relationship between annotation elements, CSS selectors, and annotation data IDs through the rendering process; The sorting module is used to obtain all the annotation data of the current webpage when the rendering process listens for layout change events, sort all the annotation data according to spatial position, and obtain a queue to be scanned. The queue to be scanned prioritizes the annotation data in the current viewport. The original DOM element determination module is used to determine the original DOM elements corresponding to the labeled data in the queue to be scanned by using CSS selectors; The retest module is used to remeasure the position and size information of the original DOM element in the current viewport; The annotation element determination module is used to find the corresponding annotation element through the annotation data ID; The update position information determination module is used to dynamically calculate the update position information of the labeled element based on the position information and size information, and then overlay the labeled element onto the original DOM element according to the update position information.
[0063] In one possible implementation, the device further includes: The judgment module is used to determine whether the updated position information of the labeled element is within the range of change; The update queue generation module is used to, if not, obtain the update position information and annotation data ID of the labeled element and generate an update queue; An update queue sending module is used to send the update queue to the main process via an inter-process communication mechanism. The update module is used to read the currently stored annotation data file through the main process, and update the position information and version information of the corresponding annotation data according to the annotation data ID in the update queue using an incremental update strategy.
[0064] Thirdly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-described methods for detecting the direction of pedestrian gaze based on a top-mounted camera.
[0065] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).
[0066] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0067] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are described simply because they are similar to the method embodiments; relevant parts can be referred to the descriptions of the method embodiments.
[0068] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes to the above embodiments within the scope of the present invention without departing from the principles and spirit of the present invention.
Claims
1. A method for annotating static web pages, characterized in that, The method includes: A browser window is created by the main process, and a rendering process loads and displays the static webpage to be annotated in the browser window. The static webpage to be annotated includes annotation tools. The rendering process listens for user action events, which are interactive behaviors on the original DOM elements using the annotation tool. The annotation elements generated by the user action event are sent to the main process through an inter-process communication mechanism, so that the main process can serialize the annotation elements to obtain structured annotation data and persistently store the structured annotation data locally. The structured annotation data is physically isolated from the source file of the static webpage to be annotated. When the main process listens to the rendering process's webpage loading completion event, it reads the annotation data from the local machine and sends the annotation data to the rendering process through the inter-process communication mechanism, so that the rendering process can display the annotation elements at the corresponding positions on the static webpage according to the received annotation data.
2. The method according to claim 1, characterized in that, When the main process listens to the rendering process's webpage loading completion event, it reads annotation data from the local machine and sends the annotation data to the rendering process via an inter-process communication mechanism, so that the rendering process displays annotation elements at the corresponding positions on the static webpage based on the received annotation data. This includes: When the main process listens to the rendering process's webpage loading completion event, it reads the annotation data from the local machine and sends the annotation data to the rendering process through the inter-process communication mechanism; When the rendering process listens to the annotation data sent by the main process, it parses the annotation data, determines the original DOM element to be annotated through the CSS selector, obtains the position and size of the original DOM element to be annotated, creates an annotation element corresponding to the annotation data, adds the annotation element to the transparent annotation layer overlaid on the static webpage, calculates the display position of the annotation element based on the position and size of the original DOM element to be annotated, and displays the annotation element at the display position.
3. The method according to claim 1, characterized in that, The method further includes: The rendering process maintains a mapping relationship including annotation elements, CSS selectors, and annotation data IDs. When the rendering process detects a layout change event, it obtains all the annotation data of the current webpage, sorts all the annotation data according to spatial location, and obtains a queue to be scanned. The queue to be scanned prioritizes the annotation data within the current viewport. The original DOM elements corresponding to the labeled data in the queue to be scanned are determined by using CSS selectors; Remeasure the position and size information of the original DOM element in the current viewport; Find the corresponding annotation element using the annotation data ID; Based on the position and size information, the updated position information of the labeled element is dynamically calculated, and the labeled element is overlaid on the original DOM element according to the updated position information.
4. The method according to claim 3, characterized in that, The step of dynamically calculating the updated position information of the labeled element based on position and size information, and then overlaying the labeled element onto the original DOM element according to the updated position information, includes: If the annotation element is a highlighted text element, it will completely cover the original DOM element; if the annotation element is a comment bubble for text, it will maintain the relative position of the annotation element and the original DOM element; if the annotation element is an arrow for text, it will point from the updated starting point to the original DOM element.
5. The method according to claim 3, characterized in that, The method further includes: Determine whether the updated position information of the labeled element is within the range of change; If not, obtain the update location information and annotation data ID of the labeled element, and generate an update queue; The update queue is sent to the main process via an inter-process communication mechanism; The main process reads the currently stored annotation data file and updates the location and version information of the corresponding annotation data according to the annotation data ID in the update queue using an incremental update strategy.
6. A labeling device for static web pages, characterized in that, The device includes: A static webpage loading and display module is used to create a browser window through the main process, and load and display the static webpage to be annotated on the browser window through the rendering process. The static webpage to be annotated includes annotation tools. The user action event listening module is used to listen for user action events through the rendering process. The user action events are the interactive behaviors of the original DOM elements using the annotation tool. The annotation information persistence module is used to send the annotation elements generated by the user action event to the main process through the inter-process communication mechanism, so that the main process can serialize the annotation elements to obtain structured annotation data and persistently store the structured annotation data locally. The structured annotation data is physically isolated from the source file of the static webpage to be annotated. The annotation information display module is used to read annotation data from the local machine and send the annotation data to the rendering process through an inter-process communication mechanism when the main process listens to the webpage loading completion event of the rendering process, so that the rendering process can display annotation elements at the corresponding positions on the static webpage according to the received annotation data.
7. The apparatus according to claim 6, characterized in that, The annotation information display module is specifically used for: When the main process listens to the rendering process's webpage loading completion event, it reads the annotation data from the local machine and sends the annotation data to the rendering process through the inter-process communication mechanism; When the rendering process listens to the annotation data sent by the main process, it parses the annotation data, determines the original DOM element to be annotated through the CSS selector, obtains the position and size of the original DOM element to be annotated, creates an annotation element corresponding to the annotation data, adds the annotation element to the transparent annotation layer overlaid on the static webpage, calculates the display position of the annotation element based on the position and size of the original DOM element to be annotated, and displays the annotation element at the display position.
8. The apparatus according to claim 6, characterized in that, The device further includes: The mapping module is used to maintain the mapping relationship between annotation elements, CSS selectors, and annotation data IDs through the rendering process; The sorting module is used to obtain all the annotation data of the current webpage when the rendering process listens for layout change events, sort all the annotation data according to spatial position, and obtain a queue to be scanned. The queue to be scanned prioritizes the annotation data in the current viewport. The original DOM element determination module is used to determine the original DOM elements corresponding to the labeled data in the queue to be scanned by using CSS selectors; The retest module is used to remeasure the position and size information of the original DOM element in the current viewport; The annotation element determination module is used to find the corresponding annotation element through the annotation data ID; The update position information determination module is used to dynamically calculate the update position information of the labeled element based on the position information and size information, and then overlay the labeled element onto the original DOM element according to the update position information.
9. The apparatus according to claim 8, characterized in that, The device further includes: The judgment module is used to determine whether the updated position information of the labeled element is within the range of change; The update queue generation module is used to, if not, obtain the update position information and annotation data ID of the labeled element and generate an update queue; An update queue sending module is used to send the update queue to the main process via an inter-process communication mechanism. The update module is used to read the currently stored annotation data file through the main process, and update the position information and version information of the corresponding annotation data according to the annotation data ID in the update queue using an incremental update strategy.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the annotation method for any of the static web pages described in claims 1-5.