Multi-level two-way communication system and method for browser plug-in and client application program

Abstract

The invention provides a multi-level two-way communication system and method for a browser plug-in and a client application program, and relates to the technical field of browser plug-in and client application program communication. According to the application, the limitation of a security sandbox is broken through; the establishment of the connection between the client and the browser plug-in is safely and reliably realized through the combination of Puppeteer injection and a CustomEvent mechanism; based on the BaseMesager abstract design, a unified communication interface is provided for different execution contexts, and the development complexity is reduced.

Description

Technical Field

[0001] This invention relates to the field of browser plugin and client application communication technology, and more specifically, to a multi-level bidirectional communication system and method between browser plugins and client applications. Background Technology

[0002] With the rapid development of Web technology, browser extensions have become a key tool for enhancing browser functionality and improving user experience. In many application scenarios such as data collection, automated testing, web page analysis, and robotic process automation (RPA), browser extensions need to communicate with local client applications in real time and efficiently to realize complex business logic and automated operations. However, the browser security sandbox mechanism imposes strict restrictions on the communication between extensions and external applications, and traditional communication methods such as HTTP requests and direct socket connections are difficult to meet the requirements of real-time performance and reliability. Furthermore, browser plugins often contain multiple execution contexts (such as Background Script, ContentScript, iframe components, etc.), and these components also require unified and efficient message passing to achieve complex functional integration. When solving the communication problem between browser plugins and client applications, the following challenges are often faced: how to establish a stable and reliable communication connection while ensuring security; how to achieve low-latency data transmission to meet real-time control requirements; and how to design a scalable communication architecture to support various application scenarios and complex functional integration. Therefore, we propose an improvement, namely a multi-level bidirectional communication system and method between browser plugins and client applications. Summary of the Invention

[0003] This invention provides a multi-level bidirectional communication system between a browser plugin and a client application, including a client layer, a browser plugin layer, and a communication bridge layer; The client layer includes: The Socket.IO server module runs in the client application and is responsible for creating and managing WebSocket services. The Puppeteer control module is responsible for launching the browser, loading plugins, and injecting communication parameters. The message processor routing table manages the processing logic for different types of messages in a data-driven manner. The browser plugin layer includes: Background Service Worker is the plugin's background process, serving as a message relay hub. Content Script is a script injected into a webpage that is responsible for interacting with the page's DOM. Sidepanel is a sidebar UI component used to display workflows and interactive interfaces; The communication bridging layer includes: The Socket.IO client runs in the Background and establishes a persistent connection with the client Socket server. The unified messaging mechanism (Messager) provides a standardized communication interface between various components within the plugin. Port connection management is based on the runtime.connect mechanism of the Chrome Extension API.

[0004] As a preferred technical solution of this application, the browser plugin layer also includes multiple iframe components, including Smart-Hint, Preview, Toolbar, Modal, and JsonTree.

[0005] A multi-level bidirectional communication method between a browser plugin and a client application, using a multi-level bidirectional communication system between a browser plugin and a client application, includes the following steps: S1. Server initialization: When the client application starts, the Socket service module performs the following operations: Dynamically obtain available ports; Create a Socket.IO server instance and configure CORS cross-domain support; Initialize the message handler routing table; An exponential backoff and retry mechanism is used to ensure successful service startup. S2, Browser startup and plugin loading, Puppeteer control module execution: Launch the Chrome browser by calling puppeteer-core; Load browser plugins using the --load-extension parameter; Configure browser startup parameters to disable automated feature detection; S3, port injection, a key step in establishing a connection; S4. Once the Socket connection is established, the Background Service Worker executes the following after receiving the port number: Create a Socket.IO client instance; Connect to http: / / localhost:{port}; Register event handlers for successful connection, disconnection, and errors; Send a task:join message to join a specific task room.

[0006] As a preferred technical solution of this application, the key steps for establishing a connection in S3 include: The client injects JavaScript code into the page using Puppeteer's page.evaluate() method; The injected code sets the global variable window.__OCTOPARSE_SOCKET_PORT__ to store the port number; Initialization messages are sent simultaneously through both CustomEvent and postMessage mechanisms; The Content Script listens for these events and extracts the port number; The Content Script forwards the port number to the Background Service Worker.

[0007] As a preferred technical solution of this application, it also includes a multi-layer communication architecture within the plugin. The multi-layer communication architecture within the plugin includes a message source enumeration definition, which defines the following message source types: CONTENT: Content Script; BACKGROUND: Background Service Worker; SIDEPANEL: Sidebar panel; SMART_HINT: Intelligent suggestion for iframes; PREVIEW: Data preview iframe; TOOLBAR: Toolbar iframe; MODAL: Modal frame (iframe); JSONTREE: An iframe that displays a JSON tree structure; OCTOPARSE: Injected from the client-side Puppeteer.

[0008] As a preferred technical solution in this application, the multi-layered communication architecture within the plugin also includes the basic messager abstract class BaseMessager, which is inherited by all messagers and provides: A unique message ID is generated, in the format: {source}-{timestamp}-{random}; sendMessageWithResponse, a request-response pattern with a timeout; One-way message sending (sendMessageOnly); Message processor registration and removal; Management of pending message queues.

[0009] As a preferred technical solution in this application, the multi-layer communication architecture within the plugin also includes a Background Messenger, which acts as a message relay hub to achieve the following: Maintaining a Content Script connection pool (Map)<tabId,Port> ); Maintain the Sidepanel connection pool (Map)<windowId,Port> ); Automatically route and forward messages based on the message's target field; Handle message queue caching when the Sidepanel is not ready.

[0010] As a preferred technical solution in this application, the multi-layer communication architecture within the plugin also includes a Content Messenger. The Content Messenger is responsible for communication between the Content Script and other components, including the following steps: Establish a persistent connection with Background using runtime.connect; Maintain multiple iframe component references; Communicate with the iframe component via postMessage; Listen for the ready state of the iframe.

[0011] As a preferred technical solution in this application, the multi-layered communication architecture within the plugin also includes an iframe message handler, which runs inside the iframe. The iframe message handler listens for messages from the parent window using window.addEventListener; The iframe message handler sends messages to the parent window via window.parent.postMessage; iframe messaging supports bidirectional data transmission.

[0012] As a preferred technical solution of this application, a big data fragmentation transmission mechanism is also included. This mechanism is designed for big data transmission scenarios, and the system implements the Blob fragmentation transmission protocol, including a sender (BlobSender) and a receiver (BlobReceiver). The sender, BlobSender, includes: The data is serialized into JSON and then compressed using gzip. Calculate the SHA256 hash value for integrity verification; Split the data into multiple chunks; Send blob:begin and blob:chunk sequentially. N, blob:end message; Implement a sliding window confirmation mechanism; The receiving end, BlobReceiver, includes: Listening blob:begin initializes the receive session; Receive blob:chunk and send ack confirmation; Reassemble the data after receiving blob:end; Decompress and parse the JSON; Send blob:complete to confirm completion.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: In the scheme of this application: 1. Overcoming security sandbox limitations: By combining Puppeteer injection with the CustomEvent mechanism, a secure and reliable connection between the client and the browser plugin is established; 2. Unified Communication Architecture: Based on the abstract design of BaseMessager, it provides a unified communication interface for different execution contexts, reducing development complexity; 3. High reliability: Socket connections support automatic reconnection, message requests support timeout mechanisms, iframe messages support ready status detection and queue caching, and large data transmission supports fragmented confirmation and integrity verification; 4. High scalability: The message processor routing table supports dynamic expansion; adding new communication components only requires inheriting from BaseMessager; and message types are uniformly managed through enumeration. 5. Two-way real-time communication: Based on Socket.IO WebSocket connection, it supports low-latency two-way data transmission; 6. Compatible with multiple scenarios: Supports page navigation for single-page applications (SPAs), task management for multiple tabs, and complex UI interactions across iframes. Attached Figure Description

[0014] Figure 1 A diagram illustrating the overall architecture of the multi-level bidirectional communication system between the browser plugin and the client application provided in this application. Figure 2 Establish a timing diagram for the communications provided in this application; Figure 3 The internal message flow diagram of the plugin provided in this application; Figure 4 Diagram of the big data fragmentation transmission protocol provided for this application; Figure 5 The port injection flowchart provided in this application; Figure 6 A schematic diagram illustrating the initialization of the Socket service provided in this application; Figure 7 A schematic diagram illustrating the port injection implementation provided in this application; Figure 8 A schematic diagram illustrating the Content Script listener injection message provided in this application; Figure 9 A schematic diagram illustrating the Background message routing provided in this application; Figure 10 This is a schematic diagram of the request-response pattern with timeout provided in this application. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] It should be noted that, unless otherwise specified, the embodiments and features and technical solutions in the present invention can be combined with each other.

[0017] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0018] Example 1, please refer to Figure 1A multi-level bidirectional communication system between a browser plugin and a client application, comprising a client layer, a browser plugin layer, and a communication bridge layer; The client layer includes: The Socket.IO server module runs in the client application and is responsible for creating and managing WebSocket services. The Puppeteer control module is responsible for launching the browser, loading plugins, and injecting communication parameters. The message processor routing table manages the processing logic for different types of messages in a data-driven manner. The browser plugin layer includes: Background Service Worker is the plugin's background process, serving as a message relay hub. Content Script is a script injected into a webpage that is responsible for interacting with the page's DOM. Sidepanel is a sidebar UI component used to display workflows and interactive interfaces; Multiple iframe components, including Smart-Hint, Preview, Toolbar, Modal, and JsonTree.

[0019] The communication bridging layer includes: The Socket.IO client runs in the Background and establishes a persistent connection with the client Socket server. The unified messaging mechanism (Messager) provides a standardized communication interface between various components within the plugin. Port connection management is based on the runtime.connect mechanism of the Chrome Extension API.

[0020] Example 2, please refer to Figures 2-5 A multi-level bidirectional communication method between a browser plugin and a client application, using a multi-level bidirectional communication system between a browser plugin and a client application, includes the following steps: S1. Server initialization: When the client application starts, the Socket service module performs the following operations: Dynamically obtain available ports (port range 6500-65000); Create a Socket.IO server instance and configure CORS cross-domain support; Initialize the message handler routing table; An exponential backoff and retry mechanism is used to ensure successful service startup. S2, Browser startup and plugin loading, Puppeteer control module execution: Launch the Chrome browser by calling puppeteer-core; Load browser plugins using the --load-extension parameter; Configure browser startup parameters to disable automated feature detection; S3, port injection, a key step in establishing a connection; S4. Once the Socket connection is established, the Background Service Worker executes the following after receiving the port number: Create a Socket.IO client instance; Connect to http: / / localhost:{port}; Register event handlers for successful connection, disconnection, and errors; Send a task:join message to join a specific task room.

[0021] Furthermore, in S3, the key steps for establishing a connection include: The client injects JavaScript code into the page using Puppeteer's page.evaluate() method; The injected code sets the global variable window.__OCTOPARSE_SOCKET_PORT__ to store the port number; Initialization messages are sent simultaneously through both CustomEvent and postMessage mechanisms; The Content Script listens for these events and extracts the port number; The Content Script forwards the port number to the Background Service Worker.

[0022] Furthermore, it also includes a multi-layered communication architecture within the plugin, which includes a message source enumeration definition. The message source enumeration defines the following message source types: CONTENT: Content Script; BACKGROUND: Background Service Worker; SIDEPANEL: Sidebar panel; SMART_HINT: Intelligent suggestion for iframes; PREVIEW: Data preview iframe; TOOLBAR: Toolbar iframe; MODAL: Modal frame (iframe); JSONTREE: An iframe that displays a JSON tree structure; OCTOPARSE: Injected from the client-side Puppeteer.

[0023] Furthermore, the plugin's internal multi-layered communication architecture also includes the basic messager abstract class BaseMessager, which all messagers inherit from, providing: A unique message ID is generated, in the format: {source}-{timestamp}-{random}; sendMessageWithResponse, a request-response pattern with a timeout; One-way message sending (sendMessageOnly); Message processor registration and removal; Management of pending message queues.

[0024] Furthermore, the plugin's internal multi-layered communication architecture also includes a Background Messenger, which acts as a message relay hub to achieve the following: Maintaining a Content Script connection pool (Map)<tabId,Port> ); Maintain the Sidepanel connection pool (Map)<windowId,Port> ); Automatically route and forward messages based on the message's target field; Handle message queue caching when the Sidepanel is not ready.

[0025] Furthermore, the internal multi-layered communication architecture also includes a Content Messenger, which is responsible for communication between Content Script and other components, including the following steps: Establish a persistent connection with Background using runtime.connect; Maintain multiple iframe component references; Communicate with the iframe component via postMessage; Monitor the iframe's ready state to ensure messages are delivered reliably.

[0026] Furthermore, the plugin's internal multi-layered communication architecture also includes an iframe message handler, which runs inside the iframe. The iframe message handler listens for messages from the parent window using window.addEventListener; The iframe message handler sends messages to the parent window via window.parent.postMessage; iframe messaging supports bidirectional data transmission.

[0027] Furthermore, it also includes a big data fragmentation transmission mechanism. This mechanism is designed for large data transmission scenarios (such as complete task configuration). The system implements the Blob fragmentation transmission protocol, including a sender (BlobSender) and a receiver (BlobReceiver). The sender, BlobSender, includes: The data is serialized into JSON and then compressed using gzip. Calculate the SHA256 hash value for integrity verification; The data is split into multiple chunks (each approximately 32KB); Send blob:begin and blob:chunk sequentially. N, blob:end message; Implement a sliding window confirmation mechanism; The receiving end, BlobReceiver, includes: Listening blob:begin initializes the receive session; Receive blob:chunk and send ack confirmation; Reassemble the data after receiving blob:end; Decompress and parse the JSON; Send blob:complete to confirm completion.

[0028] Example 3, the known application areas of this application include: I. Webpage Data Acquisition System: Application method: The client application controls the browser plugin to perform data collection tasks in real time through the communication system of this invention, including setting collection rules, starting / stopping collection, and receiving collection results; Typical products: Octopus data collector, Locomotive data collector, etc.; II. RPA (Robotic Process Automation): Application: The RPA platform uses this invention to automate browser control, performing operations such as form filling, button clicking, and data extraction; Typical scenarios: automated report generation, batch data entry, and cross-system data synchronization; III. Automated Testing Framework: Application: The testing framework communicates with the browser plugin through this invention to perform page operations and collect test results, screenshots, performance metrics, etc. Typical products: Selenium Grid extension, Cypress plugin, Playwright enhancement; Potential application areas for this application include: I. Browser security check: Application: The security detection tool uses this invention to analyze web page content in real time, detect security threats such as XSS and phishing, and report the results to the client. Application scenarios: Enterprise network security, website security auditing; II. Intelligent Customer Service / AI Assistant: Application: The AI ​​assistant client uses this invention to obtain the content of the webpage currently being viewed by the user and provides context-sensitive intelligent suggestions; Application scenarios: e-commerce shopping assistant, travel planning assistant; III. Webpage Content Aggregation: Application: Content aggregation platforms use this invention to obtain updated content from multiple web pages in real time and integrate and display it. Application scenarios: news aggregation, price monitoring, public opinion monitoring; IV. Accessibility Assistive Devices: Application: The assistive tool uses this invention to obtain the webpage structure and provides voice navigation and operation guidance for visually impaired users; Application scenarios: Accessibility browsers, screen reader enhancements; V. Enhanced Developer Tools: Application: Developer IDEs can communicate with browser plugins through this invention to achieve functions such as real-time preview, hot update, and enhanced debugging; Application scenarios: front-end development tools, low-code platforms.

[0029] The usage process of the multi-level bidirectional communication system and method between browser plugins and client applications provided by this invention is as follows: Example 4 further includes: I. Socket service initialization: The specific implementation of the client application starting the Socket service is as follows... Figure 6 As shown; II. Port Injection Implementation: Puppeteer injects port numbers using page.evaluate, as detailed below. Figure 7 As shown; III. Content Script listens for injected messages, specifically as follows: Figure 8 As shown; IV. Background message routing, as detailed below Figure 9 As shown; V. Request-response mode with timeout, as detailed below. Figure 10 As shown.

[0030] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0031] Obviously, the embodiments described above are merely some embodiments of the present invention, not all embodiments. The accompanying drawings show preferred embodiments of the present invention, but do not limit the patent scope of the present invention. The present invention can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the patent protection scope of this invention.

Claims

1. A multi-level bidirectional communication system between a browser plugin and a client application, characterized in that, This includes the client layer, browser plugin layer, and communication bridging layer; The client layer includes: The Socket.IO server module runs in the client application and is responsible for creating and managing WebSocket services. The Puppeteer control module is responsible for launching the browser, loading plugins, and injecting communication parameters. The message processor routing table manages the processing logic for different types of messages in a data-driven manner. The browser plugin layer includes: Background Service Worker is the plugin's background process, serving as a message relay hub. Content Script is a script injected into a webpage that is responsible for interacting with the page's DOM. Sidepanel is a sidebar UI component used to display workflows and interactive interfaces; The communication bridging layer includes: The Socket.IO client runs in the Background and establishes a persistent connection with the client Socket server. The unified messaging mechanism (Messager) provides a standardized communication interface between various components within the plugin. Port connection management is based on the runtime.connect mechanism of the Chrome Extension API.

2. The multi-level bidirectional communication system between browser plugins and client applications according to claim 1, characterized in that, The browser plugin layer also includes several iframe components, including Smart-Hint, Preview, Toolbar, Modal, and JsonTree.

3. A multi-level bidirectional communication method between a browser plugin and a client application, using the multi-level bidirectional communication system between a browser plugin and a client application as described in claim 2, characterized in that... Includes the following steps: S1. Server initialization: When the client application starts, the Socket service module performs the following operations: Dynamically obtain available ports; Create a Socket.IO server instance and configure CORS cross-domain support; Initialize the message handler routing table; An exponential backoff and retry mechanism is used to ensure successful service startup. S2, Browser startup and plugin loading, Puppeteer control module execution: Launch the Chrome browser by calling puppeteer-core; Load browser plugins using the --load-extension parameter; Configure browser startup parameters to disable automated feature detection; S3, port injection, a key step in establishing a connection; S4. Once the Socket connection is established, the Background Service Worker executes the following after receiving the port number: Create a Socket.IO client instance; Connect to http: / / localhost:{port}; Register event handlers for successful connection, disconnection, and errors; Send a task:join message to join a specific task room.

4. The multi-level bidirectional communication method between browser plugins and client applications according to claim 3, characterized in that, In S3, the key steps for establishing a connection include: The client injects JavaScript code into the page using Puppeteer's page.evaluate() method; The injected code sets the global variable window.__OCTOPARSE_SOCKET_PORT__ to store the port number; Initialization messages are sent simultaneously through both CustomEvent and postMessage mechanisms; The Content Script listens for these events and extracts the port number; The Content Script forwards the port number to the Background Service Worker.

5. The multi-level bidirectional communication method between browser plugins and client applications according to claim 3, characterized in that, It also includes a multi-layered communication architecture within the plugin, which includes a message source enumeration definition. The message source enumeration defines the following message source types: CONTENT: Content Script; BACKGROUND: Background Service Worker; SIDEPANEL: Sidebar panel; SMART_HINT: Intelligent suggestion for iframes; PREVIEW: Data preview iframe; TOOLBAR: Toolbar iframe; MODAL: Modal frame (iframe); JSONTREE: An iframe that displays a JSON tree structure; OCTOPARSE: Injected from the client-side Puppeteer.

6. The multi-level bidirectional communication method between browser plugins and client applications according to claim 5, characterized in that, The plugin's internal multi-layered communication architecture also includes the basic messager abstract class BaseMessager, from which all messagers inherit, providing: A unique message ID is generated, in the format: {source}-{timestamp}-{random}; sendMessageWithResponse, a request-response pattern with a timeout; One-way message sending (sendMessageOnly); Message processor registration and removal; Management of pending message queues.

7. The multi-level bidirectional communication method between a browser plugin and a client application according to claim 6, characterized in that, The plugin's internal multi-layered communication architecture also includes a Background Messenger, which acts as a message relay hub to achieve the following: Maintaining a Content Script connection pool (Map)<tabId,Port> ); Maintain the Sidepanel connection pool (Map)<windowId,Port> ); Automatically route and forward messages based on the message's target field; Handle message queue caching when the Sidepanel is not ready.

8. The multi-level bidirectional communication method between a browser plugin and a client application according to claim 7, characterized in that, The plugin's internal multi-layered communication architecture also includes a Content Messenger, which is responsible for communication between ContentScript and other components, including the following steps: Establish a persistent connection with Background using runtime.connect; Maintain multiple iframe component references; Communicate with the iframe component via postMessage; Listen for the ready state of the iframe.

9. The multi-level bidirectional communication method between a browser plugin and a client application according to claim 8, characterized in that, The plugin's internal multi-layered communication architecture also includes an iframe message handler, which runs inside the iframe. The iframe message handler listens for messages from the parent window using window.addEventListener; The iframe message handler sends messages to the parent window via window.parent.postMessage; iframe messaging supports bidirectional data transmission.

10. The multi-level bidirectional communication method between a browser plugin and a client application according to claim 3, characterized in that, It also includes a big data fragmentation transmission mechanism. This mechanism is designed for large data transmission scenarios. The system implements the Blob fragmentation transmission protocol, including a sender (BlobSender) and a receiver (BlobReceiver). The sender, BlobSender, includes: The data is serialized into JSON and then compressed using gzip. Calculate the SHA256 hash value for integrity verification; Divide the data into multiple chunks; Send blob:begin and blob:chunk sequentially. N, blob:end message; Implement a sliding window confirmation mechanism; The receiving end, BlobReceiver, includes: Listening blob:begin initializes the receive session; Receive blob:chunk and send ack confirmation; Reassemble the data after receiving blob:end; Decompress and parse the JSON; Send blob:complete to confirm completion.

Images