A browser plugin-based method for voice interaction on web pages and browser plugins

By enabling zero-intrusion voice interaction through a browser plugin, and utilizing wake words and real-time analysis to adapt to various websites, the problem of inconvenient browser page interaction has been solved. This achieves plug-and-play and low-power, high-efficiency voice control, improving the interaction efficiency in scenarios such as large commercial screens.

CN123067950APending Publication Date: 2026-11-10CHINA UNITED NETWORK COMM GRP CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611012578.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-08
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing browser page interaction methods are inconvenient in scenarios such as commercial large screen presentations, especially when users cannot operate manually, interaction is limited, and existing voice control solutions require secondary development or rely on external hardware, lacking universality and efficiency.

Method used

Achieving zero-intrusion page voice interaction through a browser plugin, utilizing a wake word mechanism and real-time voice analysis, it adapts to various websites without requiring code modification. Combined with continuous voice monitoring in the wake-up ready state, it achieves a balance between low power consumption and high efficiency.

Benefits of technology

It enables plug-and-play voice interaction without manual clicking or external hardware, adapting to the personalized needs of different business scenarios and improving the convenience of interaction and real-time response capabilities in scenarios such as large commercial screens.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123067950A_ABST
    Figure CN123067950A_ABST
Patent Text Reader

Abstract

This invention relates to the field of human-computer interaction technology, and more particularly to a page voice interaction method and browser plugin based on a browser plugin. The method includes: responding to the detection of a user accessing a website, determining whether the address of the accessed website belongs to a pre-configured list of monitored websites; when the address of the accessed website belongs to the pre-configured list of monitored websites, initiating voice monitoring in the page environment of the accessed website; performing real-time analysis on the audio stream collected through voice monitoring, and entering a wake-up ready state when a target wake-up keyword is identified; in the wake-up ready state, continuously performing real-time analysis on the collected audio stream, and executing a page interaction command corresponding to the target operation keyword on the page of the accessed website when a target operation keyword is identified. This method achieves zero-intrusion page voice interaction through a browser plugin, requiring no code modification or secondary development of the target webpage, and can be adapted to various websites.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, and in particular to a page voice interaction method and browser plugin based on a browser plugin. Background Technology

[0002] With the deepening of digital transformation strategies, enterprises are increasingly improving their informatization levels. To unlock the value of data, various data visualization applications have emerged, among which business intelligence dashboards, data dashboards, and management cockpits, which run in browser-based form, are particularly common and have become important tools for daily operational analysis and decision support.

[0003] However, existing browser page interaction methods still have many inconveniences. In scenarios such as commercial large-screen presentations and exhibition hall explanations, users usually need to manually operate the page using input devices such as mice and keyboards, or rely on assistants for control. When the presenter is far from the screen, their hands are occupied, or it is inconvenient for them to access the input device, page interaction is limited, affecting the continuity and effectiveness of the presentation. In addition, the content displayed on traditional large-screen pages is often static or only refreshed according to predetermined logic, lacking flexible and real-time interactive capabilities. Although some solutions have attempted to introduce voice control, they often require secondary development of the target page to embed voice recognition logic, lacking universality and having a long deployment cycle. Alternatively, they rely on dedicated external hardware terminals to receive voice commands, increasing costs and deployment complexity. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a page voice interaction method and a browser plugin based on a browser plugin, so as to solve the above-mentioned technical problem.

[0005] The technical solution of this invention to solve the above-mentioned technical problems is as follows: A page voice interaction method based on a browser plugin, comprising: in response to detecting that a user visits a website, determining whether the address of the visited website belongs to a pre-configured list of monitoring websites; when the address of the visited website belongs to the pre-configured list of monitoring websites, starting voice monitoring in the page environment of the visited website; performing real-time analysis on the audio stream collected through voice monitoring, and entering a wake-up ready state when a target wake-up keyword is identified; in the wake-up ready state, continuously performing real-time analysis on the collected audio stream, and executing a page interaction instruction corresponding to the target operation keyword in the page of the visited website when a target operation keyword is identified.

[0006] The beneficial effects of this invention are as follows: This method achieves zero-intrusion page voice interaction through a browser plugin, requiring no code modification or secondary development of the target webpage, thus adapting to various websites and exhibiting excellent versatility and plug-and-play characteristics. Employing a wake-word mechanism, users only need to speak preset keywords to activate the interactive function, eliminating the need for manual clicking or reliance on external hardware devices, truly freeing up their hands. This is particularly suitable for scenarios where manual operation is inconvenient, such as large-screen commercial presentations. By pre-configuring monitoring of website addresses, voice monitoring is automatically initiated only on designated pages. Combined with continuous voice analysis in the wake-up-ready state, this ensures real-time interactive response while avoiding resource waste on irrelevant pages, achieving a balance between low power consumption and high efficiency.

[0007] Based on the above technical solution, the present invention can be further improved as follows.

[0008] Furthermore, the step of executing the page interaction instruction corresponding to the target operation keyword in the page of the accessed website includes: when the target operation keyword is a script execution keyword, loading and executing the script code associated with the target operation keyword in the page context of the accessed website.

[0009] The beneficial effects of adopting the above-mentioned further solution are: by associating voice commands with custom script code, any complex interactive behavior can be triggered by voice without modifying the original logic of the page, which greatly expands the flexibility of voice control and can adapt to the personalized needs of different business scenarios.

[0010] Furthermore, executing the page interaction instruction corresponding to the target operation keyword on the page of the accessed website includes: when the target operation keyword is an element operation keyword, locating the target document object model element pre-associated with the element operation keyword on the page of the accessed website; triggering a preset event of the target document object model element, or modifying the visual style attributes of the target document object model element; wherein the visual style attributes include at least one of zoom in, zoom out, show, hide, and full screen.

[0011] The beneficial effects of adopting the above-mentioned further solution are: by directly locating DOM elements in the page and triggering their preset events or modifying their visual style attributes, intuitive visual control of components such as charts, buttons, and panels can be achieved without pre-implanting any voice logic into the page, thus achieving zero-intrusion operation.

[0012] Furthermore, the continuous real-time analysis of the acquired audio stream includes: acquiring the current microphone volume, filtering the audio stream according to the current microphone volume to obtain a pre-processed audio stream; dividing the pre-processed audio stream into multiple audio stream segments according to a preset segmentation strategy; sequentially sending the audio stream segments to a voice server for speech recognition, and receiving the recognition results returned by the voice server.

[0013] The beneficial effects of adopting the above-mentioned further solution are: by filtering the audio stream by acquiring the microphone volume in real time, eliminating silent or low-quality invalid audio segments, and dividing continuous speech into independent audio segments according to the preset segmentation strategy, the processing pressure of the voice server is effectively reduced, the recognition accuracy is improved, and the network bandwidth consumption is reduced.

[0014] Furthermore, executing the page interaction instruction corresponding to the target operation keyword on the page of the accessed website includes: when the target operation keyword is a query keyword, parsing the query conditions corresponding to the query keyword from the recognition results; sending the query conditions to the business server to request query result data; receiving the query result data returned by the business server; and generating and rendering a chart on the page of the accessed website for visually displaying the query result data.

[0015] The beneficial effects of adopting the above-mentioned further solution are: by automatically parsing the query conditions from the speech recognition results and sending them to the business server to obtain data, and then dynamically generating visual charts on the page, users can complete data Q&A and chart display without manual input or mouse clicks, which significantly improves the convenience and real-time nature of data query, and is particularly suitable for instant Q&A in large screen demonstration scenarios.

[0016] Furthermore, the step of sequentially sending the audio stream segments to the voice server for speech recognition and receiving the recognition results returned by the voice server includes: establishing a WebSocket connection with the voice server; sequentially sending the audio stream segments to the voice server based on the WebSocket connection and receiving the recognition results returned by the voice server.

[0017] The advantages of adopting the above-mentioned further solution are: by establishing a WebSocket long connection with the voice server, low-latency real-time transmission of audio stream segments and rapid return of recognition results can be achieved, ensuring smooth and continuous voice interaction, and providing a good user experience even under average network conditions.

[0018] Furthermore, the method also includes: in the wake-up ready state, when a target close keyword is identified, exiting the wake-up ready state.

[0019] The beneficial effect of adopting the above-mentioned further solution is that by recognizing preset shutdown keywords, users can exit the wake-up state at any time by voice command and stop responding to subsequent operation commands.

[0020] Furthermore, before starting voice monitoring, it also includes obtaining the browser's voice recording permissions.

[0021] The beneficial effect of adopting the above-mentioned further solution is that it obtains the browser's microphone access permission, ensuring that the plugin can legally and stably collect user audio data.

[0022] To address the aforementioned technical problems, the present invention also provides a browser plugin for page voice interaction, comprising: The judgment module is used to respond to the detection of a user accessing a website and determine whether the address of the accessed website belongs to the pre-configured list of monitored websites. The voice monitoring module is used to start voice monitoring in the page environment of the accessed website when the address of the accessed website belongs to the pre-configured list of monitored websites. The wake-up module is used to perform real-time analysis on the audio stream collected through voice monitoring. When the target wake-up keyword is identified, it enters the wake-up ready state. The execution module is used to continuously analyze the acquired audio stream in real time in the wake-up ready state, and when the target operation keyword is identified, execute the page interaction instruction corresponding to the target operation keyword on the page of the accessed website.

[0023] To address the aforementioned technical problems, the present invention also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the browser plugin-based page voice interaction method as described above. Attached Figure Description

[0024] Figure 1 This is a flowchart of the page voice interaction method based on browser plugins according to the present invention; Figure 2 This is a flowchart illustrating the plugin execution operation of Embodiment 1 of the present invention; Figure 3 This is a flowchart of the voice monitoring and recording processing according to Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of the browser plugin for page voice interaction according to the present invention. Detailed Implementation

[0025] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0026] As mentioned earlier, although some solutions have attempted to introduce voice control, the current mainstream voice interaction solutions still have obvious limitations: one type of solution requires secondary development of the target page to embed voice recognition and command execution logic into the page code. This approach requires custom development for each page, lacks universality, and requires repeated development, deployment, and update processes, resulting in poor timeliness; another type of solution, although it uses browser plugins, still relies on external hardware devices to communicate with the server and forward commands to the plugin, resulting in high hardware costs. Alternatively, although it does not require external devices, it still requires manual voice monitoring, failing to fundamentally solve the problem of inconvenient manual operation in scenarios such as demonstrations.

[0027] Example 1 Based on this, such as Figure 1 As shown, this embodiment provides a page voice interaction method based on a browser plugin, including: S101. In response to detecting that a user is accessing a website, determine whether the address of the accessed website belongs to the pre-configured list of monitored websites. S102. When the address of the accessed website belongs to the pre-configured list of monitored websites, start voice monitoring in the page environment of the accessed website. S103. Perform real-time analysis on the audio stream collected through voice monitoring, and enter the wake-up ready state when the target wake-up keyword is identified. S104. In the wake-up ready state, continuously analyze the acquired audio stream in real time. When the target operation keyword is identified, execute the page interaction instruction corresponding to the target operation keyword on the page of the accessed website.

[0028] This method achieves zero-intrusion on-page voice interaction through a browser plugin, requiring no code modification or secondary development of the target webpage. It adapts to various websites, exhibiting excellent versatility and plug-and-play characteristics. Employing a wake-word mechanism, users simply need to speak preset keywords to activate the interactive function, eliminating the need for manual clicking or external hardware, truly freeing up their hands. This is particularly suitable for scenarios where manual operation is inconvenient, such as large-screen commercial presentations. By pre-configuring monitoring of website addresses, voice monitoring is automatically initiated only on designated pages. Combined with continuous voice analysis in the wake-up-ready state, this ensures real-time interactive response while avoiding resource waste on irrelevant pages, achieving a balance between low power consumption and high efficiency.

[0029] Optionally, in an embodiment, before initiating voice monitoring, the method further includes obtaining the browser's voice recording permission.

[0030] Specifically, this method is implemented by a browser plugin. The user first installs the browser plugin in their browser. During installation, the plugin requests voice recording permission, i.e., microphone permission, from the browser. After the user grants this permission, the plugin gains the ability to capture audio in real time.

[0031] After the plugin is installed, a configuration interface (popup page) is provided, where users can pre-configure parameters, including but not limited to: List of websites to monitor: Configure the website addresses (host list) for which you want to enable voice monitoring. The plugin will only start voice monitoring if the accessed website address belongs to this list.

[0032] Wake-up Keyword: Configure the word used to wake up the plugin, such as "Hello Big Screen" or "Start". Multiple synonymous wake-up words can be configured.

[0033] Customize operation keywords and bound behaviors: Configure script execution keywords (e.g., refresh report) and bound JavaScript code; configure element operation keywords (e.g., zoom in on chart, hide title) and bound Document Object Model (DOM) element selectors and operation types, including click, zoom in, zoom out, show, hide, full screen, etc.; configure query keywords (e.g., query last quarter's sales) and corresponding query templates.

[0034] Close keywords: Configure words used to exit the wake-up state, such as "exit", "close", "stop listening", etc.

[0035] Time configuration: Configure the delay time for speech segmentation, the time for automatic plugin exit, the maximum time for uploading speech files to the server, etc.

[0036] After the user saves the configuration, the plugin stores all configuration data in the browser's local cache for later use.

[0037] The plugin runs continuously in the browser background, monitoring user behavior on the website by listening to the browser's page navigation events. Page navigation events include opening new tabs, refreshing the page, and switching between open tabs.

[0038] When the plugin detects a user visiting a website, such as when the user enters a URL in the address bar or clicks a link, it retrieves the URL of the currently active tab and reads a pre-configured list of monitored websites from its local cache. The plugin then determines whether the currently visited website address belongs to this list. Figure 2 As shown, if the site belongs to the category, global voice monitoring will be started in the page environment of the visited website; otherwise, no voice collection will be performed to save resources.

[0039] The specific method for initiating voice monitoring is as follows: the plugin injects a script into the current page. This script calls the browser's relevant interface to obtain microphone permissions and creates an audio capture context. Afterward, the browser begins to capture the user's audio stream in real time. Simultaneously, the plugin establishes a WebSocket connection with the voice server for subsequent real-time transmission of voice data.

[0040] After voice monitoring starts, the plugin continuously collects audio streams once the volume reaches a preset threshold. To reduce the load on the voice server and ensure recognition stability, the plugin first performs preprocessing on the raw audio stream.

[0041] Specifically, the volume amplitude of each frame of the audio stream is acquired in real time, and audio data with a volume below a threshold is discarded, retaining only valid speech segments with a volume at or above the threshold. This filters out low-frequency noise such as ambient noise and silent segments, improving the accuracy of subsequent recognition. The volume amplitude can be obtained by calculating the root mean square or peak value of the sampling points.

[0042] like Figure 3 As shown, based on a pre-configured speech segmentation time threshold, the continuous pre-processed audio stream is divided into multiple audio segments. Each segment corresponds to an independent speech input unit, such as a command. Each audio segment is sent sequentially to the speech server. The speech server runs an Automatic Speech Recognition (ASR) model, converts the audio segments into text, and returns the recognition result text to the plugin.

[0043] After receiving the recognition results, the plugin performs keyword matching. First, it matches the wake-up keyword: if the recognition result text contains a pre-configured wake-up keyword, the plugin switches from a non-wake-up state to a wake-up-ready state. At this point, the plugin can begin responding to user commands. Optionally, the plugin can notify the user that it has been woken up by playing a notification sound or displaying a notification icon in the corner of the page.

[0044] Optionally, in this embodiment, the acquired audio stream is continuously analyzed in real time, including: obtaining the current microphone volume and filtering the audio stream according to the current microphone volume to obtain a preprocessed audio stream; dividing the preprocessed audio stream into multiple audio stream segments according to a preset segmentation strategy; sending the audio stream segments sequentially to the voice server for speech recognition, and receiving the recognition results returned by the voice server.

[0045] Optionally, in this embodiment, the process of sequentially sending audio stream segments to a voice server for speech recognition and receiving the recognition results returned by the voice server includes: establishing a WebSocket connection with the voice server; sequentially sending audio stream segments to the voice server based on the WebSocket connection and receiving the recognition results returned by the voice server.

[0046] When the plugin is in the wake-up ready state, it continues to perform the same volume filtering, segmentation, and server-side recognition process on subsequently acquired audio streams, and matches the operation keywords contained in the recognition results in real time. Based on the type of operation keyword, it executes the corresponding page interaction command, specifically in three cases.

[0047] Optionally, in an embodiment, executing a page interaction instruction corresponding to the target operation keyword on the page of the accessed website includes: when the target operation keyword is a script execution keyword, loading and executing script code associated with the target operation keyword in the page context of the accessed website.

[0048] For example, if a user says "execute refresh report," where "execute refresh report" is a pre-configured script execution keyword, the plugin retrieves the JavaScript code bound to that keyword from the local cache and loads and executes the script in the context of the current page using `eval` or the `Function` constructor. The script can call existing JavaScript functions on the page, modify page data structures, trigger network requests, and so on. Because the script injected by the plugin shares the same DOM environment as the original page, arbitrarily complex interactive logic can be implemented non-intrusively.

[0049] Optionally, in an embodiment, executing a page interaction instruction corresponding to the target operation keyword on the page of the accessed website includes: when the target operation keyword is an element operation keyword, locating a target document object model element pre-associated with the element operation keyword on the page of the accessed website; triggering a preset event of the target document object model element, or modifying the visual style attributes of the target document object model element; wherein, the visual style attributes include at least one of zooming in, zooming out, showing, hiding, and full screen.

[0050] For example, if a user says "zoom in on the sales trend chart," where "zoom in on the sales trend chart" is a pre-configured keyword for element operation, the plugin will locate the target Document Object Model element based on the DOM element selector saved in the configuration. Then, it will trigger the element's preset events or directly modify the element's visual style properties, such as zoom in, zoom out, show, hide, or full screen. In this way, without needing to pre-implant any voice monitoring logic on the page, the visual effects and interactive behavior of any element on the page can be controlled by voice.

[0051] Optionally, in an embodiment, executing a page interaction instruction corresponding to the target operation keyword on the page of the accessed website includes: when the target operation keyword is a query keyword, parsing the query conditions corresponding to the query keyword from the recognition results; sending the query conditions to the business server to request query result data; receiving the query result data returned by the business server; and generating and rendering a chart on the page of the accessed website for visually displaying the query result data.

[0052] For example, if a user says "query" for the sales figures in East China last quarter, where "query" is the keyword, the plugin will perform the following steps: From the complete recognition results returned by the voice server, the query conditions are parsed out. For example, regular expressions can be used to extract the time period (last quarter), the region (East China), and the metric (sales volume).

[0053] Based on the pre-configured business server interface address, the query conditions are encapsulated in JSON format and sent to the business server via an HTTP POST request. The business server executes the query, such as retrieving data from the data warehouse, and returns the query results. In this embodiment, the query results include the chart title, data values, horizontal and vertical axis labels, and units.

[0054] After receiving the data, the plugin dynamically creates a floating layer or pop-up window on the page and calls the chart rendering library to generate a visual chart, which is then displayed on the current page without disrupting the original page layout.

[0055] Optionally, in an embodiment, the method further includes: in the wake-up ready state, when a target close keyword is identified, exiting the wake-up ready state.

[0056] In this embodiment, the plugin monitors two exit conditions simultaneously while in the wake-up ready state. One is a shutdown keyword. If the user's voice is identified as containing a pre-configured shutdown keyword, such as "exit" or "close," the plugin immediately exits the wake-up ready state and reverts to the non-wake-up state, no longer responding to subsequent operation commands, but still maintaining listening for the wake-up keyword for the next wake-up. Simultaneously, a shutdown script can be executed, such as cleaning up injected overlay elements or stopping certain animations.

[0057] Another approach is timeout exit. This involves setting a timer within the plugin, which resets each time the plugin enters wake-up mode or a valid operation command is detected. If no valid voice command is detected within the pre-configured automatic exit time (i.e., no operation keyword is matched), the timer triggers, and the plugin automatically exits wake-up mode and enters a low-power non-wake-up monitoring mode. This avoids prolonged stay in wake-up mode, which consumes resources, or erroneously responds to ambient sounds.

[0058] If a user wishes to completely stop voice monitoring, they can manually disable monitoring on the current page in the plugin's popup interface.

[0059] To facilitate understanding, the following complete example illustrates the application process of this method: Users install browser plugins on the large-screen computers in the company's showroom and configure a list of websites to listen to. The wake word is "Hello," and the operation keywords include: "Zoom in on sales images" (bind to a DOM element and perform zoom), "Query this month's performance" (bind to a query operation), and "Close assistant" (bind to a close keyword).

[0060] When a user opens a website in a large-screen browser, the plugin automatically detects that the URL belongs to the monitoring list, immediately starts voice monitoring, and displays a "Voice monitoring enabled" message. When the user says "Hello" to the large screen, the plugin recognizes the wake word and enters a wake-up ready state, with an "Assistant awakened" icon appearing in the corner of the screen.

[0061] The user then said "zoom in on the sales chart". After the plugin recognized the keyword, it found the corresponding DOM element and set the transform property of its Cascading Style Sheet to scale(1.5), and the chart was enlarged on the screen.

[0062] The user then says "Check this month's performance." The plugin recognizes the query keyword, extracts "this month" from the speech as the time condition, and sends it to the business server. The server returns the sales data for this month, and the plugin generates a bar chart pop-up in the center of the page.

[0063] After the user views the data, they say "Close Assistant." The plugin recognizes the "close" keyword, exits the wake-up state, and the icon in the corner of the large screen turns off. However, it continues to listen for "Hello" in the background, waiting for the next wake-up. If the user does not perform any voice operation within 60 seconds, the plugin automatically exits the wake-up state and switches to wake-word-only listening mode.

[0064] Through the above process, users can complete complex large-screen interactions without using a mouse or keyboard, greatly improving presentation efficiency. This method can be widely applied to scenarios such as enterprise data dashboards, exhibitions, smart city command centers, and smart conference rooms, significantly improving the convenience of human-computer interaction and user experience, and has high industrial practical value.

[0065] Example 2 like Figure 4 As shown, this embodiment provides a browser plugin 200 for page voice interaction, including: The judgment module 201 is used to determine whether the address of the accessed website belongs to the pre-configured list of monitored websites in response to the detection of a user accessing a website. The voice monitoring module 202 is used to start voice monitoring in the page environment of the accessed website when the address of the accessed website belongs to the pre-configured list of monitored websites. The wake-up module 203 is used to perform real-time analysis on the audio stream collected through voice monitoring. When the target wake-up keyword is identified, it enters the wake-up ready state. The execution module 204 is used to continuously analyze the acquired audio stream in real time when it is in the wake-up ready state. When the target operation keyword is identified, the page interaction instruction corresponding to the target operation keyword is executed on the page of the accessed website.

[0066] Optionally, in an embodiment, the execution module 204 includes: The script execution unit is used to load and execute the script code associated with the target operation keyword in the page context of the visited website when the target operation keyword is the script execution keyword.

[0067] Optionally, in an embodiment, the execution module 204 includes: The positioning unit is used to locate the target document object model element that is pre-associated with the element operation keyword in the visited website page when the target operation keyword is the element operation keyword. Trigger unit, used to trigger preset events of target document object model elements, or to modify the visual style properties of target document object model elements; The visual style attributes include at least one of the following: zoom in, zoom out, show, hide, and full screen.

[0068] Optionally, in an embodiment, the execution module 204 includes: The volume acquisition unit is used to acquire the current microphone volume and filter the audio stream according to the current microphone volume to obtain a preprocessed audio stream. The audio segmentation unit is used to divide the preprocessed audio stream into multiple audio stream segments according to a preset segmentation strategy. The speech recognition unit is used to send audio stream segments sequentially to the speech server for speech recognition and to receive the recognition results returned by the speech server.

[0069] Optionally, in an embodiment, the execution module 204 includes: The parsing unit is used to parse the query conditions corresponding to the query keywords from the recognition results when the target operation keyword is the query keyword. The data request unit is used to send query conditions to the business server and request query result data; The data receiving unit is used to receive query result data returned by the business server. The display unit is used to generate and render charts on the pages of the visited website to visualize and display query results data.

[0070] Optionally, in an embodiment, the speech recognition unit includes: The connection establishment subunit is used to establish a WebSocket connection with the voice server; The speech recognition subunit is used to send audio stream segments sequentially to the speech server via a WebSocket connection and to receive the recognition results returned by the speech server.

[0071] Optionally, in this embodiment, the browser plugin further includes: The status exit module is used to exit the wake-up ready state when the target close keyword is detected.

[0072] Optionally, in this embodiment, before initiating voice monitoring, the method further includes obtaining the browser's voice recording permission.

[0073] The modules described in the embodiments of this invention can be implemented in software. The names of the modules do not necessarily limit the functionality of the module itself.

[0074] Example 3 This embodiment provides a non-transitory computer-readable storage medium that stores computer instructions for causing a computer to execute a browser plugin-based page voice interaction method as described in Embodiment 1.

[0075] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0076] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the aforementioned browser plugin-based page voice interaction method.

[0077] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer. In cases involving a remote computer, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0078] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0079] The computer-readable storage medium provided in this invention can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EEPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0080] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0081] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0082] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0083] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this invention can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0084] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A page voice interaction method based on a browser plugin, characterized in that, include: In response to detecting a user accessing a website, determine whether the address of the accessed website belongs to the pre-configured list of monitored websites; When the address of the accessed website belongs to the pre-configured list of monitored websites, voice monitoring is initiated in the page environment of the accessed website. The audio stream acquired through voice monitoring is analyzed in real time, and when the target wake-up keyword is identified, it enters the wake-up ready state. In the wake-up ready state, the acquired audio stream is continuously analyzed in real time. When a target operation keyword is identified, the page interaction instruction corresponding to the target operation keyword is executed on the page of the accessed website.

2. The page voice interaction method based on browser plugins according to claim 1, characterized in that, Executing the page interaction instruction corresponding to the target operation keyword on the page of the accessed website includes: When the target operation keyword is a script execution keyword, the script code associated with the target operation keyword is loaded and executed in the page context of the visited website.

3. The page voice interaction method based on browser plugins according to claim 1, characterized in that, Executing the page interaction instruction corresponding to the target operation keyword on the page of the accessed website includes: When the target operation keyword is an element operation keyword, locate the target document object model element in the accessed website page that is pre-associated with the element operation keyword; Trigger a preset event of the target document object model element, or modify the visual style attributes of the target document object model element; The visual style attributes include at least one of zoom in, zoom out, show, hide, and full screen.

4. The page voice interaction method based on browser plugins according to claim 1, characterized in that, The continuous real-time analysis of the acquired audio stream includes: Obtain the current microphone volume and filter the audio stream according to the current microphone volume to obtain a preprocessed audio stream; According to a preset segmentation strategy, the preprocessed audio stream is divided into multiple audio stream segments; The audio stream segments are sent sequentially to the voice server for speech recognition, and the recognition results returned by the voice server are received.

5. The page voice interaction method based on browser plugins according to claim 4, characterized in that, Executing the page interaction instruction corresponding to the target operation keyword on the page of the accessed website includes: When the target operation keyword is a query keyword, the query conditions corresponding to the query keyword are parsed from the recognition results; Send the query conditions to the business server to request the query result data; Receive the query result data returned by the business server; On the page of the visited website, a chart is generated and rendered to visualize the query results data.

6. The page voice interaction method based on browser plugins according to claim 4, characterized in that, The step of sequentially sending the audio stream segments to a voice server for speech recognition and receiving the recognition results returned by the voice server includes: Establish a WebSocket connection with the voice server; The audio stream segments are sequentially sent to the voice server based on the WebSocket connection, and the recognition results returned by the voice server are received.

7. The page voice interaction method based on browser plugins according to claim 1, characterized in that, Also includes: In the wake-up ready state, when a target close keyword is identified, the wake-up ready state is exited.

8. The page voice interaction method based on browser plugins according to claim 1, characterized in that, Before starting voice monitoring, you also need to obtain the browser's voice recording permissions.

9. A browser plugin for on-page voice interaction, characterized in that, include: The judgment module is used to respond to the detection of a user accessing a website and determine whether the address of the accessed website belongs to the pre-configured list of monitored websites. The voice monitoring module is used to start voice monitoring in the page environment of the accessed website when the address of the accessed website belongs to the pre-configured list of monitored websites. The wake-up module is used to perform real-time analysis on the audio stream collected through voice monitoring. When the target wake-up keyword is identified, it enters the wake-up ready state. The execution module is used to continuously analyze the acquired audio stream in real time in the wake-up ready state, and when the target operation keyword is identified, execute the page interaction instruction corresponding to the target operation keyword on the page of the accessed website.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the browser plugin-based page voice interaction method as described in any one of claims 1 to 8.