A context-aware intelligent key response method

Abstract

The application discloses a context-aware intelligent key response method. The method comprises: obtaining current context information, including at least one of a current foreground application, a user-selected content type, and a selected content feature; based on the context information, filtering and recommending functions from preset AI functions such as translation, polishing, proofreading, intelligent response, intelligent screenshot, image processing, conference recording, and audio / video transcription through correlation scoring; in response to a first trigger operation of a mouse physical key, displaying the recommended functions in the form of a floating menu near the cursor; and in response to user wheel switching and confirmation of execution through a second trigger operation. The application supports configuring preferred functions for specific applications or content types, and realizes personalized recommendation. The application integrates AI functions in the mouse physical key, and interacts with the wheel through context-adaptive recommendation, significantly improving the AI function calling efficiency and user experience, and is suitable for chat, document editing, image processing, and other desktop application scenarios.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction and artificial intelligence application technology, and in particular to a context-aware intelligent key response method. Background Technology

[0002] With the rapid development of Large Language Models (LLM) and multimodal AI technologies, AI-assisted functions such as translation, text editing, intelligent reply, and image recognition are gradually being integrated into the daily use scenarios of desktop operating systems and application software. However, the current mainstream AI function entry point design has the following shortcomings:

[0003] (1) Dispersed entry points and deep calling paths: Users usually have to click through multiple menus or independent application windows to trigger the required AI functions. The operation path is lengthy and interrupts the user's main workflow.

[0004] (2) Fixed functions and lack of context adaptation: Existing solutions often provide AI functions in the form of fixed shortcut keys or fixed toolbars. They cannot dynamically adjust the recommended function list according to the user's current operation context (such as the type of application in front-end, the type of selected content and content characteristics), which requires users to manually filter irrelevant options and reduces operation efficiency.

[0005] (3) Low utilization of interactive devices: The physical buttons (such as side buttons) and scroll wheel of the mouse have rich interactive potential, but they are generally limited to fixed back / forward or page scrolling functions and have failed to be deeply integrated with AI functions.

[0006] (4) Insufficient personalization: The existing system lacks a learning and adaptation mechanism for user habits, and cannot provide priority recommendations for specific applications or content types, which reduces the efficiency of operation in repetitive scenarios.

[0007] In summary, current technology lacks an intelligent interaction method that can sense the user's operational context, invoke AI functions with a single mouse button press, and switch between AI functions using the scroll wheel. Therefore, it is necessary to provide a context-aware intelligent button response method to address the aforementioned technical issues. Summary of the Invention

[0008] The technical problem to be solved by this invention is: how to quickly and accurately recommend and execute AI-assisted functions adapted to the current scenario based on the user's current operating context through a combination of mouse physical buttons and scroll wheel interaction, thereby reducing the cost of calling AI functions and improving the efficiency of human-computer interaction.

[0009] To achieve the above objectives, the present invention provides a context-aware intelligent key response method, the method comprising the following steps:

[0010] Step S1: Obtain current context information, which includes at least one of the current foreground application, the type of content selected by the user, and the characteristics of the selected content; wherein, the type of selected content includes at least one of text, image, table, code, and UI element, and the characteristics of selected content include at least one of language type, text length, and the domain to which the content belongs.

[0011] Step S2: Calculate the relevance score of each AI function based on the context information, and select the N functions with the highest scores as recommended functions. Each AI function includes at least one of the following: translation, polishing, proofreading, intelligent reply, intelligent screenshot, image processing, meeting recording, and audio / video transcription, where N is a preset number.

[0012] Step S3: In response to the user's first trigger operation on the mouse physical button, display the recommended function list in the form of a floating menu near the mouse cursor.

[0013] Step S4: In response to the user switching selections in the recommended functions by scrolling the mouse wheel and confirming the execution of the currently selected recommended function through the second trigger operation of the physical button.

[0014] Based on the above method, the present invention also has the following preferred embodiments:

[0015] Preferably, the method further includes obtaining the user's preferred function preset for a specific application or content type; when the context information matches the specific application or content type, the preferred function is used as the highest priority item of the recommended function, thereby achieving personalized adaptation.

[0016] Preferably, when the current foreground application is a chat application and the user selects text, the filtered recommendation function includes the Smart Reply function; the Smart Reply function is used to automatically generate reply content based on screen semantic understanding to assist the user in responding to messages quickly.

[0017] Preferably, when the current foreground application is a document editing application and the user selects text, the filtered recommended functions include polishing, proofreading, and translation functions to meet the core AI-assisted needs in document processing scenarios.

[0018] Preferably, when the user selects an image, the recommended filtering functions include image processing and AI drawing functions to adapt to the operational needs of image processing scenarios.

[0019] Preferably, the first trigger operation and the second trigger operation are short press operations on the same physical mouse button, and the floating menu is displayed at a preset offset position of the mouse cursor to avoid obscuring the content selected by the user.

[0020] Preferably, the relevance score is calculated using a multi-dimensional weight matrix. The dimensions of the weight matrix include application type weight, content type weight, and content feature weight. The weight values ​​for each dimension are dynamically adjusted based on historical usage data. Specifically, the dynamic adjustment employs the following strategy: the system records the user's historical call behavior for each AI function in various application scenarios, using the number of calls within a preset time window T (e.g., the last 30 days) as the frequency statistics; an exponential decay formula is used to time-weight the historical call frequency, i.e., W_adjusted = W_base × (1 + α × Σ(e^(-λ×Δt_i))), where α is the learning rate parameter, λ is the decay coefficient, and Δt_i is the time interval between the i-th call and the current moment; the weights of each dimension are updated based on the weighted frequency statistics, so that frequently used functions receive higher recommendation priority in their corresponding scenarios.

[0021] To achieve the above objectives, the present invention also provides a context-aware intelligent key response device applied to a computer system equipped with a mouse input device, including a context-aware module, an AI function filtering module, a floating menu interaction module, and an AI function execution module, wherein each module works together to implement the above method. To achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned context-aware intelligent key response method.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] (1) Significantly improved operation efficiency: By integrating AI function calls into the physical mouse button, users can trigger the AI ​​function floating menu with a single button press without leaving the current operation interface, greatly shortening the function call path.

[0024] (2) Context-adaptive recommendation: The system dynamically filters and recommends functions based on the current foreground application, selected content type and content characteristics, ensuring that users see the AI ​​functions most relevant to the current operation first and reducing invalid operations.

[0025] (3) Smooth and natural interaction: The interaction mode of using the scroll wheel to switch options and physical buttons for secondary confirmation is highly compatible with the user's existing mouse operation habits and has a low learning cost.

[0026] (4) Support for personalized configuration: By pre-setting preferred function mechanism, the system can learn and adapt to the user's personal operation preferences, further improving the efficiency of use in repetitive scenarios.

[0027] (5) Wide range of application scenarios: The method covers a variety of typical application scenarios such as chat, document editing, and image processing, and has strong versatility and scalability. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments are briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and therefore should not be regarded as a limitation on the scope.

[0029] Figure 1 This is a flowchart illustrating the overall process of the context-aware intelligent key response method described in this invention.

[0030] Figure 2 This is a schematic diagram of the AI ​​function screening and relevance scoring mechanism described in this invention.

[0031] Figure 3 This is a schematic diagram of the floating menu interaction interface described in this invention.

[0032] Figure 4 This is a schematic diagram of the user preferred function configuration and priority insertion mechanism described in this invention.

[0033] Figure 5 This is a schematic diagram illustrating a typical application scenario of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of this invention.

[0035] The method described in this invention runs on a computer system equipped with a mouse input device. Its core modules include: a context-aware module, an AI function filtering module, a floating menu interaction module, a preferred function configuration module, and an AI function execution module. These modules work together to achieve context-aware intelligent key response functionality.

[0036] Step S1: Obtaining Context Information

[0037] The context awareness module monitors system events in real time. When it detects that a user's physical mouse button has been pressed, it initiates a context information collection process, which includes:

[0038] (a) Foreground application identification: Obtain the identifier and name of the currently active application by calling the operating system interface (such as Windows' GetForegroundWindowAPI or macOS's NSWorkspace.shared.frontmostApplication).

[0039] (b) Selected content type detection: Obtain the currently selected content by the user through system clipboard monitoring or accessibility APIs (such as UIAutomation for Windows and Accessibility API for macOS); determine its type by analyzing the content format, including but not limited to: text (including plain text and rich text), images, tables, code, and UI elements.

[0040] (c) Selected content feature extraction: For text-type content, extract language type (e.g., Chinese, English), text length (number of characters / number of words), and content domain (determined by keywords or classification models, such as technical documents, legal documents, daily conversations, etc.); for image-type content, extract basic features such as image resolution and image source application.

[0041] Step S2: AI Function Filtering and Recommendation

[0042] Based on the context information obtained in step S1, the AI ​​function filtering module generates a recommended function list using the following mechanism:

[0043] (a) Relevance Scoring: A multi-dimensional weight matrix is ​​set for each preset AI function, including application type weight, content type weight, and content feature weight. The comprehensive relevance score of each AI function is calculated based on the current context information and the weights of each dimension. Specifically, the formula for calculating the comprehensive relevance score Score(f) of each AI function is: Score(f) = W_app × R_app(f) + W_content × R_content(f) + W_feature × R_feature(f), where W_app, W_content, and W_feature are the application type weight, content type weight, and content feature weight, respectively, and R_app(f), R_content(f), and R_feature(f) are the matching degree values ​​of AI function f in each dimension, with values ​​ranging from 0 to 1. For example, when the front-end application is a chat application and the selected content is Chinese text, let W_app=0.4, W_content=0.35, W_feature=0.25, and the Smart Reply function's R_app=1.0, R_content=0.9, R_feature=0.8. Then, the Score (Smart Reply) = 0.4×1.0 + 0.35×0.9 + 0.25×0.8 = 0.915. For the Translation function, R_app=0.6, R_content=0.8, R_feature=0.7, then the Score (Translation) = 0.4×0.6 + 0.35×0.8 + 0.25×0.7 = 0.695. After sorting by score, the Smart Reply function ranks first, and the Translation function ranks lower.

[0044] (b) Recommendation list generation: Sort by relevance score from high to low, and select the N AI functions with the highest scores (N is the system preset value, and the recommendation range is 3 to 5) as the recommended function list for this time.

[0045] (c) Priority handling of preferred features: If the user has already configured a preferred feature for the currently matched application or content type, then the preferred feature will be inserted at the beginning of the recommendation list, and the other recommended features will be moved to the end in turn.

[0046] Step S3: Display the floating menu

[0047] In response to the user's first trigger action on the physical mouse button (such as a short press), the floating menu interaction module pops up a semi-transparent floating menu near the current mouse cursor position, displaying recommended functions in a list format. The top-ranked recommended function is highlighted by default, with the remaining functions arranged sequentially. During the menu display, the system enters a function selection waiting state.

[0048] Step S4: Scroll wheel switching and button confirmation

[0049] When the floating menu is displayed:

[0050] (a) Users can cycle through the highlighted items in the recommended function list by scrolling the mouse wheel up or down; each scroll increment corresponds to one function item in the list.

[0051] (b) The user confirms the execution of the currently highlighted recommended function by a second trigger operation on the physical mouse button (such as a short press again); the system responds to the confirmation operation, closes the floating menu, and calls the corresponding AI function execution module to start the selected function.

[0052] Example 1: Smart reply function call in a chat application scenario

[0053] The user is using an instant messaging application (such as WeChat Work or DingTalk) and selects a Chinese text message sent by another user. The user clicks the physical mouse button, and the system detects that the current foreground application is a chat application and the selected content is Chinese text. After calculating the relevance score, the "Smart Reply" function ranks first. A floating menu pops up and highlights "Smart Reply" by default. The user does not need to switch using the scroll wheel; they can simply click the physical button again to confirm. The system calls the screen semantic understanding module to automatically generate candidate reply content based on the dialogue context and displays it in the input box for the user to edit and confirm.

[0054] Example 2: Using the polishing function in a document editing scenario

[0055] The user is editing a Chinese report in Microsoft Word and selects a section of technical description text. The user clicks the physical mouse button, and the system detects that the current foreground application is a document editing application and the selected content is Chinese text (content characteristic: technical field). The recommended function list is "Polishing," "Proofreading," and "Translation." A floating menu pops up, highlighting "Polishing" by default. If the user wishes to perform "Proofreading," scrolling down one position with the mouse wheel switches the highlight to "Proofreading." Clicking the physical button again confirms, and the system invokes the proofreading function to check the grammar and word choice of the selected text and outputs suggested revisions.

[0056] Example 3: Image processing function calls in an image processing scenario

[0057] The user is using an image viewer and selects an image. The user clicks the physical mouse button, and the system detects that the selected content is an image. The recommended function list shows "Image Processing" and "AI Drawing". A floating menu pops up, and the user switches to the target function using the scroll wheel and confirms the execution. The system then calls the corresponding AI function module to process the selected image.

[0058] Example 4: User Preferred Features Configuration Example

[0059] Users can configure the preferred feature for the "Text" content type in the "Google Chrome" application through the system settings interface. Afterward, whenever a user selects text in the Chrome browser and triggers a physical key press, the "Translate" function will always be displayed as the first item in the recommendation list, taking priority over the system relevance score, thus meeting the user's personalized usage needs.

[0060] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A context-aware intelligent key response method, applied to a computer system equipped with a mouse input device, characterized in that, include: Obtain current context information, which includes at least one of the following: current foreground application, user-selected content type, and selected content characteristics; Based on the context information, the relevance score of each AI function is calculated, and the N functions with the highest scores are selected as recommended functions. Each AI function includes at least one of the following: translation, polishing, proofreading, intelligent reply, intelligent screenshot, image processing, meeting recording, and audio / video transcription, where N is a preset number. In response to the user's first trigger operation on the physical mouse button, the recommended function is displayed in the form of a floating menu near the mouse cursor; The system responds to the user switching between recommended functions by scrolling the mouse wheel and confirming the execution of the currently selected recommended function through a second trigger operation of the physical button.

2. The method according to claim 1, characterized in that, The selected content type includes at least one of the following: text, image, table, code, and UI element; the selected content features include at least one of the following: language type, text length, and content domain.

3. The method according to claim 1, characterized in that, Also includes: Get the user's preferred features preset for a specific application or content type; When the context information matches the specific application or content type, the preferred function is selected as the highest priority item for the recommended function.

4. The method according to claim 1, characterized in that, Also includes: When the current foreground application is a chat application and the user selects text, the recommended functions include the Smart Reply function, which automatically generates reply content based on screen semantic understanding; when the current foreground application is a document editing application and the user selects text, the recommended functions include polishing, proofreading, and translation functions; when the user selects an image, the recommended functions include image processing and AI drawing functions.

5. The method according to claim 1, characterized in that, The first trigger operation and the second trigger operation are short press operations on the same physical mouse button, and the floating menu is displayed at a preset offset position of the mouse cursor to avoid obscuring the content selected by the user.

6. The method according to claim 1, characterized in that, The relevance score is calculated using a multi-dimensional weight matrix. The dimensions of the weight matrix include application type weight, content type weight, and content feature weight. The weight values ​​of each dimension are dynamically adjusted based on historical usage data. The dynamic adjustment includes: statistically analyzing the historical call frequency of each AI function by users in various application scenarios, using the call frequency within a preset time window as the frequency statistics value, weighting the historical call frequency using an exponential decay method, and updating the weight values ​​of each dimension based on the weighted frequency statistics value.

7. A context-aware intelligent key response device, applied to a computer system equipped with a mouse input device, characterized in that, include: The context awareness module is used to acquire current context information, which includes at least one of the current foreground application, the type of content selected by the user, and the characteristics of the selected content. The AI ​​function filtering module is used to filter at least one recommended function from a set of preset AI functions based on the context information. The set of AI functions includes at least one of translation, polishing, proofreading, intelligent feedback, intelligent screenshot, image processing, meeting recording, and audio / video transcription. The filtering includes calculating the relevance score of each AI function based on the context information and selecting the N functions with the highest scores as recommended functions, where N is a preset number. The floating menu interaction module is used to display the recommended functions in a floating menu near the mouse cursor in response to the user's first trigger operation on the mouse physical button, and to confirm the execution of the currently selected recommended function by the user scrolling the mouse wheel or by the second trigger operation on the physical button. The AI ​​function execution module is used to call and execute the currently selected recommended function.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method of any one of claims 1 to 6.

Images