Browser intelligent search method, computer device, storage medium and program product
By introducing intelligent search plug-ins into the browser, automatically obtaining and classifying search content and calling AI processing modules for search, the existing browser's AI search function is solved, and efficient and accurate search experience and security are achieved sex.
Patent Information
- Application Number
- CN202410967318.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing browsers have limitations in integrating artificial intelligence (AI) technology and cannot provide seamless AI search query solutions, resulting in users needing to frequently switch tabs or windows during browsing, increasing operational complexity and extending information retrieval time. , and the integrated AI function is single, which cannot meet the needs of diversified information search.
Access AI to the current page of the browser through browser plug-in, provide intelligent search methods, automatically obtain and classify search content, call corresponding AI processing modules to search, display search results, and reduce users between different websites or search engines The need for switching improves search efficiency and accuracy.
A seamless and consistent search experience is achieved, reducing user operation complexity, improving search efficiency and relevance and accuracy of results, and reducing the risk of malicious websites and phishing attacks.
Smart Images

Figure CN118964711A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of Internet technology, and in particular to a browser intelligent search method, a computer device, a storage medium, and a program product. Background Art
[0002] In the current digital age, users have increasing expectations for the speed and accuracy of online information retrieval. However, existing browsers have limitations in integrating artificial intelligence (AI) technology, and most browsers have not yet provided a seamlessly integrated AI search query solution. This results in users being unable to directly use AI technology during browsing, and having to open additional web pages, jump to different AI service platforms, and perform tedious operations such as registration and login before they can access AI search tools. Users have to frequently switch between multiple tabs or windows, which not only increases the complexity of operations, but also prolongs the time for information retrieval.
[0003] In addition, even if the browser integrates certain AI functions, these technologies are often single-function and cannot meet users' diverse information search needs. For example, users may need to open multiple AI service web pages separately to perform different tasks such as language translation, image recognition or data analysis. This decentralized usage mode reduces efficiency and affects user experience. Summary of the invention
[0004] In view of this, the embodiments of the present disclosure provide a browser intelligent search method, a computer device, a storage medium, and a program product, which can use a browser plug-in to access AI to the current page of the browser, and perform AI search queries more quickly and conveniently to obtain accurate search results while reducing the operational complexity of users frequently switching between multiple tabs or windows.
[0005] In a first aspect, the embodiments of the present disclosure provide a browser intelligent search method, which adopts the following technical solutions:
[0006] Building browser plugins;
[0007] Creating a user interface on a current page of the browser based on the browser plug-in;
[0008] Get the user's search content in the browser;
[0009] Classifying the search content based on the browser plug-in to obtain the type of the search content;
[0010] Based on the type, a corresponding AI processing module is called to search the search content to obtain search results;
[0012] The search results are displayed on the user interface.
[0013] Optionally, creating a user interface on a current page of the browser based on the browser plug-in includes:
[0014] A pop-up window or a content script of the browser plug-in is used to create a user interface on the current page.
[0015] Optionally, obtaining the search content of the user in the browser includes:
[0016] Adopt any one of the first rule, the second rule, the third rule, the fourth rule, the fifth rule, and the sixth rule to obtain the search content;
[0017] The priorities of the first rule, the second rule, the third rule, the fourth rule, the fifth rule, and the sixth rule are reduced in sequence;
[0018] The user interface includes a first input field; the first rule includes:
[0019] In response to a search request initiated by a user in the user interface, obtaining search content in the first input field;
[0020] The browser includes a second input field; the second rule includes:
[0021] In response to a search request initiated by a user in the browser, obtaining input content in the second input field, and copying the input content into the input field of the user interface as the search content;
[0022] The third rule includes:
[0023] Monitor the user's mouse selection operation and obtain the selected content;
[0024] Copying the circled content into the input field of the user interface as the search content;
[0025] The fourth rule includes:
[0026] Adding an AI search item in the right-click menu, in response to a user clicking operation on the AI search item, obtaining a specified image selected by the user, and copying the specified image to an input field of the user interface as the search content;
[0027] The fifth rule includes:
[0028] Detect whether audio is played in the current page;
[0029] If the current page plays audio, convert the audio into text content, and obtain the search content based on the text content;
[0030] If the current page does not play audio, then detect whether the current page plays video;
[0031] If the current page plays a video, the metadata of the video is obtained, and YY+242143P is obtained based on the metadata.
[0032] The search content;
[0033] The sixth rule includes:
[0034] Get the static time of the user's mouse;
[0035] When the static time reaches a time threshold, the screen content under the current position of the user's mouse is obtained;
[0036] The screen content is copied into an input field of the user interface as the input field content.
[0037] Optionally, acquiring the search content based on the text content includes:
[0038] Converting the text content into a target language text;
[0039] The most frequent words are obtained from the target language text, where the most frequent words are the search content.
[0040] Optionally, acquiring the search content based on the text content includes:
[0041] Converting the text content into a target language text;
[0042] Acquire a plurality of first keywords from the target language text;
[0043] Generate a plurality of first Prompt words based on the plurality of first keywords;
[0044] In response to the user's request for the current first Prompt word, the current first Prompt word is used as the search content.
[0045] Optionally, the acquiring metadata of the video and acquiring the search content based on the metadata includes:
[0046] Obtaining a non-advertising segment of the video;
[0047] Detect the non-advertising segment to obtain the characters and objects that appear most frequently in the non-advertising segment;
[0048] Get the video uploader and the name of the video background music;
[0049] The person, the object, the video uploader, and the name of the video background music are all taken as a metadata;
[0050] Generate multiple second prompts based on multiple metadata;
[0051] In response to the user's request for the current second Prompt word, the current second Prompt word is used as the search content.
[0052] Optionally, the calling a corresponding AI processing module based on the type to search the search content and obtain search results includes:
[0053] When the type is text, calling a text AI processing module to pre-process the search content;
[0054] Extract keywords from the preprocessed search content to obtain a second keyword;
[0055] Perform semantic understanding on the preprocessed search content to obtain semantic features;
[0056] Acquire the user's query intention based on the second keyword, the semantic feature and a preset classification model;
[0057] Based on the query intent, a corresponding search engine API is called to perform a search and obtain the search results.
[0058] Optionally, the calling a corresponding AI processing module based on the type to search the search content and obtain search results includes:
[0059] When the type is a picture, calling the picture AI processing module to compress the search content;
[0060] Preprocess the compressed search content;
[0061] Perform feature extraction on the preprocessed search content to obtain multiple image features;
[0062] Setting image labels based on multiple image features, where one image label corresponds to several image features;
[0063] Get the third prompt word;
[0064] The search result is obtained based on the third Prompt word, the multiple image features, the image tag and a preset search index library.
[0065] Optionally, the calling a corresponding AI processing module based on the type to search the search content and obtain search results includes:
[0066] When the type is video, in response to a user's screenshot instruction for playing a video on the current page, obtaining a video screenshot;
[0067] The video screenshot is converted into a wallpaper, where the wallpaper is the search result.
[0068] In a second aspect, the embodiments of the present disclosure further provide a browser intelligent search system, which adopts the following technical solutions:
[0069] Plugin building module, used to build browser plugins;
[0070] An interface creation module, used to create a user interface on the current page of the browser based on the browser plug-in;
[0071] Content acquisition module, used to obtain the user's search content in the browser;
[0072] A type acquisition module, used to classify the search content based on the browser plug-in and acquire the type of the search content;
[0073] A result acquisition module, used to call the corresponding AI processing module based on the type to search the search content and obtain search results;
[0074] A result display module is used to display the search results on the user interface.
[0075] In a third aspect, the embodiments of the present disclosure further provide a computer device, which adopts the following technical solution:
[0076] The computer device comprises:
[0077] YY+242143P
[0078] at least one processor; and,
[0079] a memory communicatively connected to the at least one processor; wherein,
[0080] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the above-mentioned browser intelligent search methods.
[0081] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any of the above-mentioned browser intelligent search methods.
[0082] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program / instruction, which implements the steps of any of the above methods when executed by a processor.
[0083] The browser intelligent search method provided by the embodiment of the present disclosure directly integrates the search function through the browser plug-in, so that the user can search without leaving the current page, providing a seamless and consistent search experience. Among them, the browser plug-in is a software program that can add additional functions and features to the web browser. Through the browser plug-in, the user's search content in the browser is obtained automatically or at the request of the user. The search content is processed instantly by the AI processing module, and the user can quickly obtain the search results, which improves the search efficiency. In this process, the search content is also automatically classified, and the appropriate AI processing module is selected according to the type of search content for efficient search. This processing method that supports different types of search content can ensure the relevance and accuracy of the search results, so that users can obtain a more customized search experience. The above-mentioned intelligent search method based on browser plug-ins can reduce the need for users to switch between different websites or search engines, reduce user operations, save user time and reduce the demand for computing resources, and can better control the search process and reduce the risk of malicious websites and phishing attacks.
[0084] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following preferred embodiments are specifically cited and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0085] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0086] Figure 1 A flowchart of a browser intelligent search method provided by an embodiment of the present disclosure;
[0087] Figure 2 A schematic diagram of a pop-up window user interface provided by an embodiment of the present disclosure;
[0088] Figure 3 A schematic diagram of a user interface of a UI injection method provided in an embodiment of the present disclosure;
[0089] Figure 4A flowchart of a text-based search method for content search provided by an embodiment of the present disclosure;
[0090] Figure 5 A schematic diagram of a flow chart of a method for searching content by image type provided in an embodiment of the present disclosure;
[0091] Figure 6 A block diagram of the principle of a browser intelligent search system provided by an embodiment of the present disclosure;
[0092] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0093] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.
[0094] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.
[0095] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.
[0096] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.
[0097] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.
[0098] Reference Figure 1 The present disclosure provides a browser intelligent search method, comprising the following steps:
[0099] S1: Build browser plugins;
[0100] S2: Create a user interface on the current page of the browser based on the browser plug-in;
[0101] S3: Get the user's search content in the browser;
[0102] S4: classify the search content based on the browser plug-in to obtain the type of the search content;
[0103] S5: Based on the type, the corresponding AI processing module is called to search the search content and obtain search results;
[0104] S6: Display the search results on the user interface.
[0105] The browser intelligent search method disclosed in the present invention directly integrates the search function through the browser plug-in, so that the user can search without leaving the current page, providing a seamless and consistent search experience. Among them, the browser plug-in is a software program that can add additional functions and features to the web browser. Through the browser plug-in, the user's search content in the browser is obtained automatically or at the request of the user. The search content is processed instantly by the AI processing module, and the user can quickly obtain the search results, which improves the search efficiency. In this process, the search content is also automatically classified, and the appropriate AI processing module is selected according to the type of search content for efficient search. This processing method that supports different types of search content can ensure the relevance and accuracy of the search results, so that users can obtain a more customized search experience. The above-mentioned intelligent search method based on browser plug-ins can reduce the need for users to switch between different websites or search engines, reduce user operations, save user time and reduce the demand for computing resources, and can better control the search process and reduce the risk of malicious websites and phishing attacks.
[0106] In summary, using a browser plug-in to connect AI to the current page of the browser means that users do not need to frequently switch between multiple tabs or windows when performing AI searches, which reduces the complexity of operations. Users can perform AI search queries more quickly and obtain more accurate search results. In order to realize the search function of the full-name application intelligent AI, it can learn, reason, understand and perform tasks. It covers technologies such as machine learning, deep learning, natural language processing, computer vision, etc., and can be applied to language translation, image recognition, smart assistants and other fields.
[0107] In S1, the present disclosure adopts the Chrome manifest V3 standard to design and implement a browser plug-in. Of course, different standards can be adopted to create a browser plug-in according to different browsers.
[0108] A manifest file (manifest.json file) is configured for the browser plug-in, which specifies the user interface form of the browser plug-in and the key permissions required for the browser plug-in to run. Among them, the key permissions include at least one of the webRequest permission, Storage permission, activeTab permission, and contextMenus permission.
[0109] Among them, the webRequest permission is used by the browser plug-in to accurately monitor the network requests issued by the browser and the responses received. This function is crucial for understanding and optimizing the web page loading process; the Storage permission is used to store user identification records, thereby providing a personalized user experience and cross-session data retention; the activeTab permission is used to allow the browser plug-in to temporarily access the currently active tab and perform operations on the currently active tab to achieve interaction and content operations within the current page of the browser; the contextMenus permission is used to use the right-click menu to search for user-selected content, that is, to enable users to quickly access the search function through the right-click menu to achieve instant search of selected content.
[0110] In S2, the present disclosure provides two solutions, a pop-up window and a UI injection method, to implement the user interface design of the browser plug-in. One of them is selected according to the needs and configured to the manifest file, and a user interface is created on the current page of the browser through the manifest file.
[0111] Reference Figure 2The schematic diagram of the user interface of the pop-up window is shown. For the user interface design of the first browser plug-in, a popup is specifically used to build an intuitive plug-in icon, and the plug-in icon is configured in a fixed display position on the browser toolbar. The plug-in icon can not only serve as the visual logo of the plug-in, but also control the display and closing of the user interface. The user clicks the plug-in icon with the mouse to trigger a pop-up window, which is used to display the user interface, including user interaction elements and information display. When the user completes the operation or wants to hide the user interface, the pop-up window is closed by clicking the plug-in icon again with the mouse, so that the user interface is no longer displayed. The simplicity of this click operation ensures that the user interface is within reach when the user needs it, and does not interfere with the user's browsing experience when it is not needed. In addition, when the user does not perform any mouse click operation, the user interface will remain hidden, further ensuring the neatness of the user interface and the user's focus on browsing. In this way, the user experience is improved, allowing users to flexibly access and use browser plug-ins according to their personal needs, while keeping the browser interface clean and interference-free.
[0112] Furthermore, in order to attract users to use the AI search function, when the user is detected clicking the browser's built-in search button or the user uses the mouse to circle the content, an indicator light or other visual indicator is displayed on the plug-in icon to remind the user that they can use the AI search function by clicking the plug-in icon.
[0113] Reference Figure 3 The schematic diagram of the user interface of the UI injection method is shown. For the user interface design of the second browser plug-in, the UI injection method of Content Script is specifically adopted to directly add a customized user interface to the current page of the browser. Through Content Script, the browser plug-in can inject HTML, CSS and JavaScript code into the user's current page, thereby enhancing the interactivity of the page. Moreover, Content Script allows the browser plug-in to embed new controls in the current page, such as buttons, input boxes or other UI elements. These controls can trigger the functions of the browser plug-in, providing users with a more flexible and dynamic interactive experience.
[0115] In S3, the present disclosure provides several rules for obtaining the search content of the user in the browser, wherein the first rule (user interface triggering) includes: the user interface includes at least one of its own input bar, search button and display area. In order to facilitate subsequent distinction, the input bar of the user interface is recorded as the first input bar, and the search button of the user interface is recorded as the first search button. The user enters the search content in the first input bar through the terminal device and clicks the first search button, triggering a search request in the user interface. In response to the search request initiated by the user in the user interface, the browser plug-in obtains the search content in the first input bar for later AI search. This method provides a direct expression of user intent, allowing users to initiate search requests through clear search actions.
[0116] The second rule (browser search button triggering) includes: the browser usually has an input bar and a search button, the browser input bar is recorded as the second input bar, and the browser search button is recorded as the second search button. When the user enters content in the second input bar and then clicks the second search button to search, a search request in the browser is triggered, and the browser plug-in will monitor this action, respond to the search request initiated by the user in the browser, automatically obtain the input content entered by the user in the second input bar, and the browser plug-in will copy the input content to the first input bar as the search content.
[0117] The third rule (mouse circle selection trigger type) includes: monitoring the circle selection operation of the user's mouse, obtaining the circled content, and copying the circled content to the first input field as the search content. Specifically, the browser plug-in detects the circled content by monitoring the user's mouse operation. When the user uses the mouse to circle text or drag to select content on the web page, the browser plug-in automatically copies the circled content (selected text) to the first input field as the search content until it is detected that the circle selection operation is completed.
[0118] The fourth rule (right-click menu AI search item trigger) includes: adding an AI search item to the right-click menu, in response to the user's click operation on the AI search item, obtaining the specified image selected by the user, and copying the specified image to the input bar of the user interface as the search content. Specifically, in the settings of the browser plug-in, add a dedicated right-click menu item, name the right-click menu item AI search, and when the user right-clicks the mouse on the current page, in addition to displaying the standard context menu, the added AI search item is also displayed. When browsing the web, the user can right-click the specified image with the mouse, and then click the AI search item in the right-click menu. The browser plug-in obtains the specified image selected by the user through the click event of the right-click menu item, and copies the specified image to the first input bar as the search content.
[0119] The fifth rule (audio and video playback trigger) includes: detecting whether audio is played in the current page. If audio is played, converting the audio into text content, and obtaining the search content based on the text content; if audio is not played, detecting whether video is played in the current page. If video is played, obtaining the metadata of the video, and obtaining the search content based on the metadata; if video is not played, it means that there is no video type search content at present.
[0120] When the audio is detected playing in the current page, the audio AI processing module is called. The audio AI processing module encapsulates multi-language translation AI and provides the API of multi-language translation AI. Get the target language set by the user and translate the text content into the target language text.
[0121] In one implementation, the TF-IDF algorithm is used to obtain the keyword with the highest frequency of occurrence from the target language text, and the keyword is recorded as the highest frequency word, and the highest frequency word is the search content.
[0122] In another implementation scheme, a TF-IDF algorithm is used to obtain multiple first keywords from the target language text, and a first Prompt word is automatically generated for each first keyword. The generated multiple first Prompt words are displayed in a display area. According to the user's own needs, the user uses a terminal device or a mouse to click on the current first Prompt word among the multiple first Prompt words, thereby triggering the user's request for the current first Prompt word. In response to the user's request for the current first Prompt word, the current first Prompt word is used as the search content.
[0123] When it is detected that a video is being played on the current page, the video AI processing module is called, and the video AI processing module uses a preset method to identify advertising clips and non-advertising clips in the video, wherein the preset method uses any one of an AI automatic video editing tool, a video advertising recognition method based on image fingerprints, a highlight detection method, and a streaming detection method. Object recognition technology is used to obtain the objects with the highest frequency of non-advertising clips, and person recognition technology is used to obtain the people with the highest frequency of non-advertising clips, wherein the object recognition technology uses any one of deep learning-based image recognition, object detection algorithm, scene understanding, and feature-based recognition, and the person recognition technology uses any one of face recognition, social network analysis, person re-recognition, and multimodal recognition. The video uploader and the name of the video background music are also obtained, and the person, the object, the video uploader, and the name of the video background music are all used as metadata, and a second prompt word is automatically generated for each metadata, and the generated multiple second prompt words are displayed in the display area. According to the user's own needs, the user uses a terminal device or a mouse to click on the current second Prompt word among multiple second Prompt words, triggering the user's request for the current second Prompt word. In response to the user's request for the current second Prompt word, the current second Prompt word is used as the search content.
[0124] Specifically, by detecting the <audio>The element determines whether the audio is playing. The code is as follows:
[0125] / / Check if there are any <audio>Elements are playing
[0126] YY+242143P
[0127] var audioPlaying = false;
[0128] var audioElements=document.querySelectorAll('audio');
[0129] audioElements.forEach(function(audio){
[0130] if(!audio.paused){
[0131] audioPlaying = true;
[0132] console.log('The current page is playing audio');
[0133] / / Convert audio to text and copy to search input
[0134] / / Example: convertAudioToText(audio);
[0135] return; / / If the audio being played is found, exit the loop
[0136] }
[0137] });
[0138] if(!audioPlaying){
[0139] console.log('The current page does not play audio');
[0140] }
[0141] By detecting the <video>The element determines whether the video is playing. The code is as follows:
[0142] / / Check if there are any <video>Elements are playing
[0143] var videoPlaying = false;
[0144] var videoElements=document.querySelectorAll('video');
[0145] videoElements.forEach(function(video){
[0146] if(!video.paused){
[0147] videoPlaying = true;
[0148] console.log('The current page is playing a video');
[0149] / / Extract video information and copy to AI input
[0150] / / Example: extractVideoInfo(video);
[0151] return; / / If the video being played is found, exit the loop
[0152] }
[0153] });
[0154] if(!videoPlaying){
[0155] console.log('The video is not playing on the current page');
[0156] }
[0157] YY+242143P
[0158] The above code example uses document.querySelectorAll to detect whether the current page is playing audio and video, combining JavaScript and HTML5 media API.
[0159] The sixth rule (timeout trigger) includes: setting a time threshold to determine whether the user's mouse is in a stationary state. This time threshold can be adjusted according to user needs to adapt to different usage scenarios. When the stationary time of the mouse reaches the time threshold, the browser plug-in will capture the screen content under the current position of the mouse, the screen content is at least one of text and picture, and copy the screen content to the input bar of the user interface as the input bar content.
[0160] Among them, the priorities of the first rule, the second rule, the third rule, the fourth rule, the fifth rule, and the sixth rule are in descending order, the sixth rule has the lowest priority, the fifth rule is the second, and so on, and the first rule has the highest priority. When multiple rules are triggered at the same time, the rule with the highest priority is executed.
[0161] Furthermore, users are allowed to set automatic search. When automatic search is turned on, for the above-mentioned methods of obtaining search content, users do not need to click the first search button. The browser plug-in directly performs AI search on the obtained search content. This method improves the convenience of search; when automatic search is turned off, users can modify the search content of the first input bar. After the modification, click the first search button. The browser plug-in performs AI search on the modified search content. This method improves the flexibility of search.
[0162] Furthermore, after obtaining the search content, it is detected whether there is a user ID. If there is a user ID, the search history record is returned to the user terminal based on the user ID; if the user ID is not detected, the user is prompted to register. After the user successfully registers, a new user ID is generated for him / her to provide personalized services and record management
[0163] In S4, the browser plug-in classifies the search content and obtains the type of the search content, which includes at least one of text, picture, audio and video. The type of search content obtained by the above-mentioned user interface trigger, browser search button trigger and mouse circle trigger is text, the type of search content obtained by the above-mentioned right-click menu AI search item trigger is picture, the type of search content obtained by the above-mentioned audio and video playback trigger is audio or video, and <audio>Elements and <video>The element can specifically determine whether the type is audio or video, and the type of the search content obtained through the timeout trigger is at least one of text and picture.
[0164] In S51, the corresponding AI processing module is called based on the type to search the search content and obtain search results.
[0165] For text-based search content, the text AI processing module is called to process the search content and obtain search results. The text AI processing module integrates multiple search engines and provides a search engine API. Figure 4 The flowchart of the search method for text type search content is shown. The following are the steps for the text AI processing module to process the search content:
[0166] S511: pre-processing the search content;
[0167] S512: extracting keywords from the preprocessed search content to obtain a second keyword;
[0168] S513: Performing semantic understanding processing on the preprocessed search content to obtain semantic features;
[0169] S514: Obtaining the user's query intention based on the second keyword, the semantic feature, and the preset classification model;
[0170] S515: Based on the query intent, the corresponding search engine API is called to perform a search and obtain search results.
[0171] In S511 , the preprocessing of the text-type search content includes text cleaning and word segmentation, and supports multiple languages.
[0172] In S512, a TF-IDF algorithm is used to extract keywords from the search content to obtain a plurality of second keywords.
[0173] In S513, a preset BERT model is used to perform semantic understanding processing on the search content. The BERT model can capture the contextual relationship in the search content, understand the deep meaning of the vocabulary, perform deep semantic analysis, and obtain the semantic features of the search content.
[0174] In S514, the second keyword and the semantic feature are fused to obtain a comprehensive feature, which is used as an input to the classification algorithm. The classification algorithm processes and analyzes the comprehensive feature to identify the user's query intention. Specifically,
[0175] Among them, the cross-validation method is used to select the optimal classification model, specifically including: selecting logistic regression, support vector machine, random forest and neural network architecture as the initial classification algorithms respectively, training these classification models respectively on the data set of known query intent, and after training, using the validation set to evaluate these classification models, and selecting the best performance as the final classification model according to the evaluation results, and fine-tuning the final classification model according to the evaluation results. After the fine-tuning is completed, the final classification model will be deployed in the actual environment.
[0176] In S515, different query intents are suitable for different search engines to perform searches. Therefore, according to the acquired query intent, the corresponding search engine API is called to perform searches on the search engine that best matches the query intent and obtain search results.
[0177] Collect user feedback on search results, such as satisfaction, relevance evaluation, etc., and optimize keyword extraction, semantic understanding and query intent identification processes based on user feedback.
[0178] Furthermore, for text-type search content, a sensitive word filtering strategy is formulated to ensure that search results do not contain sensitive content that violates laws and regulations, social customs or platform policies, and to avoid the risk of returning results. Specifically, a sensitive word library is constructed, and words that violate these standards are collected based on standards such as laws and regulations, social customs and platform policies, and these words are stored in the sensitive word library. The management of the sensitive word library will adopt a dynamic update mechanism to adapt to social changes and policy adjustments. A preset matching algorithm is used to match the preprocessed search content with the sensitive word library to identify sensitive words in the preprocessed search content, and the identified sensitive words are blocked or deleted or replaced with preset symbols (such as asterisks). Among them, the matching algorithm can select any one of exact matching, fuzzy matching and root matching. The selected matching algorithm can match sensitive words to different degrees, including exact matching, fuzzy matching of similar word forms, and root-based variant recognition.
[0179] In order to improve the performance of sensitive word filtering, two efficient data structures, Trie tree (also known as prefix tree or dictionary tree) and Aho-Corasick automaton, can also be used. Trie tree will be used for fast retrieval and prefix matching, while Aho-Corasick automaton will realize simultaneous search of multiple second keywords and optimize the fallback and transfer in the search process. Specifically, Trie tree will store the vocabulary in the sensitive word library and allow fast prefix matching and automatic completion. In the filtering process, Trie tree will help browser plug-ins quickly locate and identify potential sensitive words. The Aho-Corasick algorithm will be used to simultaneously detect multiple sensitive words in the preprocessed search content. By constructing an automaton structure containing a failure pointer, the algorithm will guide the browser plug-in to quickly jump to the possible correct path when encountering a mismatch, which is used to guide the rapid fallback and transfer in the matching process, improve the efficiency and accuracy of filtering, and ensure the comprehensiveness and efficiency of the search.
[0180] For image-type search content, call the image AI processing module to process the search content and obtain search results. Figure 5 The following is a flowchart of the search method for searching content by image type. The following are the steps for the image AI processing module to process the search content:
[0181] S521: compressing the search content;
[0182] S522: pre-processing the compressed search content;
[0183] S523: extracting features from the preprocessed search content to obtain multiple image features;
[0184] S524: Setting an image label based on multiple image features, where one image label corresponds to multiple image features;
[0185] S525: Obtain the third Prompt word;
[0186] S526: Obtain search results based on the third Prompt word, multiple image features, image tags, and a preset search index library.
[0187] In S521, the search content is essentially a copy of the specified image, so compressing the search content can reduce the file size, speed up subsequent processing, and reduce storage and transmission costs.
[0188] In S522, preprocessing of the search content of the picture type includes operations such as adjusting the image size, normalizing pixel values, and data enhancement to adapt to the input requirements of the deep learning model.
[0189] In S523, the preprocessed search content is input into a pre-trained deep learning model, such as VGG or ResNet, to extract deep features of the preprocessed search content, to achieve accurate recognition of objects, scenes and faces, and these extracted features are recorded as image features.
[0190] Among them, the VGG (Visual Geometry Group) network is a deep convolutional neural network (CNN) architecture that uses multiple consecutive 3x3 convolutional layers to build. Compared with the larger filter size used in other networks, this design reduces the number of parameters while maintaining strong expressive power. VGG has two configurations, VGG16 and VGG19, with 16 and 19 weight layers respectively. These networks are used for image classification, object detection, and image feature extraction tasks to complete automatic labeling of image labels. Another contribution of the VGG network is its standardized architectural layout, that is, consecutive convolutional layers are followed by maximum pooling layers, and there are several fully connected layers at the end for classification output. This design principle has influenced many subsequent CNN structure designs and improved overall performance.
[0191] Among them, ResNet (Residual Network) is a deep convolutional neural network architecture that solves the problems of gradient vanishing and degradation in deep neural network training, allowing the network to easily reach a depth of hundreds or even thousands of layers while maintaining excellent training performance and generalization capabilities, and extracting deep image features.
[0192] In S524, an automatic labeling algorithm is used to generate descriptive labels of the image, referred to as image labels, such as "sunset", "beach", "portrait", etc., based on the image features.
[0193] In S525, the third Prompt word (AI model prompt word) can be input by the user, or the browser plug-in automatically generates the third Prompt word according to the user's search history or preference.
[0194] In S526, several search tags are determined from multiple image tags based on the third Prompt word. For example, there are three image tags in total, the first image tag is beach, the second image tag is sunset, and the third image tag is clouds. The third Prompt word is "Detect beach sunset scenes similar to the specified picture" input by the user, then two search tags are determined, the first search tag is beach, and the second search tag is sunset.
[0195] The preset search index library is constructed based on pre-stored system images, which contains multiple indexes and system image features corresponding to the indexes (the image features here are extracted based on the system images, and the index is the label of the system image).
[0196] Traverse all search tags, record the search tag being traversed as the current search tag, match the current search tag with the index in the search index library, record the index that successfully matches the search tag as the current index, and obtain the system image features corresponding to the current index.
[0197] Based on a preset similarity algorithm, the similarity between the image feature corresponding to the current search tag and the system image feature corresponding to the current index is calculated, and the similarity is recorded as the first similarity. The similarity algorithm may use Euclidean distance, cosine similarity or other similarity measurement methods.
[0198] After all search tags are traversed, the first similarities belonging to the same system image are weighted averaged to obtain the second similarity between the search content and the system image. All the obtained second similarities are sorted from large to small, and the system images corresponding to the first K second similarities are selected as search results. The value of K can be set by the user or automatically by the browser plug-in.
[0199] Furthermore, you can define additional Prompt return content examples so that AI results can be returned according to the example format standard, reducing the chances of inaccurate front-end page display caused by incorrect formats.
[0200] For audio-type search content, the audio AI processing module is called. The audio AI processing module can directly perform AI search according to the search content to obtain search results.
[0201] For the search content of video type, the video AI processing module is called, and the video AI processing module can directly perform AI search according to the search content to obtain the search results. Furthermore, when the user is watching the video being played on the current page, he may think that a certain scene is beautiful and want to make it into a wallpaper. The user can associate the screenshot function of the video software or the screenshot function of other software or devices in the settings of the browser plug-in. The user can also turn on the screenshot function of the browser plug-in itself, and then take a screenshot of the video playing on the current page, triggering the screenshot instruction. In response to the user's screenshot instruction for the video playing on the current page, the video AI processing module obtains the video screenshot and edits the video screenshot, including adjusting the color tone, cropping or adding filters, removing watermarks when necessary, etc., thereby converting the video screenshot into a wallpaper, which is the search result.
[0202] In S6, you can choose to return search results in JSON format. The code example is as follows:
[0203] {"title":"","description":"","url":"","contentList":[{"content":"","img_url":"","code":{"language":"","data":""}}]}
[0204] The search results are displayed in the display area of the user interface. Users can choose the display effect of the search results in the display area. The first is that the display area only displays the current search results; the second is that the search content and search results are presented in the display area in the form of a dialogue. The display area provides a copy function, allowing users to copy the search results with one click.
[0205] Furthermore, the browser intelligent search method disclosed in the present invention also provides offline experience, cached content, and push notification functions. Among them, offline experience and cached content are local caches of the user's historical search content and historical search results, which are cached in real time using the LocalStorage method. For the cached content, text export and import functions are provided, which are stored and imported according to the search standard format, and Service Workers are used to create an offline experience, so that users can also check historical search content and historical search results offline without a network.
[0206] Push notification is an advanced network feature that allows browser plug-ins to send timely information to users when the user is not interacting with the current page or the browser is inactive. This feature is implemented through a standardized API provided by the browser, greatly enhancing the real-time interactive capabilities of web applications. In particular, in search applications, the push notification mechanism is designed to optimize the delayed return of search results or long-term batch processing. When the search results are fully generated and successfully returned, the browser plug-in will automatically trigger the browser's built-in notification service to remind users to check their search results in a timely manner. This mechanism not only improves the user experience, but also ensures that users will not miss any important search feedback even when multitasking or switching applications. Through carefully designed notification content and timing, push notifications have become an effective bridge to connect users with services and deliver key information.
[0207] Reference Figure 6 The present disclosure provides a browser intelligent search system, comprising:
[0208] A plug-in building module 101 is used to build a browser plug-in;
[0209] The interface creation module 102 is used to create a user interface on the current page of the browser based on the browser plug-in;
[0210] Content acquisition module 103, used to acquire the search content of the user in the browser;
[0211] A type acquisition module 104, used to classify the search content based on the browser plug-in and acquire the type of the search content;
[0212] A result acquisition module 105 is used to call a corresponding AI processing module based on the type to search the search content and obtain search results;
[0213] The result display module 106 is used to display the search results on the user interface.
[0214] The various variations and specific examples of the browser intelligent search method provided above are also applicable to the browser intelligent search system provided in the present disclosure. Through the above detailed description of the browser intelligent search method, those skilled in the art can clearly know the implementation method of the browser intelligent search system. For the sake of brevity of the specification, it will not be described in detail here.
[0215] The computer device according to the embodiment of the present disclosure includes a memory and a processor. The memory is used to store non-temporary computer-readable instructions. Specifically, the memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may, for example, include a read-only memory (ROM), a hard disk, a flash memory, etc.
[0216] The processor may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory, so that the computer device executes all or part of the steps of the browser intelligent search method of each embodiment of the present disclosure.
[0217] Those skilled in the art should be able to understand that in order to solve the technical problem of how to obtain a good user experience, the present embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the protection scope of the present disclosure.
[0218] like Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure is shown, which is a schematic diagram of the structure of a computer device suitable for implementing the embodiment of the present disclosure. Figure 7 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0219] like Figure 7 As shown, the computer device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer device are also stored. The processor, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.
[0220] Typically, the following devices can be connected to the I / O interface: input devices such as sensors or visual information acquisition devices; output devices such as display screens; storage devices such as tapes, hard disks, etc.; and communication devices. The communication device can allow the computer device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Figure 7 A computer device having various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.
[0221] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the browser intelligent search method of the embodiment of the present disclosure are executed.
[0222] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.
[0223] According to the computer-readable storage medium of the embodiment of the present disclosure, non-transitory computer-readable instructions are stored thereon. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the browser intelligent search method of each embodiment of the present disclosure are executed.
[0224] The above-mentioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or mobile hard disk), media with built-in rewritable non-volatile memory (e.g., memory card) and media with built-in ROM (e.g., ROM box).
[0225] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.
[0226] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.
[0227] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be recognized by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, meaning "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The word "such as" used here refers to the phrase "such as but not limited to", and can be used interchangeably with it.
[0228] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a disjunctive list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.
[0229] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.
[0230] Various modifications of the technology described herein may be made without departing from the technology of the teachings defined by the appended claims.
[0231] Various changes, substitutions and modifications are contemplated herein. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include within their scope such processes, machines, manufactures, compositions of events, means, methods or actions.
[0232] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
[0233] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.< / video> < / audio> < / video> < / video> < / audio> < / audio>
Claims
1. A browser intelligent search method, characterized in that: include: Building browser plugins; Creating a user interface on a current page of the browser based on the browser plug-in; Get the user's search content in the browser; Classifying the search content based on the browser plug-in to obtain the type of the search content; Based on the type, a corresponding AI processing module is called to search the search content to obtain search results; The search results are displayed on the user interface.
2. The browser intelligent search method according to claim 1, characterized in that: The obtaining of the search content of the user in the browser includes: Adopt any one of the first rule, the second rule, the third rule, the fourth rule, the fifth rule, and the sixth rule to obtain the search content; The priorities of the first rule, the second rule, the third rule, the fourth rule, the fifth rule, and the sixth rule are reduced in sequence; The user interface includes a first input field; the first rule includes: In response to a search request initiated by a user in the user interface, obtaining search content in the first input field; The browser includes a second input field; the second rule includes: In response to a search request initiated by a user in the browser, obtaining input content in the second input field, and copying the input content into the input field of the user interface as the search content; The third rule includes: Monitor the user's mouse selection operation and obtain the selected content; Copying the circled content into the input field of the user interface as the search content; The fourth rule includes: Adding an AI search item in the right-click menu, in response to a user clicking operation on the AI search item, obtaining a specified image selected by the user, and copying the specified image to an input field of the user interface as the search content; The fifth rule includes: YY+242143P Detect whether audio is played in the current page; If the current page plays audio, convert the audio into text content, and obtain the search content based on the text content; If the current page does not play audio, then detect whether the current page plays video; If the current page plays a video, metadata of the video is obtained, and the search content is obtained based on the metadata; The sixth rule includes: Get the static time of the user's mouse; When the static time reaches a time threshold, the screen content under the current position of the user's mouse is obtained; The screen content is copied into an input field of the user interface as the input field content.
3. The browser intelligent search method according to claim 2, characterized in that: The acquiring the search content based on the text content includes: Converting the text content into a target language text; The most frequent words are obtained from the target language text, where the most frequent words are the search content.
4. The browser intelligent search method according to claim 2, characterized in that: The acquiring the search content based on the text content includes: Converting the text content into a target language text; Acquire a plurality of first keywords from the target language text; Generate a plurality of first Prompt words based on the plurality of first keywords; In response to the user's request for the current first Prompt word, the current first Prompt word is used as the search content.
5. The browser intelligent search method according to claim 2, characterized in that: The step of obtaining metadata of the video and obtaining the search content based on the metadata includes: Obtaining a non-advertising segment of the video; Detect the non-advertising segment to obtain the characters and objects that appear most frequently in the non-advertising segment; Get the video uploader and the name of the video background music; The person, the object, the video uploader, and the name of the video background music are all taken as a metadata; Generate multiple second prompts based on multiple metadata; In response to the user's request for the current second Prompt word, the current second Prompt word is used as the search content.
6. The browser intelligent search method according to claim 1, characterized in that: The calling of a corresponding AI processing module based on the type to search the search content and obtain search results includes: When the type is text, calling a text AI processing module to pre-process the search content; Extract keywords from the preprocessed search content to obtain a second keyword; Perform semantic understanding on the preprocessed search content to obtain semantic features; Acquire the user's query intention based on the second keyword, the semantic feature and a preset classification model; Based on the query intent, a corresponding search engine API is called to perform a search and obtain the search results.
7. The browser intelligent search method according to claim 1, characterized in that: The calling of a corresponding AI processing module based on the type to search the search content and obtain search results includes: When the type is a picture, calling the picture AI processing module to compress the search content; Preprocess the compressed search content; Perform feature extraction on the preprocessed search content to obtain multiple image features; Setting image labels based on multiple image features, where one image label corresponds to several image features; Get the third prompt word; The search result is obtained based on the third Prompt word, the multiple image features, the image tag and a preset search index library.
8. A computer device, characterized in that: The computer device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the browser intelligent search method described in any one of claims 1-7.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the browser intelligent search method described in any one of claims 1-7.
10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Method and device for generating word selecting searching result based on browsing content
CN102929924A
Method for carrying out network search on browser side and browser
CN103631875A
Data search method and device and device for data search
CN106886540A
Cited By
Browser control method and device, electronic equipment and medium
CN119828937A
