Method and apparatus for searching a rendered region in a browser

Abstract

In one general aspect, a method can include displaying a webpage within a display area of ​​a browser and displaying a search area indicator associated with a region of the webpage. The method can include receiving a selection of the search area indicator. In response to receiving a selection of the search area indicator, the method can include determining an image corresponding to the region and triggering a search query based on the image. The method can include receiving search results in response to the search query and displaying the search results in a search results area outside the display area within the browser, the search results area being within the browser.

Description

[Technical Field]

[0001] Related Applications This application is a continuation of and claims the benefit of U.S. Application No. 18 / 056,045, filed November 16, 2022, which claims priority to and the benefit of U.S. Provisional Application No. 63 / 264,213, filed November 17, 2021, and which claims priority to and the benefit of U.S. Provisional Application No. 63 / 266,296, filed December 31, 2021, all of the foregoing disclosures are incorporated herein by reference in their entireties. [Background technology]

[0002] A web browser can be used to display content found by a search engine, and the content can be displayed within the web browser in response to a user selecting one or more of the search results. Summary of the Invention

[0003] The present disclosure relates to searching for content displayed on a webpage by capturing at least a portion of the webpage content. Specifically, an area of ​​the webpage can be selected (e.g., selected using gestures and / or an input device), and the selected area can be searched in response to the selection of the area. Search results can be displayed within a search results area of ​​the browser or a search results area of ​​another browser, such that the content of the webpage is maintained (i.e., persists) in the browser while the search results are displayed. At least one of the search results can be selected, and content associated with the selected search result can be displayed. In some implementations, the content associated with the search result can replace the content of the webpage in the browser. In some implementations, the content associated with the search result can be displayed in another browser. The search results can continue to be displayed (e.g., can be maintained or can persist) while the content of the search results is navigated and / or displayed.

[0004] In one general aspect, a method can include displaying a web page within a display area of ​​a browser and displaying a search area indicator associated with a region of the web page. The method can include receiving a selection of the search area indicator. In response to receiving the selection of the search area indicator, the method can include determining an image corresponding to the region and triggering a search query based on the image. The method can include receiving search results in response to the search query and displaying the search results in a search results area outside the display area, the search results area being within the browser. [Brief explanation of the drawings]

[0005] [Figure 1A] FIG. 1 illustrates retrieval of content displayed on a web page by capturing a selection of at least a portion of an area of ​​the content within the web page. [Figure 1B] FIG. 1 illustrates retrieval of content displayed on a web page by capturing a selection of at least a portion of an area of ​​the content within the web page. [Figure 1C] FIG. 1B illustrates the content of a web page rendered in a navigation area within the browser tab shown in FIG. 1A. [Figure 2] FIG. 1 illustrates the content of a web page rendered in a navigation area within a browser tab. [Figure 3] FIG. 1 illustrates a search results area and a navigation area rendered within a browser tab. [Figure 4] 4A and 4B are diagrams illustrating a search results area, a navigation area, and a display area rendered within a browser tab. [Figure 5] FIG. 1 illustrates security features associated with implementations described herein. [Figure 6] 6A and 6B are diagrams illustrating a search query that is triggered based on the selection of a region that spans multiple elements in a web page. [Figure 7]FIG. 1 illustrates a computing system and search server for implementing the concepts described herein. [Figure 8] 1 is a flowchart illustrating a method for implementing at least some of the concepts described herein. [Figure 9] FIG. 1 illustrates an embodiment in which history related to a search query is shared between devices. [Figure 10] FIG. 10 illustrates an indicator of selectable areas for a search query. [Figure 11] 11A and 11B are diagrams illustrating the activation of a search input area that can be used to trigger a search area indicator. [Figure 12] 1A-1C illustrate examples of general-purpose computing devices and general-purpose mobile computing devices that can be used with the technology described herein. DETAILED DESCRIPTION OF THE INVENTION

[0006] The present disclosure relates to searching for content displayed on a webpage by capturing at least a portion of the webpage content. Specifically, an area of ​​the webpage can be selected (e.g., selected using gestures and / or an input device), and the selected area can be searched in response to the selection of the area. Search results can be displayed within a search results area of ​​the browser or a search results area of ​​another browser, such that the content of the webpage is maintained (i.e., persists) in the browser while the search results are displayed. At least one of the search results can be selected, and content associated with the selected search result can be displayed. In some implementations, the content associated with the search result can replace the content of the webpage in the browser. In some implementations, the content associated with the search result can be displayed in another browser. The search results can continue to be displayed (e.g., can be maintained or persisted) while the content of the search results is navigated and / or displayed. A browser may be referred to as a browser window when associated with an operating system (OS) window.

[0007] A technical issue related to web page content is that the content may not be easily selectable and / or immediately available for searching. While content within a web page may be available to users and / or links within the web page may be accessible, the content itself may not be easily searchable using a search engine. For example, text within a web page may be copied and pasted into a search engine, but this process may require multiple manual steps, time, storage resources, and / or other resources.

[0008] Another technical issue related to content within a web page is that even after the content to search within a web page has been identified, the content within the web page may not be maintained during the search unless a separate browser, tab, or application is used. For example, the content within the web page can be copied and searched, the web page can be replaced with a search engine interface, or a separate browser or tab can be used to search for the content.

[0009] At least one technical solution to the above technical problem is to allow selection of a region of a web page within a browser, and the selected region can also be searched in response to the region selection (e.g., with little or no manual intervention). Search results can be displayed within the search results region of the browser, such that the web page content is maintained (i.e., persists) in the browser in a separate display region while the search results are displayed.

[0010] At least one technical effect of this technical solution is that the original web page content is not lost while the user is using the search results. The search results in the search results area and the web page content in the display area can be displayed simultaneously within the browser (e.g., in a browser tab). Thus, the proposed solution makes the search results available to the user in another area of ​​the browser, allowing the user to interact with the search results, e.g., the presented content is associated with the search results, while at the same time allowing the user to still and simultaneously interact with the web page, in particular the selection area that was used to trigger the search query.

[0011] In some embodiments, the search results area can be integrated as part of the browser so that it cannot be spoofed (e.g., imitated) by a third party or a web page content owner. This can be implemented as a security feature, allowing a user to distinguish between legitimate (authentic) search results obtained from a search query and potentially illegitimate search results that may be inserted by a third party or a web page provider. Thus, the technical problem of search result spoofing can be avoided by the technical solution of integrating the search results area within the browser as described above. The technical effect of integrating the search results area within the browser is that spoofing can be prevented.

[0012] As described above, in some implementations, search results may continue to be displayed (e.g., may be maintained or persist) in the search results area while the content of the search results is navigated and / or displayed. Content associated with the search results may be displayed in a navigation area within the browser (e.g., in a browser tab) simultaneously with the search results area. Thus, the technical problem of being unable to view search results in the search results area and simultaneously navigate the content associated with the search results is solved by the above technical solutions and technical effects.

[0013] The browsers described herein may execute within a computing device. For example, the browser may execute within a laptop device. In some implementations, the browser may execute within a mobile device.

[0014] 1A and 1B illustrate a search for content I2 displayed on a web page W1 by capturing a selection of an area SA of at least a portion of the content I2 within the web page W1. The web page W1 is displayed (e.g., rendered) within a display area 120 within a tab 112 of a browser 110. In response to the selection of the area SA of the web page W1 (illustrated by the dashed line), a search query for the portion of the content I2 is triggered. Thus, the search query is triggered without or with little manual intervention by the user. Specifically, once the selection of the area SA of the web page W1 is completed, the search query is triggered without manual intervention. Triggering a search query does not require additional steps (such as saving information, entering a search query, or interacting with a user interface such as a separate browser to initiate a search).

[0015] For example, a user can select area SA using an input device such as a mouse or touchpad (not shown), triggering a search query for a portion of content I2. Thus, the search query is triggered with little or no manual intervention by the user other than selecting area SA. In some cases, no additional input other than the selection is required to trigger the search query.

[0016] Once the search query is completed, the search results S1 are rendered in the search results area 130 of the tab 112 of the browser 110, as shown in FIG. 1B. A copy I2' of the portion of the content I2 is shown in the search results area 130. The correspondence between the region I2 and the copy I2' is indicated by a dashed arrow. In some embodiments, the copy I2' of the portion of the content I2 is not shown in the search results area 130. In some embodiments, the copy I2' of the portion of the content I2 is scaled to a different size (e.g., a smaller size) than the original region SA.

[0017] In response to a search query being triggered, the search results S1 can be rendered in the search results region 130 with little or no manual intervention by the user (other than selecting the search area SA). In some implementations, in response to completing the selection of the area SA of the web page W1, the search query and the rendering of the search results S1 in the search results region 130 are triggered without manual intervention. Additional steps for triggering the search query and rendering the search results S1 in the search results region 130 (e.g., saving information, entering a search query, interacting with a user interface such as a separate browser to initiate the search) can be eliminated.

[0018] The search results S1 may be search results (e.g., ranked search results) that include and / or are linked to content associated with area SA (e.g., copy I2′ of area SA). In some implementations, the search results S1 may include text or be represented as text. In some implementations, the search results S1 may include images. In some implementations, the search results S1 may include images associated with text. In some implementations, the search results S1 may include links to content (e.g., images, web pages, etc.) associated with area SA. In some implementations, the search results S1 may be prioritized so that the most relevant search results (or a subset thereof) are rendered in search results area 130. In some implementations, the search results S1 may be prioritized so that the most relevant search results (or a subset thereof) are rendered toward the top of search results area 130.

[0019] In some embodiments, the search of area SA may be an image search. The selected area SA may be captured as an image (e.g., a collection of pixels associated with or representing an image). This allows a search engine (not shown), for example, on a search server (not shown), to receive the captured image of area SA and perform a search based on the captured image. A search query by the search engine may include or be based on the captured image. In some embodiments, the captured image may be, for example, a bitmap image. In some embodiments, the captured image may be, for example, a screen capture of area SA.

[0020] A search of area SA can be a search used to identify landmarks, products, wildlife, logos, brands, and / or various other objects. A search of area SA can be a search used to identify objects similar to those captured in area SA. In some embodiments, text can be identified and / or searched from area SA. In some embodiments, text in an SA area can be translated into another language. In some embodiments, the text is expressed in a language different from the original language of the original text.

[0021] 1A, content I1, I2, and T1 are rendered within web page W1. In some implementations, content I1, I2, and / or T1 may be images, text, and / or other content rendered within web page W1. Content I1, I2, and / or T1 may be supplied by the provider of web page W1 or may be supplied to web page W1 by another provider.

[0022] In some implementations, a search query by a search engine may include or be based on a captured image of a dynamic element (e.g., a video, a series of moving elements, a moving object) rendered on web page W1. Thus, for example, a search may be performed on a portion (e.g., a screen capture) of a video or other dynamic element rendered within web page W1.

[0023] In some implementations, the selection of the region SA can be performed in response to an input device (not shown). For example, the selection of the region SA can be triggered in response to a touchpad device, a keyboard, a mouse input device, a controller, and / or other input device. In some implementations, the selection of the region SA can be performed in response to a gesture (e.g., a gesture triggered via a touchpad device, a hand gesture, a gesture performed using an input device, etc.).

[0024] In this implementation, region SA does not exactly coincide with the region of content I2. In some implementations, region SA may be smaller than the region of content I2. In some implementations, region SA may be larger than the region of content I2. More details regarding region selection across multiple elements are described in connection with at least Figures 6A and 6B.

[0025] 1A and 1B, the browser includes an address bar area 114. The address of web page W1 may be shown in address bar area 114 (e.g., input address area 113). Other controls, icons, and / or the like may be included in address bar area 114. Address bar area 114 may be controlled by and / or associated with browser 110 (e.g., a browser application). Address bar area 114 may be controlled by browser 110 rather than by web page W1 and / or a provider of web page W1.

[0026] As shown in FIG. 1B , search results area 130 is separated from address bar area 114 by line 131. In some implementations, for security purposes, line 131 can be removed (e.g., omitted) in any of the implementations described herein. This is a line that cannot be removed (e.g., painted over, erased) by a third party (e.g., the provider of web page W1). This is a line that is controlled (e.g., provided, removed) by the provider of browser 110 (e.g., the browser application). Further details regarding this security feature are provided below in connection with at least FIG. 5.

[0027] As shown in FIG. 1B , in response to rendering search results region 130 in response to the search query, the size (e.g., area) of display region 120 is reduced to the size of display region 125. In other words, in response to displaying search results region 130, the size (e.g., area) of display region 120 is changed (e.g., reduced in size). Web page W1 may be re-layed (e.g., scaled) to fit (e.g., fit to display) within display region 125. In some implementations, display region 120 shown in FIG. 1A may be divided into display region 125 and search results region 130, as shown in FIG. 1B . The combined area of ​​display region 125 and search results region 130 shown in FIG. 1B may be the same as the area of ​​display region 120 (e.g., the original display area of ​​browser 110) shown in FIG. 1A . In some implementations, the combined area of ​​display region 125 and search results region 130 may be different (e.g., smaller, larger) than the area of ​​display region 120. Display area 120 is resized to display area 125 before receiving a selection of an area (e.g., search area S1) so that search area 130 is positioned within the area allocated to display area 120 (e.g., the entire display area 120).

[0028] Although not shown in FIG. 1B , the location of search results region 130 may differ from the location shown in FIG. 1B . For example, the size of search results region 130 may be reduced. In some implementations, search results region 130 may be included along different sides of browser 110 (e.g., along the bottom, along the top, along the left side, and / or along the like). In some implementations, search results region 130 may float above display region 120 (e.g., may be rendered in an upper window or may be rendered in an upper window that moves independently of browser 110) to prevent the size of display region 120 of FIG. 1A from being reduced to the area of ​​display region 125 shown in FIG. 1B . In some implementations, search results region 130 may be a different shape (e.g., circular) rather than square or rectangular.

[0029] In some implementations, after search result S1 is rendered in search results region 130 as shown in at least FIG. 1B , search results region 130 can be used to trigger additional search queries. For example, a search result in search results region 130 can be selected and used to trigger the rendering of one or more images and / or text associated with the selected search result from search results region 130.

[0030] In some implementations, after the search results S1 are rendered in the search results area 130 as shown in at least FIG. 1B , additional search queries can be triggered using the browser 110 (e.g., the display area 125). In some implementations, the additional search queries can include selection of additional areas in the display area 125 of the web page W1 other than the search area SA. In some implementations, the additional search queries can be text-based search queries that modify (e.g., further refine, filter) the search results S1 in the search results area 130. The additional search queries can be image-based search queries that modify (e.g., further refine, filter) the search results S1 in the search results area 130. In other words, the additional search queries can modify the search results S1 to further prioritize the search results S1, filter the search results S1, and / or otherwise.

[0031] In some embodiments, although not shown, the search result S1 may be bookmarked or associated with a bookmark in the browser 110. In some embodiments, each of the search results may be bookmarked or associated with a bookmark in the browser 110. In some embodiments, only a limited number (or only a portion) of the search results (e.g., the top five) may be bookmarked.

[0032] In some implementations, not shown, the search result S1 (or a prioritized subset thereof) can be associated with a tab group (e.g., a set of tabs in the browser 110 that are manually associated based on content, and / or the like). For example, a first search result from the search result S1 can be associated with a first tab (not shown), and a second search result can be associated with a second tab (not shown). The first and second search results can be associated with a tab group (e.g., a group of tags associated with a single name). In some implementations, only a limited number of the search results (e.g., the top five) may be associated with a tab and / or a tab group.

[0033] 1A and 1B, in some embodiments, after search result region 130 is defined, a new web address (instead of the web address of web page W1) can be entered in input address region 113 of address bar region 114. In response to the new web address region, web page W1 in display region 125 can be replaced with another web page (not shown) (or other content) while maintaining search result S1 in search result region 130. That is, search result S1 can be maintained while navigating display region 125 using address bar region 114 of browser 110.

[0034] In some implementations, a user interface element (e.g., a back button, a back arrow) may be used to remove (e.g., clear, navigate elsewhere) search result S1 in search results area 130. In some implementations, a user interface element may be used to remove (e.g., clear) search result S1 and / or search results area 130. In some implementations, a user interface element may be used to navigate (e.g., go back) from web page W1, search result S1, and / or search results area 130.

[0035] 1C illustrates the content of web page W2 rendered in navigation region 140 within tab 112 of browser 110 shown in FIG. 1A. The content of web page W2 is rendered in navigation region 140 in response to an interaction (e.g., a user interaction) with search result S1 in search results region 130. In this implementation, display region 125 is replaced (e.g., changed) by navigation region 140. In this implementation, web page W1 displayed in display region 125 is replaced with web page W2 displayed in navigation region 140.

[0036] Navigation area 140 can be used to consume (e.g., explore) the search results S1 in search results area 130. For example, in response to a first search result being selected from search results S1, first content may be rendered in navigation area 140. In response to a second search result being selected from search results S1, second content may be rendered in navigation area 140 instead of or in addition to the first content. Content can be navigated while search results S1 are maintained in search results area 130.

[0037] 1C , in some embodiments, after search results region 130 and navigation region 140 are defined, a new web address can be entered into input address region 113 of address bar region 114. In response to the new web address region, web page W2 (or other content) in navigation region 140 can be replaced with another web page (not shown) while maintaining search result S1 in search results region 130. That is, search result S1 can be maintained while navigating within navigation region 140 using address bar region 114 of browser 110.

[0038] In some implementations, user interface elements (e.g., back button, back arrow) can be used to remove (e.g., clear, navigate elsewhere) content rendered in navigation region 140 (e.g., web page W2). In some implementations, user interface elements can be used to remove (e.g., clear) search results S1, search results region 130, and / or navigation region 140. In some implementations, user interface elements can be used to navigate (e.g., go back) from web page W2, search results S1, and / or search results region 130.

[0039] Although described in the context of web page W2, in some embodiments, the content rendered in navigation region 140 may not be a web page. In some embodiments, the content displayed in navigation region 140 may not be a web page and / or may not be associated with a web page. In some embodiments, the content displayed in navigation region 140 may be among images, links, videos, text, and / or the like.

[0040] In some implementations, an interaction may include a selection. In some implementations, the interaction and / or selection (or an indication thereof) may be used to trigger an action or result.

[0041] 2 shows the contents of web page W2 rendered in navigation area 240 within tab 212 of browser 210 (e.g., a browser window) separate from browser 110 (e.g., a browser window). FIG. 2 is a variation of the implementation described in connection with FIGS. 1A-1C. Thus, the descriptions of and variations described therein for FIGS. 1A-1C are also applicable to FIG. 2. Specifically, FIG. 1C corresponds to the state of browser 110 shown on the left side of FIG. 2.

[0042] In this implementation, in response to an interaction (e.g., a user interaction) with search result S1 in search results area 130 of browser 110, the content of web page W2 is rendered in navigation area 240 of browser 210 (which is a separate browser from browser 110 and may also be referred to as an additional browser). This is in contrast to FIG. 1C , in which web page W2 is rendered within browser 110 (e.g., within the same browser). Navigation area 240 of browser 210 does not replace display area 125 of browser 110 (as shown in FIG. 1C ), but rather is opened in browser 210. Thus, the content of web page W1 can be maintained within display area 125 of browser 110 alongside search results area 130.

[0043] Additionally, in response to rendering web page W2 (or other content) in navigation area 240 of browser 210, search result S1 in search results area 130 is maintained in browser 110. That is, search result S1 can be maintained while navigating within navigation area 240 of browser 210.

[0044] 3 illustrates a search results region 330 and a navigation region 340 rendered in a tab 312 of a browser 310 (e.g., a browser window) separate from browser 110 (e.g., a browser window). In response to a search query being completed, search results S1 are rendered in search results region 330 of tab 312 of browser 310, as shown in FIG. 3. A copy I2' of a portion of content I2 is shown in search results region 330. Thus, search results region 330 of browser 310 does not replace (or reduce the area of) display area 120 (which contains web page W1) of browser 110.

[0045] Search results S1 in search results region 330 can be navigated within navigation region 340 in substantially the same manner as described above in connection with at least FIG. 1B. Thus, search results region 330 and navigation region 340 of browser 310 do not replace (or reduce the area of) display region 120 (which includes web page W1) of browser 110. Thus, the content of web page W1 can be maintained within display region 120 of browser 110 without the search results region.

[0046] 3, search results region 330 is generated in a new browser—browser 310. Search result S1 in search results region 330 may be retained in browser 310. Search result S1 may be maintained in search results region 330 in browser 310 while navigating within navigation region 340 of browser 310. Thus, original web page W1 may be maintained in browser 110, and further navigation may occur in browser 310 while maintaining search results region 330 in browser 310.

[0047] In some embodiments, the browser window in which search results are rendered may be referred to as a search result browser window. In this case, browser 310 may be the search result browser window. In some embodiments, the browser window in which navigation occurs may be referred to as a navigation browser window. In this case, browser 310 may be the navigation browser window. In some embodiments, the browser window containing original web page W1 may be referred to as a main browser window. In this case, browser 110 may be the main browser window. Thus, as shown in FIG. 3 , browser 310 may be the search result browser window and the navigation browser window, and browser 110 may be the main browser window.

[0048] Figure 4A shows search results region 130 and navigation region 140 rendered within tab 112-2 of browser 110, separate from tab 112-1 (not shown in Figure 4A) in which display region 120 is rendered. The contents of tab 112-1 (existing tabs, main tabs, etc.) are shown in Figure 4B. Figure 4B shows display region 120 at least similar to that shown in Figure 1A.

[0049] In response to the search query being completed, search results S1 are rendered in search results area 130 of tab 112-2 (e.g., a new tab, a secondary tab) of browser 110, as shown in FIG. 4A. A copy I2′ of a portion of content I2 is shown in search results area 130. Thus, search results area 130 of tab 112-2 of browser 110 does not replace (or reduce the area of) display area 120 (including web page W1) of tab 112-1 of browser 110.

[0050] Search results S1 in search results region 130 of tab 112-2 of browser 110 can be navigated within navigation region 140 in at least substantially the same manner as described above in connection with FIG. 1B. Thus, search results region 130 and navigation region 140 of tab 112-2 of browser 110 do not replace (or reduce the area of) display region 120 (including web page W1) of tab 112-1 of browser 110. Thus, the content of web page W1 can be maintained within display region 120 of tab 112-1 of browser 110 without the search results region.

[0051] In some embodiments, navigation of search result S1 can be rendered in an existing tab (rather than a new tab). In some embodiments, the content of the existing tab can be replaced. In some embodiments, new tabs can be opened to navigate each search result from the selected search result S2.

[0052] In some implementations, the tab on which search results are rendered may be referred to as a search results tab (e.g., as shown in FIG. 4A). In some implementations, the tab on which navigation occurs may be referred to as a navigation tab.

[0053] FIG. 5 illustrates security features associated with embodiments described herein. As shown in FIG. 5, the address bar area 114 and the search results area 130 of the browser 110 are combined into one continuous area (e.g., a continuous area). This continuous area (e.g., the combined address bar area 114 and the search results area 130) may be referred to as the browser search area 135, indicated by the continuous gray area between the address bar area 114 and the search results area 130. This embodiment shown and described in connection with FIG. 5 may be combined with any of the embodiments described above in connection with, for example, FIGS. 1A through 4B (and other figures below (e.g., FIG. 6B)). Specifically, by way of example, FIG. 5 may be a transition from FIG. 1A, and the browser search area 135 may be implemented in FIGS. 1B and 1C. Similarly, the browser search area 135 may be implemented in at least FIGS. 2, 3, 4A, 6B, 9, 11A, 11B, etc.

[0054] The search results area 130, like the browser search area 135, can be integrated as part of the browser 110 (e.g., the address bar area 114), such that the search results area 130 cannot be disguised (e.g., imitated) by, for example, a third party or the owner of the content of web page W1 (not shown) and / or the content of web page W2. Because the search results area 130 is part of the browser 110 application, the integration of the search results area 130 is difficult to imitate. There is no clear separation (e.g., no separate line) between the search results area 130 and the address bar area 114 (e.g., the line 131 shown in FIG. 1B has been removed). Any background or theme applied to the address bar area 114 flows into (and is continuous with) the search results area 130 (as shown by the gray area). The continuous background is difficult for a third party (e.g., the provider of web page W1) to imitate within the browser 110 application. Additionally, the browser search area 135 may be implemented as a security feature, allowing a user to distinguish between legitimate search results resulting from a search query and search results that may be inserted by a third party or web page provider.

[0055] A continuous background throughout the browser search area 135 (including the search results area 130) may be an indicator of control of the entire browser search area 135 by the browser 110. If the entire browser search area 135 is not continuous (e.g., there are breaks or discontinuities in the background theme, image, and / or other), this may indicate that the entire browser search area 135 and / or the content contained therein is not controlled by the browser 110 and / or may not be authentic.

[0056] Stated another way, the content rendered in navigation area 140 can be attributed to (e.g., controlled, generated by) the content producer of web page W2 (after navigation to web page W2 is triggered via search result S1). The content rendered in browser search area 135 can be attributed to (e.g., controlled, generated by) browser 110. The content of web page W2 can be independently controlled by the content owner of web page W2 and can be separate from browser 110. The content in browser search area 135 (specifically search result S1) is independently controlled (and generated) by browser 110 and can be separate from web page W2. The content producer of web page W2 cannot generate or modify search result S1.

[0057] Combining the search results area 130 with the address bar area 114 can be an indicator of the trustworthiness of the content in the search results area 130. When the search results area 130 and the address bar area 114 are combined, it means that the browser 110 (or a provider of the browser 110) has provided the search results S1 in the search results area 130. If a separating line is present (e.g., line 131 shown in FIG. 1B ), the results in the area of ​​the search results area 130 may be provided by an untrusted provider (e.g., a third party).

[0058] 5, navigation area 140 is shown adjacent to browser search area 135 (or a portion thereof). In some implementations, display area 125 (e.g., as shown in FIG. 1B) can be included in browser 110 adjacent to browser search area 135 (or a portion thereof).

[0059] 6A and 6B illustrate a search query that is triggered based on the selection of a region SA2 that spans multiple elements (e.g., at least two elements or images) within a web page W1. The concepts associated with Figures 6A and 6B may be combined with any of the features described above in connection with Figures 1A through 5 (or any of the figures described after Figures 6A and 6B).

[0060] In this exemplary embodiment, selection area SA2 spans at least a portion of images I1 and I2. Thus, an image capture associated with selection area SA2 includes portions of images I1 and I2 and portions of web page W1 surrounding images I1 and I2. The image capture of area SA2 can be used to trigger a search query, as described above, and search result S1 (which may be persistent, as described above) can be generated.

[0061] Because region SA2 spans multiple regions of interest (e.g., images, text regions), disambiguation may be required before triggering (performing) a search. For example, a search server (not shown) can be configured to receive region SA2. The search server can be configured to trigger a search in region I2', which corresponds primarily to image I2. The search server can be configured to discard regions that should not be searched (e.g., the region between images I1 and I2). The search server can apply one or more criteria to determine whether to search region I2'. The criteria can include, but are not limited to, that the area of ​​image I2 is larger than the area of ​​image I1, that search results for image I2 are more robust (or complete) than search results for image I1, and / or others. As a specific example, if there are more search results associated with image I2 than image I1, then search results for image I2 may be more robust (or complete) than search results for image I1. As another specific example, if image I2 is associated with more reliable search results (e.g., a higher confidence score for the search result hit) than image I1, the search results for image I2 may be more robust (or complete) than the search results for image I1.

[0062] In some implementations, multiple different search queries may be triggered depending on a region, such as region SA2, that spans multiple regions of interest (e.g., multiple regions that can be searched independently.) For example, the search server may be configured to perform a first search query based on the portion of image I1 that is included in region SA2 and a second search query based on the portion of image I2 that is included in region SA2.

[0063] This type of multiple search trigger can be useful, for example, when navigating a shopping website. If web page W1 is a shopping website, images I1 and I2 may be associated with different products. A single selection of area SA2 can trigger a search for multiple products.

[0064] In some implementations, search results from multiple searches triggered by a single area (e.g., area SA2) can be presented in a variety of ways. In some implementations, search results from multiple searches triggered by a single area (e.g., area SA2) can be rendered in search results area 130. Although not shown, in some implementations, search results from multiple searches can be associated with different search results areas within tab 112 of browser 110. For example, search results from a first search can be associated with a first search results area, and search results from a second search can be associated with a second search results area.

[0065] In some implementations, search results from multiple searches can be associated with different tabs, for example, search results from a first search can be associated with a first tab and search results from a second search can be associated with a second tab.

[0066] In some implementations, search results from multiple searches can be associated with different windows. For example, search results from a first search can be associated with (e.g., triggered to be rendered and / or launched within) a first browser window, and search results from a second search can be associated with (e.g., triggered to be rendered and / or launched within) a second browser window. The search results associated with the first browser window and the search results associated within the second browser window can be rendered within search result regions in the first and second browser windows, respectively. One or more of the search result regions in each of the first and second browser windows can be persistent search result regions, as described above in connection with at least Figures 1A through 5 (or in connection with any figure after Figure 5).

[0067] In some implementations, search results from multiple searches can be associated with different tabs within a single browser window. For example, search results from a first search can be associated with (e.g., triggered to be rendered and / or launched within) a first tab, and search results from a second search can be associated with (e.g., triggered to be rendered and / or launched within) a second tab. The search results associated with the first tab and the search results associated with the second tab can be rendered within search result regions within the first tab and the second tab, respectively. One or more of the search result regions of each of the first and second tabs of the browser can be persistent search result regions, as described above in connection with FIGS. 1A-5.

[0068] In some implementations, search results from multiple searches can be associated with (e.g., stored as) different bookmarks. For example, search results from a first search can be associated with a first bookmark (which can be accessed later), and search results from a second search can be associated with a second bookmark (which can be accessed later).

[0069] FIG. 7 illustrates a system 700 including a computing system 702 and a search server 740 configured to implement the concepts and various embodiments shown and described herein. As shown in FIG. 7, the computing system 702 is configured to communicate with the search server 740 and / or a web page provider 710 (e.g., a web server) over a network 750. The computing system 702 includes at least a browser manager 720 and a selection manager 730. In some embodiments, the browser manager 720 is configured to manage content, such as web page content, provided by the web page provider 710 (e.g., also referred to as a web server). In some embodiments, the browser manager 720 is configured to operate as one of several application(s) 728 executed via an operating system (O / S) 729. The browser manager 720 can be configured to implement a user interface, search results, windows, and / or other features as described in connection with the embodiments described herein.

[0070] 7, computing system 702 includes several hardware components, including a communications module 761, one or more cameras 762, memory 763, a central processing unit (CPU) and graphics processing unit (GPU) 764, one or more input devices 767, and one or more output devices 768. The hardware components may be used to facilitate the operation of browser manager 720, selection manager 730, and / or other components of computing system 702.

[0071] Browser manager 720 includes a user interface (UI) generator 721 configured to generate and / or manage various user interface elements of a browser, such as browser 110 shown and described herein. For example, UI generator 721 may generate UI elements including the various windows of browser 110, such as display area 120, search results area 130, and / or navigation area 140 shown in at least Figures 1A-1C (and other figures).

[0072] Browser manager 720 includes tab manager 722 configured to generate and / or manage various tabs for a browser, such as browser 110, as shown and described herein. For example, tab manager 722 can generate tabs in browser 110, such as tabs 112-1, 112-2 shown in at least FIG. 4A (and other figures).

[0073] Browser manager 720 includes a search results manager 723 configured to generate and / or manage content renderings, such as search results S1, in search results area 130 (shown in at least FIG. 1B and other figures). Browser manager 720 includes a display area manager 724 configured to generate and / or manage content renderings associated with web page W1 in display areas 120, 125 (shown in FIG. 1B and other figures). Web page W1 content can be provided to computing system 702 by web page provider 710. Browser manager 720 includes a navigation manager 725 configured to generate and / or manage content renderings associated with navigation area 140 (shown in at least FIG. 1C and other figures).

[0074] 7, session data 727, which may be stored in memory 763 (not shown), may be managed as or by one of applications 728. Session data 727 may include data related to one or more browser sessions. Application information 726 may include information related to various applications running within and / or executable by O / S 729.

[0075] Selection manager 730 can be configured to manage one or more elements selected for a search. Information from selection manager 730 can be communicated to search server 740 and trigger one or more search queries. Selection manager 730 includes area detector 732 that can detect one or more areas selected for a search (e.g., search area SA shown in FIG. 1B and other figures). Selection manager 730 also includes capture manager 731 that can manage communication of content captured within one or more areas detected by area detector 732. Selection of one or more areas for a search can be triggered by one or more input devices 767 (e.g., a mouse, a gesture, etc.) and communicated to selection manager 730 (and area detector 732).

[0076] 7 , communications module 244 can be configured to facilitate communication with web page provider 710 and / or search server 740 over network 750 via one or more communications protocols. Camera 762 can be used to capture one or more images, memory 763 can be used to store information associated with browser manager 720 and / or selection manager 730, and CPU / GPU 764 can be used to process the information and / or images associated with browser manager 720 and / or selection manager 730. Computing system 702 also includes one or more output devices 768, such as a communications port, speakers, a display, and / or the like. The functionality described in this application can be implemented based on one or more policies 765 and / or settings 766 stored in memory 763.

[0077] FIG. 7 illustrates some aspects of a search server 740. For example, the search server 740 includes one or more processors 746 and one or more memory devices 748. The search server 740 includes an image receiver 744 configured to receive one or more images from the computing system 702 that can be used to trigger a search query that can be executed by the search server 740. The one or more images may correspond to a search area (e.g., search area SA shown in at least FIG. 1A and other figures), for example. The search server 740 includes a search disambiguator 742 configured to determine a particular area to search when multiple areas that can be searched (e.g., multiple images) are sent to the search server 740. For example, the search disambiguator 742 can be used to resolve a search query that covers multiple images, such as the situation described in connection with FIGS. 6A and 6B. The search disambiguator 742 can be configured to implement the algorithms described in connection with FIGS. 6A and 6B.

[0078] 8 is a flowchart illustrating a method for implementing at least some of the concepts described herein in the various figures. Many elements of the flowchart can be implemented by at least the components shown in FIG.

[0079] The method includes displaying a web page within a display area of ​​a browser (block 802). The web page may be displayed, for example, as shown in display area 120 of browser 110 of FIG. 1A. Display of the web page may be triggered, for example, by display area manager 724 shown in FIG. 7.

[0080] The method includes receiving a selection of a region of a web page (block 804). An example of a region selection is shown as the selection of search region SA of image I2 shown in Figure 1B. The selection of a region of a web page can be managed by selection manager 730 shown in Figure 7.

[0081] The method includes determining an image corresponding to the region (block 808) and triggering a search query based on the image (block 810), both in response to receiving a selection of the region (block 806). For example, this process is illustrated at least illustratively in FIG. 1B. The search query can be triggered by a search region SA shown in FIG. 1B. The image corresponding to the region can be determined by a capture manager 731 of a selection manager 730 shown in FIG. 7. The image for search can be sent by the selection manager 730 to a search server 740 shown in FIG. 7. An image receiver 744 can receive the image, and the search query can be executed on the search server 740 shown in FIG. 7.

[0082] The method includes receiving search results in response to the search query (block 812). For example, the search results S1 may be received by the browser 110 shown in FIG. 1A. The search results may be transmitted from the search server 740 to the computing system 702 over the network 750. The receipt and processing of the search results may be handled by the search result manager 723 of the browser manager 720.

[0083] The method includes displaying the search results in a search results area outside the display area, the search results area being within the browser (block 814). As shown in FIG. 1B, search results S1 are rendered in search results area 130. In some implementations, the rendering of the search results can be managed by a search results manager 723 of the browser manager 720. Further navigation in a search results navigation area (e.g., navigation area 140 shown in FIG. 1C) triggered by interaction with the search results can be managed by a navigation manager 725 of the browser manager 720 shown in FIG.

[0084] Figure 9 illustrates an embodiment in which history associated with a search query is shared between devices. The diagram in Figure 9 is based on at least the embodiment described in connection with Figures 1A-1C. The embodiment in Figure 9 is applicable to any of the embodiments described herein. Search history may be shared with permission from the owner of the account associated with the search history.

[0085] In this embodiment, browser 110 is displayed within display 974 of computing device 970. At least a portion of a search history associated with search area SA (or another search area) can be stored in memory 972 of computing device 970. In some embodiments, the search history can be stored in response to a search query triggered by selection of search area SA. In some embodiments, the search history can be search result S1 (or a representation of search result S1) shown in FIG. 9 and other figures, such as FIG. 1A. At least a portion of the search history can be shared with server 980 (e.g., search server 740 shown in FIG. 7) and / or companion device 990 based on permissions granted via computing device 970.

[0086] At least a portion of the search history may be shared from computing device 970 with server 980. At least a portion of the search history may be stored in memory 982 of server 980. In some implementations, the search history may be shared with server 980 if server 980 is associated with the account associated with the search history.

[0087] In some implementations, at least a portion of the search history may be shared by the computing device 970 with the companion device 990. In some implementations, the search history may be shared by the computing device 970 with the companion device 990 if permission is given by the owner of the account associated with the search history. In some implementations, the search history may be shared with the companion device 990 if the companion device 990 is associated with the account associated with the search history.

[0088] In some implementations, the server 980 can be configured to share at least a portion of the search history with the companion device 990. In some implementations, if permission is granted by the owner of the account associated with the search history, at least a portion of the search history can be shared by the server 980 with the companion device 990. At least a portion of the search history received from the server 980 can be stored in memory 992 of the companion device 990.

[0089] In some implementations, the companion device 990 can be configured to share at least a portion of the search history with the server 980. In some implementations, if permission is granted by the owner of the account associated with the search history, at least a portion of the search history can be shared by the companion device 990 with the server 980. At least a portion of the search history received from the companion device 990 can be stored in the memory 982 of the server 980.

[0090] In some embodiments, the search history may include a history of search queries. In some embodiments, the search history may include a history of one or more objects (e.g., search areas SA) that have been searched. In some embodiments, the search history may include metadata identifying the search queries and / or the objects that have been searched.

[0091] In some embodiments, the search history may be shared between the computing device 970, the server 980, and / or the companion device 990 for collaboration purposes. For example, the companion device 990 may use the search history to further refine a search query included in the search history. In some embodiments, the refined search query may be shared with the computing device 970. As another example, the companion device 990 may use the search history to refine a search on the companion device 990 for use on the companion device 990.

[0092] 10 is a diagram illustrating an indicator of an area that can be selected for a search query. An indicator of a searchable area can be referred to as a search area indicator. The search area indicator can be applied to any of the implementations described herein. The search area indicator can be generated and rendered to a user so that the user can be directed to and / or alerted to an area that can be selected to trigger a search query. In some implementations, the search area indicator can be rendered and / or managed by the display area manager 724.

[0093] In this implementation, the search space indicators are shown as a set of circles superimposed on one or more portions of web page W1 and / or one or more portions of objects I1, I2, T1 contained within web page W1. The set of circles are grouped as indicated by the dashed lines of search space indicators G1 through G5. In some implementations, one or more of search space indicators G1 through G5 may differ from those shown in FIG. 10. One or more of search space indicators G1 through G5 may be different types of affordances, icons, and / or other elements.

[0094] In some implementations, search area indicators G1 through G5 can be displayed for areas with high search potential. Search potential can be based on historical data related to the content of web page W1. Search potential can be based on historical data (using anonymized data) collected from many users who accessed web page W1 or web pages similar to web page W1.

[0095] In some implementations, search area indicators G1 through G5 may be displayed for areas within web page W1 that have not yet been linked. For example, one or more of search area indicators G1 through G5 may be associated with images, text, and / or other elements embedded within web page W1. In some implementations, one or more of search area indicators G1 through G5 may be associated with translatable text.

[0096] In some implementations, one or more of the search space indicators G1 through G5 can be displayed based on an initial analysis of the web page W1 without connecting to a search server. For example, one or more of the search space indicators G1 through G5 can be determined in response to a local search performed on a computing device that renders the web page W1.

[0097] In some implementations, one or more search area indicators G1-G5 may be associated with shoppable objects (e.g., objects associated with commerce). In some implementations, one or more search area indicators G1-G5 may be associated with publicly available objects.

[0098] In some implementations, search area indicators G1-G5 may be shown in response to web page W1 being loaded. Selecting at least one of search area indicators G1-G5 may search the search area associated with the search area indicator. In other words, a search query may be triggered in response to the search area indicator being triggered (the search query may be triggered, for example, as shown in FIG. 1B). As a specific example, search area indicator G2 associated with image I2 may be selected (instead of selecting search area SA as shown and described in FIG. 1A). Selecting search area indicator G2 associated with image I2 may result in at least a portion of image I2 (e.g., search area SA) being searched, as shown in FIG. 1B.

[0099] In some implementations, a search area indicator indicating a searchable area may be displayed in response to activation of a user interface for searching. FIG. 11A illustrates activation (e.g., activation by a user) of a search input area 136 that can be used to define (e.g., trigger a search query). The search input area 136 may include a text and / or image input area (not shown), where a user can, for example, input text and / or images to trigger a search (e.g., one or more search queries) by a search server. In response to activation of the search input area 136, search area indicators G1 through G5 are triggered (e.g., rendered), as shown in FIG. 11B.

[0100] 12 illustrates examples of a general-purpose computing device 2000 and a general-purpose mobile computing device 2050 that can be used with the techniques described herein. Computing device 2000 is intended to represent various types of digital computers, such as laptops, desktops, tablets, workstations, personal digital assistants, televisions, servers, blade servers, mainframes, and other suitable computing devices. Computing device 2050 is intended to represent various types of mobile devices, such as personal digital assistants, mobile phones, smartphones, and other similar computing devices. The components illustrated herein, their connections and relationships, and their functions are for illustrative purposes only and are not intended to limit the scope of the invention as described and / or claimed in this document.

[0101] Computing device 2000 includes a processor 2002, memory 2004, a storage device 2006, a high-speed interface 2008 connecting to memory 2004 and a high-speed expansion port 2010, and a low-speed interface 2012 connecting to a low-speed bus 2014 and storage device 2006. Processor 2002 may be a semiconductor-based processor. Memory 2004 may be semiconductor-based memory. Each of components 2002, 2004, 2006, 2008, 2010, and 2012 are interconnected using various buses and may reside on a common motherboard or otherwise exist as desired. Processor 2002 processes instructions for execution within computing device 2000, including instructions stored in memory 2004 or storage device 2006, and can display graphical information for a GUI on an external input / output device, such as a display 2016 connected to high-speed interface 2008. In other implementations, multiple processors and / or multiple buses may be used as needed, along with multiple memories and memory types. Also, multiple computing devices 2000 may be connected, each performing some of the required operations (eg, as a bank of servers, a group of blade servers, or a multi-processor system).

[0102] The memory 2004 stores information within the computing device 2000. In one implementation, the memory 2004 is a volatile memory unit or units. In another implementation, the memory 2004 is a non-volatile memory unit or units. The memory 2004 may also be another form of computer-readable medium, such as a magnetic disk or optical disk.

[0103] The storage device 2006 can provide mass storage for the computing device 2000. In one embodiment, the storage device 2006 can be or include a computer-readable medium, such as a floppy disk device, a hard disk device, an optical disk device, or a number of devices, including a tape device, a flash memory or other similar solid-state memory device, or a storage area network or other configuration of devices. The computer program product can be tangibly embodied on an information carrier. The computer program product can also include instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer-readable or machine-readable medium, such as the memory 2004, the storage device 2006, or the memory of the processor 2002.

[0104] The high-speed controller 2008 manages the bandwidth-intensive operations of the computing device 2000, while the low-speed controller 2012 manages the less bandwidth-intensive operations. This allocation of functionality is merely exemplary. In one embodiment, the high-speed controller 2008 is coupled to the memory 2004, the display 2016 (e.g., via a graphics processor or accelerator), and the high-speed expansion port 2010, which can accept various expansion cards (not shown). In an embodiment, the low-speed controller 2012 is coupled to the storage device 2006 and the low-speed expansion port 2014. The low-speed expansion port may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet, etc.) and can connect to one or more input / output devices, such as a keyboard, pointing device, scanner, or network devices, such as a switch or router, via a network adapter or the like.

[0105] The computing device 2000 can be implemented in many different forms, as shown. For example, it can be implemented as a standard server 2020, or multiple times in a group of such servers. It can also be implemented as part of a rack server system 2024. It can also be implemented in a personal computer, such as a laptop computer 2022. Alternatively, the components of the computing device 2000 can be combined with other components in a mobile device (not shown), such as device 2050. Each such device can include one or more of the computing devices 2000, 2050, and the entire system may consist of multiple computing devices 2000, 2050 communicating with each other.

[0106] Computing device 2050 includes components such as a processor 2052, memory 2064, input / output devices such as a display 2054, a communications interface 2066, and a transceiver 2068. Device 2050 may also be provided with a storage device such as a microdrive or other device to provide additional storage. Each of components 2050, 2052, 2064, 2054, 2066, and 2068 are interconnected using various buses, and some of the components may be mounted on a common motherboard or present in other ways as desired.

[0107] The processor 2052 can execute instructions within the computing device 2050, including instructions stored in the memory 2064. The processor may be implemented as a chipset of chips including multiple individual analog and digital processors. The processor can provide coordination of other components of the device 2050, such as control of a user interface, applications executed by the device 2050, and wireless communications by the device 2050.

[0108] The processor 2052 can communicate with a user via a control interface 2058 and a display interface 2056 connected to a display 2054. The display 2054 can be, for example, a TFT LCD (thin film transistor liquid crystal display) or an OLED (organic light emitting diode) display, or other suitable display technology. The display interface 2056 can include appropriate circuitry for driving the display 2054 to present graphical and other information to the user. The control interface 2058 receives and converts commands from the user for transmission to the processor 2052. Additionally, an external interface 2062 can be provided in communication with the processor 2052, enabling short-range communication between the device 2050 and other devices. The external interface 2062 can, for example, provide wired communication in some implementations and wireless communication in other implementations; multiple interfaces can also be used.

[0109] The memory 2064 stores information within the computing device 2050. The memory 2064 may be implemented as one or more computer-readable media, one or more volatile memory units, or one or more non-volatile memory units. Expansion memory 2074 may also be provided and connected to the device 2050 via an expansion interface 2072, which may include, for example, a SIMM (Single In-Line Memory Module) card interface. Such expansion memory 2074 may provide additional storage space for the device 2050 or may also store applications or other information for the device 2050. Specifically, the expansion memory 2074 may include instructions for performing or supplementing the processes described above and may also include secure information. Thus, for example, the expansion memory 2074 may be provided as a security module for the device 2050 and may be programmed with instructions that enable secure use of the device 2050. Furthermore, secure applications may be provided via SIMM cards, with additional information, such as identification information, placed on the SIMM card in an unhackable manner.

[0110] The memory may include, for example, flash memory and / or NVRAM memory, as described below. In one embodiment, the computer program product is tangibly embodied on an information carrier. The computer program product also includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as memory 2064, expansion memory 2074, or memory of processor 2052, and may be received, for example, via transceiver 2068 or external interface 2062.

[0111] Device 2050 can communicate wirelessly via communication interface 2066, which can include digital signal processing circuitry if necessary. Communication interface 2066 can provide communication in various modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others. Such communication may occur, for example, via radio frequency transceiver 2068. Additionally, short-range communication may occur using Bluetooth, WiFi, or other such transceivers (not shown). Furthermore, Global Positioning System (GPS) receiver module 2070 can provide additional navigation- and location-related wireless data to device 2050, which may be used as needed by applications executing on device 2050.

[0112] Device 2050 can also perform voice communications using audio codec 2060, which can receive voice information from a user and convert it into usable digital information. Audio codec 2060 can likewise generate sounds that are heard by the user, such as through a speaker in a handset of device 2050. Such sounds can include voice telephone calls, recorded sounds (e.g., voice messages, music files, etc.), and sounds generated by applications running on device 2050.

[0113] The computing device 2050, as shown, can be implemented in many different forms. For example, it may be implemented as a mobile phone 2080. It may also be implemented as part of a smartphone 2082, personal digital assistant, or other similar mobile device.

[0114] Some examples are described below. Example 1: A method comprising: Displaying a web page within the browser's viewing area; receiving a selection of a region of the web page; In response to receiving the selection of the region, An image corresponding to the region is determined, triggering a search query based on the image; receiving search results in response to the search query; and The method includes displaying the search results in a search results area outside the display area, the search results area being within the browser.

[0115] Example 2: Receiving an instruction to interact with the search results; and 10. The method of example 1, further comprising replacing the display area with a navigation area in which content associated with the search results is displayed in response to an interaction with the search results area.

[0116] Example 3: Receiving an instruction to interact with the search results; and 3. The method of any one of Examples 1 to 2, further comprising replacing the web page in the display area with content associated with the search results.

[0117] Example 4: The display area is in the first tab of the browser, The method comprises: receiving an instruction to interact with the search results; and The method of any one of Examples 1 to 3, further comprising triggering the display of content associated with the search result in a second tab of the browser.

[0118] Example 5: The browser corresponds to a first browser window, The method comprises: receiving an instruction to interact with the search results; and The method of any one of the preceding examples, further comprising triggering display of content associated with the search result in a second browser window separate from the first browser window.

[0119] Example 6: The method of any one of the preceding examples, wherein the size of the display area is changed prior to receiving the selection of the area of ​​the web page such that the search results area is displayed within the area allocated to the display area.

[0120] Example 7: The method of any one of the preceding examples, wherein the image is a screen capture of the area.

[0121] Example 8: The method of any one of the preceding examples, wherein the search results area is within a tab in the browser.

[0122] Example 9: The method of Examples 4 and 8, wherein the search results area is within the first tab. Example 10: The method of any one of the preceding examples, wherein the web page includes a plurality of images displayed within the web page, and the region includes at least a portion of each of at least two images of the plurality of images.

[0123] Example 11: The method of any one of the preceding examples, wherein the display area is reduced in size as the search result area is displayed.

[0124] Example 12: The method of any one of the preceding examples, wherein the image is a first image and the region includes at least a portion of a second image rendered within the web page.

[0125] Example 13: A method comprising: Displaying a web page within the browser's display area; displaying a search area indicator associated with the region of the web page; receiving a selection of said search area indicator; In response to receiving the selection of the search area indicator, An image corresponding to the region is determined, triggering a search query based on the image; receiving search results in response to the search query; and The method includes displaying the search results in a search results area outside the display area, the search results area being within the browser.

[0126] Example 14: The method of Example 13, further comprising activating a search input area, wherein said displaying of said search area indicator occurs in response to said search input area being activated.

[0127] Example 15: The method of Example 14, wherein the search input field corresponds to the search result field. Example 16: The method of example 14 or 15, wherein the search input area is activated in response to user interaction.

[0128] Example 17: A system comprising: a processor, and a memory device communicatively coupled to the processor and storing a plurality of instructions that, when executed by the processor, cause the processor to: Displaying a web page within the browser's display area, receiving a selection of a region of the web page; In response to receiving the selection of the region, An image corresponding to the region is determined, triggering a search query based on the image; receiving search results in response to the search query; and The system displays the search results in a search results area outside the display area, the search results area being within the browser.

[0129] Example 18: The memory device further stores instructions that, when executed by the processor, cause the processor to: receiving instructions for interacting with the search results; and 18. The system of Example 17, wherein the display region is replaced with a navigation region in which content associated with the search results is displayed in response to the interaction with the search results region.

[0130] Example 19: The memory device further stores instructions that, when executed by the processor, cause the processor to: receiving instructions for interacting with the search results; 18. The system of example 17, wherein the web page in the display region is replaced with content associated with the search result.

[0131] Example 20: The display area is in the first tab of the browser, The memory device further stores instructions that, when executed by the processor, cause the processor to: receiving instructions for interacting with the search results; 20. The system of any one of Examples 17 to 19, wherein the system triggers display of content associated with the search result in a second tab of the browser.

[0132] Example 21: A machine-readable tangible medium storing instructions, the instructions being readable and executable by one or more processors for performing a method, the method comprising: Displaying a web page within the browser's display area; receiving a selection of a region of the web page; In response to receiving the selection of the region, An image corresponding to the region is determined, triggering a search query based on the image; receiving search results in response to the search query; and and displaying the search results in a search results area outside the display area, the search results area being within the browser.

[0133] Example 22: Receiving an instruction to interact with the search result; and 22. The machine-readable tangible medium of Example 21, further comprising replacing the display region with a navigation region that displays content associated with the search results in response to an interaction with the search results region.

[0134] Various implementations of the systems and techniques described herein may be realized in digital electronic circuitry, integrated circuits, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementation in one or more computer programs executable and / or interpretable by a programmable system including at least one programmable processor, which may be specialized or general-purpose, coupled to receive data and instructions from the storage system, and to transmit data and instructions to the storage system.

[0135] These computer programs (also known as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural and / or object-oriented programming language, and / or assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., magnetic disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives the machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0136] To provide for interaction with a user, the systems and techniques described herein are implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) by which the user can provide input to the computer. Other types of devices can also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user can be received in any form, including acoustic, verbal, or tactile input.

[0137] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or that includes middleware components (e.g., an application server), or that includes a front-end component (e.g., a client computer having a graphical user interface or web browser through which a user can interact with an implementation of the systems and techniques described herein), or a combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

[0138] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and by virtue of the client-server relationship they have to each other.

[0139] A number of embodiments have been described, and it will of course be understood that various modifications can be made without departing from the spirit and scope of the invention.

[0140] Additionally, the logic flows depicted in the figures do not require the particular order or sequential order shown to achieve the desired results. Furthermore, other steps may be provided in or removed from the described flows, and other components may be added to or removed from the described systems.

Claims

1. 1. A method comprising: Displaying a web page within the browser's viewing area; receiving a selection of a region of the web page; In response to receiving the selection of the region, determining an image corresponding to the region; triggering a search query based on the image; receiving search results in response to the search query; and displaying the search results in a search result area outside the display area, the search result area being within the browser; The method, wherein the search results area is integrated with the browser and includes at least a portion of a background or address bar theme of the browser.

2. receiving an indication of an interaction with the search results; and The method of claim 1 , further comprising replacing the display area with a navigation area in which content associated with the search results is displayed in response to the interaction with the search results.

3. receiving an indication of an interaction with the search results; and The method of claim 1 or 2, further comprising replacing the web page in the display area with content associated with the search results.

4. the display area is in a first tab of the browser; The method comprises: receiving an indication of an interaction with the search results; and The method of claim 1 or 2, further comprising triggering the display of content associated with the search results in a second tab of the browser.

5. the browser corresponds to a first browser window; The method comprises: receiving an indication of an interaction with the search results; and The method of claim 1 or 2, further comprising triggering display of content associated with the search result in a second browser window separate from the first browser window.

6. 3. The method of claim 1, wherein the size of the display area is changed prior to receiving the selection of the region of the web page so that the search results region is displayed within the area allocated to the display area.

7. The method of claim 1 or 2, wherein the image is a screen capture of the area.

8. The method of claim 1 or 2, wherein the search results area is in a tab of the browser.

9. The method of claim 4 , wherein the search results area is in the first tab of the browser.

10. The method of claim 1 or 2, wherein the web page includes a plurality of images displayed within the web page, and the region includes at least a portion of each of at least two of the plurality of images.

11. The method of claim 1 or 2, wherein the size of the display area is reduced as the search result area is displayed.

12. The method of claim 1 or 2, wherein the image is a first image and the region includes at least a portion of a second image rendered within the web page.

13. 1. A method comprising: Displaying a web page within the browser's display area; causing the display of a search region indicator associated with the region of the web page; receiving a selection of the search area indicator; In response to receiving the selection of the search area indicator, determining an image corresponding to the region; triggering a search query based on the image; receiving search results in response to the search query; and displaying the search results in a search result area outside the display area, the search result area being within the browser; The method, wherein the search results area is integrated with the browser and includes at least a portion of a background or address bar theme of the browser.

14. The method of claim 13 , further comprising activating a search input area, wherein the display of the search area indicator occurs in response to the search input area being activated.

15. The method of claim 14 , wherein the search input area corresponds to the search result area.

16. The method of claim 14 or 15, wherein the search input area is activated in response to a user interaction.

17. 1. A system comprising: a processor, and a memory device communicatively coupled to the processor and storing a plurality of instructions, the plurality of instructions, when executed by the processor, causing the processor to: Displaying a web page within the browser's display area, receiving a selection of a region of the web page; In response to receiving the selection of the region, determining an image corresponding to the region; triggering a search query based on the image; receiving search results in response to the search query; displaying the search results in a search result area outside the display area, the search result area being within the browser; The system, wherein the search results area is integrated with the browser and includes at least a portion of a background or address bar theme of the browser.

18. The memory device further stores instructions that, when executed by the processor, cause the processor to: receiving an instruction to interact with the search results; 20. The system of claim 17, wherein the display area is replaced with a navigation area that displays content associated with the search results in response to the interaction with the search results area.

19. The memory device further stores instructions that, when executed by the processor, cause the processor to: receiving an instruction to interact with the search results; The system of claim 17 , further comprising: causing the web page in the display area to be replaced with content associated with the search results.

20. the display area is in a first tab of the browser; The memory device further stores instructions that, when executed by the processor, cause the processor to: receiving an instruction to interact with the search results; 19. The system of claim 17 or 18, further comprising triggering the display of content associated with the search results in a second tab of the browser.

21. A computer program comprising instructions that can be read and executed by one or more processors to perform a method, the method comprising: Displaying a web page within the browser's display area; receiving a selection of a region of the web page; In response to receiving the selection of the region, determining an image corresponding to the region; triggering a search query based on the image; receiving search results in response to the search query; and displaying the search results in a search result area outside the display area, the search result area being within the browser; The computer program, wherein the search results area is integrated with the browser and includes at least a portion of a background or address bar theme of the browser.

22. receiving an indication of an interaction with the search results; and 22. The computer program product of claim 21, further comprising replacing the display area with a navigation area in which content associated with the search results is displayed in response to an interaction with the search results area.

Images