Visual search request initiating method, visual search method and electronic equipment

Abstract

The embodiment of the invention discloses a visual search request initiating method, a visual search method and electronic equipment, and the method comprises the steps: starting the perception of an element dragging event after the loading of a visual search function of a search component in a commodity information page is completed; after sensing an element dragging event, if a dragged target element is related to visual data, activating an operation panel corresponding to the visual search function; and after the target element is dragged to the target area in the operation panel, submitting a search request for commodity search based on the visual data content corresponding to the target element. Through the embodiment of the invention, the operation complexity of initiating the visual search request can be reduced, and the probability of intention interruption caused by operation failure is reduced.

Description

Technical Field

[0001] The present application relates to the field of information search technology, and in particular to initiating a visual search request, a visual search method, and an electronic device. Background Art

[0002] In information service systems, in addition to searching for information by entering keywords (referred to as "text search"), users also commonly use visual data (including images, videos, and other visual content, or other types of visual data) to search for information (referred to as "visual search" for ease of description). For example, in product information service systems, searching for products by entering images is very common, especially in cross-border scenarios. Globalized Business Users (Business Users, individuals who make decisions, purchase, or use products / services on behalf of a business or organization in a business setting; they typically do not participate in transactions as individual consumers but rather act based on the interests and needs of the business) may not be as effective as expected when searching for products through text searches due to regional cultural and multilingual differences. Therefore, Business Users often use visual search functions more frequently during procurement and sourcing, such as initiating visual searches using existing product images to find identical or similar products.

[0003] However, when using visual search, users may encounter lengthy operation paths and a certain probability of failure. For example, before initiating an image search, users first need to obtain specific visual data, such as the image itself. This visual data may also come from the internet. For example, when browsing a product on an e-commerce platform, a user may need to search for similar or identical products based on the product image to compare across multiple merchants. Therefore, when using visual search, users may need to go through the following process: downloading the product image from the internet to a local folder and saving it, opening the visual search panel on the system page, selecting the product image in a local folder, and then uploading the product image. This process can be lengthy. For users unfamiliar with the operation process, it can be complex, and any failure at any step can disrupt the user's search intent. For example, a newly downloaded image may not be found, or even downloading may fail. For example, some images may have links attached to them. When users right-click on an image to download it, they may find that the right-click menu lacks options related to image downloading, such as "Save Image" or "Save Image As." This prevents the image from being downloaded, and consequently, prevents them from searching based on that image. Summary of the Invention

[0004] The present application provides a visual search request, a visual search method, and an electronic device, which can reduce the operational complexity of initiating a visual search request and reduce the probability of intention interruption due to operation failure.

[0005] This application provides the following solutions:

[0006] A method for initiating a visual search request, comprising:

[0007] After the visual search function of the search component on the product information page is loaded, the perception of element drag events is enabled;

[0008] After sensing an element drag event, if the dragged target element is related to the visual data, activating an operation panel corresponding to the visual search function;

[0009] After the target element is dragged to the target area in the operation panel, a search request for searching for products based on the visual data content corresponding to the target element is submitted.

[0010] Wherein, after sensing the element drag event, if the dragged element is related to the visual data, activating the operation panel corresponding to the visual search function of the search component includes:

[0011] After sensing an event in which a target page element in the product information page is dragged, it is determined whether the target page element is related to visual data. If so, an operation panel corresponding to the visual search function of the search component is activated.

[0012] Wherein, determining whether the target page element is related to the visual data includes:

[0013] Determine whether the target page element is an image element. If so, determine the image content of the image element as the visual data content corresponding to the target element.

[0014] Wherein, determining whether the target page element is related to the visual data includes:

[0015] Determine whether the target page element is a link-type element containing a background image. If so, extract the background image of the link-type element by parsing the composite style information of the link-type element, so as to determine the image content of the extracted background image as the visual data content corresponding to the target element.

[0016] Wherein, the content area of ​​the product information page includes multiple resource locations;

[0017] The link element including a background image includes: a product details page link in the form of an information card displayed in the resource position, and the information card includes the background image of the link.

[0018] Wherein, determining whether the target page element is related to the visual data includes:

[0019] It is determined whether the target page element is a canvas element and a valid image exists in the canvas element. If so, the image content of the valid image in the canvas element is used as the visual data content corresponding to the target element.

[0020] Wherein, after sensing the element drag event, if the dragged element is related to the visual data, activating the operation panel corresponding to the visual search function of the search component includes:

[0021] Upon sensing an event of dragging a page element from another information page into the product information page, if the dragged page element is related to visual data, activating an operation panel corresponding to a visual search function of the search component in the product information page;

[0022] The other information pages include other information pages in the same website to which the product information page belongs, or information pages on other websites.

[0023] Wherein, after sensing the element dragging event, if the dragged element is related to the visual data, activating the operation panel corresponding to the visual search function includes:

[0024] After sensing an event of dragging a target element in a local folder of the terminal device to the information page, if the dragged element is related to visual data, an operation panel corresponding to the visual search function is activated.

[0025] Among them, also include:

[0026] After the element drag event is sensed, a thumbnail of the dragged element is generated, and a visual effect of the thumbnail following the movement of the cursor is provided.

[0027] Wherein, after the target element is dragged to the target area in the operation panel, submitting a search request for searching for products based on the visual data content corresponding to the target element includes:

[0028] After a first event of the target element being dragged to a target area in the operation panel is sensed, submission of the search request is triggered by the first event.

[0029] Wherein, after the target element is dragged to the target area in the operation panel, submitting a search request for searching for products based on the visual data content corresponding to the target element includes:

[0030] After sensing a second event in which the target element is dragged to a target area in the operation panel and a target operation option in the operation panel is operated, the second event triggers submission of the search request.

[0031] There are multiple target elements dragged to the target area in the operation panel;

[0032] The submitting of the search request triggered by the second event includes:

[0033] After the second event is perceived, the search request is generated and submitted according to the visual data contents corresponding to the multiple target elements and the pre-configured logical relationship between the multiple target elements.

[0034] A method for initiating a visual search request, comprising:

[0035] After the visual search function of the search component in the information page is loaded, the perception of element drag events is enabled;

[0036] After sensing an element drag event, if the dragged target element is related to the visual data, activating an operation panel corresponding to the visual search function;

[0037] After the target element is dragged to the target area in the operation panel, a search request for information search based on the visual data content corresponding to the target element is submitted.

[0038] A visual search method, comprising:

[0039] After the visual search function of the search component in the information page is loaded, the perception of element drag events is enabled;

[0040] After sensing an element drag event, if the dragged target element is related to the visual data, activating an operation panel corresponding to the visual search function;

[0041] After the target element is dragged to the target area in the operation panel, a search request for information search based on the visual data content corresponding to the target element is submitted to the server, so that the server processes the search request and obtains search results;

[0042] The search results returned by the server are displayed.

[0043] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the aforementioned methods.

[0044] An electronic device, comprising:

[0045] one or more processors; and

[0046] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of any of the aforementioned methods.

[0047] A computer program product comprises a computer program / computer executable instructions, wherein the computer program / computer executable instructions are capable of implementing the steps of any of the aforementioned methods when executed by a processor in an electronic device.

[0048] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0049] Through the embodiment of the present application, after the visual search function of the search component in the product information page is loaded, the perception of the element drag event can be started. After the element drag event is perceived, if the dragged target element is related to the visual data, the operation panel corresponding to the visual search function of the search component can be automatically activated. Afterwards, after the target element is dragged to the target area in the operation panel, a search request for product search based on the visual data content corresponding to the target element can be submitted. In this way, since the drag interaction mechanism is introduced in the visual search scenario, 5 or even more click operations in the traditional visual search process can be reduced to one drag and drop, and the time it takes for the user to complete a visual search action is reduced from minutes to seconds. Therefore, the user experience can be improved, the complexity of the operation can be reduced, and the probability of intention interruption due to operation failure can be reduced.

[0050] Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 This is a schematic diagram of system interaction provided by an embodiment of the present application;

[0053] Figure 2 is a flowchart of the first method provided in an embodiment of the present application;

[0054] Figure 3 This is a schematic diagram of the interface provided by the embodiment of the present application;

[0055] Figure 4 is a flow chart of the second method provided in an embodiment of the present application;

[0056] Figure 5 is a flowchart of the third method provided in an embodiment of the present application;

[0057] Figure 6 Schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0059] In an embodiment of the present application, a corresponding solution is provided to enable users to initiate visual search requests more efficiently. In this solution, the function of the search component in the information page can be extended. The specific extension method is that if the search component has a visual search function, the perception of the element drag event can be started after the visual search function is loaded. The user does not have to download visual data such as pictures in advance, nor does he have to manually open the operation panel corresponding to the visual search function of the search component (which can be referred to as the visual search panel). Instead, he can trigger the process of entering the visual search by directly dragging the target element. Accordingly, after the search component perceives the element drag event, it can determine whether the dragged target element is related to the visual data. If so, the visual search panel of the search component can be automatically activated, and the user can drag the target element to the target area within the visual search panel. At this point, the search component can perceive the drag completion event and use it to trigger the submission of a search request for a product search based on the visual data content corresponding to the target element. In this way, the 5 or even more click operations in the traditional image search process can be reduced to a single drag operation, thereby improving efficiency and reducing the chance of interruption of search intent.

[0060] Specifically, the target element to be dragged can come from a variety of sources. For example, one option is to directly originate from the information page where the search component resides. For example, pages with a search component are often information aggregation pages, such as client homepages, store pages, event venue pages, and so on in a product information service system. The content area of ​​such pages often contains various elements, including elements related to visual data. Therefore, users can initiate a visual search process by dragging such an element. For example, the client homepage typically provides users with a recommended product information stream, which includes multiple resource slots. Each resource slot can display a product link in the form of an information card, etc. The information card often also includes a product image. In this case, although the information card is essentially a link and the product image may exist as the link's background image, users can still roughly determine whether they are interested in the product based on the product image displayed in the information card, and may have the need to search for similar products based on the product image. Therefore, users can initiate a visual search process by dragging such a product image. In addition, the page may also include some image elements. Users can also directly drag and drop these image elements to initiate the visual search process. Furthermore, the page may also include some canvas elements, which allow images to be generated directly on the web page. Therefore, if such canvas elements exist on the page and the canvas contains valid image content, the image content can also be dragged as the target element to initiate the visual search process.

[0061] In the case of directly dragging a target element within the information page where the search component resides, the search component's visual search panel will automatically activate as long as it detects that the user has begun dragging the target element and the target element is related to the visual data. For example, if a user clicks on an element with the mouse and then moves the mouse while holding down the left mouse button, the search component will detect the drag operation on the target element and activate the visual search panel.

[0062] Another source of target elements may also be other information pages. For example, the information page having the search component provided in the embodiment of the present application is page A. In the aforementioned method, the user can directly select a target element from page A and drag it. In another method, the user can also select a page element from page B and drag it to page A. At this time, the search component in page A can also perceive the element dragging operation, and when the dragged target element is related to the visual data, the visual search panel can also be activated. The user can place the dragged element in the target area of ​​the visual search panel to complete the submission of the visual search request.

[0063] For example, suppose that the above-mentioned page A is the client homepage of a certain commodity information service system. When the user opens page A through a web browser and is also browsing page B, page A and page B will become two different tabs in the web browser, and the tab bar of the browser will display the label options of the two pages. While browsing page B, if the user finds an image or other element related to a certain commodity and wants to search whether there is the same or similar commodity in the commodity information service system. At this time, if the method of the prior art is used, the user needs to first download the image in page B to the local computer, then open the visual search panel in page A, and then perform a series of operations such as selecting the image from the local folder, and then perform a series of operations such as uploading the image. In the embodiment of the present application, the user only needs to drag the image element in page B to page A. For example, while dragging a certain image element, the user can first move the mouse pointer to the label option where page A is located in the tab bar, and then move it to the page display area of ​​the browser, and then page A will gain the operation focus. At this point, the search component in Page A will be able to sense the element drag event and determine whether the dragged element is related to the visual data. For example, if it determines that the dragged element is an image, it can automatically activate the visual search panel. The user can drag the element into the target area of ​​the visual search panel to trigger the submission of the visual search request. The above-mentioned Page B and Page A can be different pages within the same website, or they can belong to a different website from Page A, which is not limited here.

[0064] Another source of target elements can be the content in the local folder of the user terminal device. For example, assuming that the information page with the search component provided by the embodiment of the present application is page A, when page A is open, the user can drag an element such as a picture in the local folder of the terminal device to page A. At this time, the search component in page A can sense the drag event and can determine whether the dragged element is related to the visual data. If so, the visual search panel of the search component in page A can be automatically activated, and the user can complete the submission of the visual search request by placing the dragged element in the target area of ​​the visual search panel.

[0065] In summary, the embodiments of this application provide a drag-and-drop interaction mechanism for visual search scenarios. By monitoring the drag event stream, the user's operation path can be simplified, reducing the traditional multi-step process of image search (save image - open panel - select file - upload) to a single drag-and-drop process. This can improve the user experience, reduce operational complexity, and reduce interruptions caused by operational failures, reducing visual search time from minutes to seconds or even milliseconds.

[0066] From the perspective of system architecture, the embodiment of the present application mainly improves the visual search function of the search component in the information page. The above-mentioned improved search component can be applied to a variety of specific information pages. In other words, any information page that uses the search component provided by the embodiment of the present application can implement the function of quickly initiating visual search provided by the embodiment of the present application. Among them, after the user opens an information page (for example, a product information page such as the client homepage in the product information service system), the components in the page will be automatically loaded, including the search component. If the search component supports the visual search function, the function will also be loaded. Specifically, taking the method of performing element dragging interaction in the information page where the search component is located as an example, see Figure 1 , the specific interaction process may include: after the information page is opened and the visual search function is loaded, the perception of the element drag event can be initiated. Afterwards, when the user is browsing the page, if he needs to initiate a visual search based on a browsed element, he can perform a specific element drag operation, and the search component can perceive the operation and automatically activate the visual search panel. After the user drags the specific element to the target area of ​​the panel, the search component can perceive the drag completion event, and therefore, it can automatically trigger the submission of the visual search request. The specific search request will be submitted to the server, and the server will perform a specific visual search, including retrieving search results that match the input visual data content from databases such as the product information library, and returning them to the client for display.

[0067] The specific implementation scheme provided in the embodiments of this application is introduced in detail below.

[0068] Example 1

[0069] First, this embodiment provides a method for initiating a visual search request for a product search application in a product information service system. The execution subject of this method can be a search component in a product information page (including the client homepage, store page, event venue page, etc. of the product information service system). Figure 2 , the method may specifically include:

[0070] S201: After the visual search function of the search component on the product information page is loaded, the perception of the element drag event is started.

[0071] In an embodiment of the present application, since the search process is initiated by dragging and dropping, the perception of the element drag event can be started after the visual search function of the search component in the page is loaded, so that the user can initiate the search process by dragging the element without manually opening the visual search panel in advance.

[0072] Among them, since some commonly used page design languages ​​usually support drag events, it is possible to determine whether the user has performed a drag operation by sensing these events. For example, the Drag and Drop event is defined in JS (JavaScript), that is, the process in which the user uses the mouse to select a draggable element, drags the element to a droppable element, and releases the mouse button to drop the elements. Specifically, it may include the dragstart event (mouse button press event, an event triggered when the element starts to be dragged), the dragenter event (mouse enter element event: an event triggered when the dragged element enters the droppable element), the dragover event (mouse on an element event: an event triggered when the dragged element moves over the droppable element), the drop event (mouse release event: an event triggered when the dragged element is on the droppable element and the mouse is released), the dragend event (an event triggered after the drag is completed), and so on. In specific implementation, in an embodiment of the present application, some or all of the above-mentioned multiple events can be sensed to sense the event of a specific element being dragged.

[0073] S202: After an element drag event is sensed, if the dragged target element is related to the visual data, an operation panel corresponding to the visual search function is activated.

[0074] After sensing a specific element drag event, it can first determine whether the specific target element is related to visual data, for example, whether it is an image element such as a picture, or a link element with a background image, etc. If so, the operation panel corresponding to the visual search function of the search component can be automatically activated.

[0075] As previously mentioned, the specific dragged element can have a variety of sources. In one case, it can be a target element in the content area of ​​the page where the search component is located. In this case, after sensing the event that the target page element is dragged in the content area of ​​the product information page, it can be determined whether the target page element is related to the visual data. If so, the operation panel corresponding to the visual search function of the search component is activated.

[0076] For example, Figure 3 As shown, it is a product information page, which includes the improved search component provided by the embodiment of the present application. Among them, 33 is the visual search operation panel of the search component. In the default state, the panel is closed. If the user performs a drag operation on the element shown in 31, the opening of the operation panel will be activated, as shown in FIG. Figure 3The state shown at 33 in the figure. During the dragging process, a thumbnail of the dragged target element can be displayed on the dragging track, for example, Figure 3 32 is a thumbnail of a certain point on the dragging track.

[0077] To determine whether the target page element being dragged is relevant to the visual data, an intelligent content recognition engine can be built to determine the type of the target element being dragged through multimodal feature extraction. Multimodal features refer to features of information from multiple different modalities. Multimodal information can include DOM (Document Object Model) structure analysis, CSSOM (CSS Object Model, CSS is the abbreviation for Cascading Style Sheets) style parsing, Canvas pixel detection, SVG (Scalable Vector Graphics) parsing, and more.

[0078] In specific implementation, it can be determined whether the target page element is an image element. If so, the image content of the image element can be directly determined as the visual data content corresponding to the target page element.

[0079] Alternatively, it is also possible to determine whether the target page element is a link-type element containing a background image. If so, the background image of the link-type element can be extracted by parsing the composite style information (CSSOM style, etc.) of the link-type element. Then, the image content of the extracted background image can be determined as the visual data content corresponding to the target element. In actual applications, it is more common for the target page element being dragged to be a link-type element. In an embodiment of the present application, even if a link is being dragged (that is, not directly an image-type element such as a picture), as long as the specific link has a background image (such as a product image displayed in a product information card-style detail page link), this background image can be extracted from it, and the dragged element can be determined as an element related to the visual data, and then the visual search panel can be automatically activated. For example, in a specific implementation, since the specific search component is located in the product information page, and the content area of ​​the product information page may include multiple resource locations, each resource location can display a product detail page link in the information card style. At this time, the product image displayed in the specific information card is actually the background image of the specific link. In traditional methods, if a user needs to search for products based on a product image, they may need to download the product image. However, since it is actually a link, there may be situations where the image cannot be downloaded, making it difficult to continue searching for products based on the product image. In the embodiment of the present application, the user only needs to perform a drag operation on the product image, and the search component will detect the drag event and extract the background image found in the link from it, and then submit a visual search request based on the background image.

[0080] Furthermore, some product information pages may include canvas elements to support users in editing images on the web page, for example Figure 3 The "Cropped Image" area displayed on the left side of the center includes a canvas element, where users can perform operations such as cropping product images based on their specific needs. Accordingly, users may also want to initiate a search based on valid images within the canvas element value. Therefore, it is also possible to determine whether the target page element is a canvas element and whether a valid image exists within the canvas element. If so, the image content of the valid image within the canvas element is used as the visual data content corresponding to the target element.

[0081] In addition to dragging page elements within the information page where the search component resides, page elements from other information pages can also be dragged. Upon detecting that a page element from another information page has been dragged into the current product information page, the search component within the current product information page can determine whether the dragged page element is related to visual data. If so, the control panel corresponding to the visual search function of the search component within the product information page can be activated. The determination of the type of page element dragged from another page can also be implemented using the aforementioned intelligent content recognition engine. In a specific implementation, for elements dragged from other pages, only image elements can be supported, or link elements containing background images can also be supported. For the latter, since the element is from another information page, if it is difficult to directly extract the background image data found in the link through composite style information analysis, the relevant image data content can be obtained by extracting information contained in the page to which the link points.

[0082] Furthermore, you can also use target elements such as images in the local folder of the terminal device as drag objects. In this case, after the search component in the current specific product information page senses the event of dragging the target element in the local folder of the terminal device to the product information page, it can also determine whether the dragged element is related to the visual data. If so, it can activate the operation panel corresponding to the visual search function. Among them, the target elements local to the terminal device can mainly be image files, etc.

[0083] Of course, in actual implementation, the target element being dragged may also have other sources, which will not be introduced here one by one.

[0084] In addition, after sensing an element drag event, a thumbnail of the dragged element can be generated, and a visual effect of the thumbnail following the cursor movement can be provided. In other words, as mentioned above, while the target element is being dragged, a thumbnail of the dragged target element can be displayed on the dragging track to achieve a real-time prompt of the image visual dragging effect. In addition, the reason why a thumbnail is displayed on the dragging track instead of the original image size is to avoid excessive obstruction of other content on the current product information page.

[0085] S203: After the target element is dragged to the target area in the operation panel, a search request for searching for products based on the visual data content corresponding to the target element is submitted.

[0086] After the target element is dragged to the target area of ​​the visual search operation panel, a search request for searching for goods based on the visual data content corresponding to the target element can be submitted. The submission of the search request can be triggered in a variety of ways. For example, in one way, the submission of a search request for searching for goods based on the visual data content corresponding to the target element can be automatically triggered based on the event of the completion of the drag (which can be called the first event). Alternatively, in another way, an operation option such as "Submit" can also be provided in the operation panel, and the user can trigger the submission of the above-mentioned search request by clicking on the operation option after completing the drag operation. In other words, for the latter, it can be after the second event that the target element is perceived to be dragged to the target area in the operation panel and the target operation option in the operation panel is operated, the submission of the search request is triggered by the second event.

[0087] Among them, regarding the visual data content corresponding to the target element, the information of the specific target element can be obtained through the drag and drop API (Application Programming Interface) provided by JS, etc. For example, for the target element of the image type, the specific information obtained may include the file name, file type, image content, etc. of the image, wherein, regarding the image content, it can be read as Base64 (one of the encoding methods for transmitting 8-bit bytecode, a method for representing binary data based on 64 printable characters) encoding and other formats. For link-type elements that contain background images, the background image in the link can be first extracted by parsing the composite style through the CSSOM interface, etc., and then the image content of the background image can be obtained as the visual data content, and so on.

[0088] It should be noted that the visual search operation panel provided in the embodiment of the present application includes, in addition to a first area for placing images, a second area for placing links. Users can drag the link of the image from the current page or other information page to the second area. At this time, the URL (Uniform Resource Locator) information of the link can be displayed in the second area, and a search request can be initiated based on the URL. In addition, the embodiment of the present application can implement a cross-protocol data processing pipeline to be compatible with the unified parsing of multiple resource formats such as Base64, Blob (Binary LargeObject, representing a large object of binary type), and URL, so as to improve the recognition coverage, support more diverse image sources, and ensure that users can successfully initiate visual searches in different scenarios.

[0089] It should also be noted that, in a specific implementation, the user can drag a target element to the target area of ​​the operation panel, and then initiate a search request based on the visual data content of the target element. Alternatively, in the case where the submission of the search request is triggered by the second event, the user can also drag multiple target elements to the target area of ​​the operation panel, and then trigger the submission of the search request by clicking the operation option. Among them, the logical relationship between the multiple target elements can be pre-configured. For example, the relationship between multiple different target elements can be defaulted to a "logical or" relationship, or a "logical and" relationship, etc. That is to say, after perceiving the aforementioned second event, the search request can be generated and submitted based on the visual data content corresponding to the multiple target elements, and the pre-configured logical relationship between the multiple target elements.

[0090] In summary, through the embodiments of the present application, after the visual search function of the search component in the product information page is loaded, the perception of the element drag event can be started. After the element drag event is perceived, if the dragged target element is related to the visual data, the operation panel corresponding to the visual search function of the search component can be automatically activated. Afterwards, after the target element is dragged to the target area in the operation panel, a search request for product search based on the visual data content corresponding to the target element can be submitted. In this way, since the drag interaction mechanism is introduced in the visual search scenario, 5 or even more click operations in the traditional visual search process can be reduced to one drag and drop, and the time it takes for the user to complete a visual search action is reduced from minutes to seconds. Therefore, the user experience can be improved, the complexity of the operation can be reduced, and the probability of intention interruption due to operation failure can be reduced.

[0091] Example 2

[0092] In the above-mentioned embodiment 1, mainly for the scenario of product search, the specific implementation scheme of initiating visual search based on drag interaction is introduced. In actual application, this scheme of initiating visual search can also be used in other scenarios, for example, for websites or applications that specialize in providing information search services, or in any information system, as long as the visual search function is required in the relevant web pages, the scheme provided by the embodiment of this application can be used to achieve a more efficient visual search initiation process. Therefore, this embodiment 2 also provides a method for initiating a visual search request, see Figure 4 , the method may include:

[0093] S401: After the visual search function of the search component in the information page is loaded, the perception of the element drag event is started;

[0094] S402: After sensing an element drag event, if the dragged target element is related to the visual data, activating an operation panel corresponding to the visual search function;

[0095] S403: After the target element is dragged to the target area in the operation panel, a search request for information search based on the visual data content corresponding to the target element is triggered.

[0096] Example 3

[0097] The above embodiment mainly introduces the solution provided by the embodiment of the present application from the perspective of submitting a search request. In this embodiment three, a visual search method is also provided. Figure 5 , the method may specifically include:

[0098] S501: After the visual search function of the search component in the information page is loaded, the perception of the element drag event is started;

[0099] S502: After sensing an element drag event, if the dragged target element is related to the visual data, activating an operation panel corresponding to the visual search function;

[0100] S503: After the target element is dragged to the target area in the operation panel, a search request for information search based on the visual data content corresponding to the target element is submitted to the server, so that the server processes the search request and obtains search results;

[0101] S504: Display the search results returned by the server.

[0102] Among them, how the server obtains search results based on the visual data content in the search request is not the focus of the embodiments of this application, so it will not be described in detail here.

[0103] For the parts not described in detail in Examples 2 and 3, please refer to the description in Example 1 and other parts of this specification, and no further details will be given here.

[0104] It should be noted that the embodiments of the present application may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described herein within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).

[0105] Corresponding to the aforementioned embodiment 1, this embodiment of the present application further provides a device for initiating a visual search request, which may include:

[0106] A first event perception initiation unit is configured to initiate perception of element drag events after the visual search function of the search component on the product information page is loaded;

[0107] a first operation panel activation unit, configured to activate an operation panel corresponding to the visual search function after sensing an element drag event and if the dragged target element is related to the visual data;

[0108] The first search request submitting unit is configured to submit a search request for searching for commodities based on the visual data content corresponding to the target element after the target element is dragged to the target area in the operation panel.

[0109] The first operation panel activation unit may be specifically used for:

[0110] After sensing an event in which a target page element in the product information page is dragged, it is determined whether the target page element is related to visual data. If so, an operation panel corresponding to the visual search function of the search component is activated.

[0111] Specifically, whether the target page element is related to the visual data can be determined in the following ways:

[0112] Determine whether the target page element is an image element. If so, determine the image content of the image element as the visual data content corresponding to the target element.

[0113] Alternatively, whether the target page element is related to the visual data is determined by:

[0114] Determine whether the target page element is a link-type element containing a background image. If so, extract the background image of the link-type element by parsing the composite style information of the link-type element, so as to determine the image content of the extracted background image as the visual data content corresponding to the target element.

[0115] Specifically, the content area of ​​the product information page may include multiple resource locations;

[0116] The link element including a background image includes: a product details page link in the form of an information card displayed in the resource position, and the information card includes the background image of the link.

[0117] In another way, whether the target page element is related to the visual data can be determined by the following method:

[0118] It is determined whether the target page element is a canvas element and a valid image exists in the canvas element. If so, the image content of the valid image in the canvas element is used as the visual data content corresponding to the target element.

[0119] In addition, the first operation panel activation unit can be specifically used to:

[0120] Upon sensing an event of dragging a page element from another information page into the product information page, if the dragged page element is related to visual data, activating an operation panel corresponding to a visual search function of the search component in the product information page;

[0121] The other information pages include other information pages in the same website to which the product information page belongs, or information pages on other websites.

[0122] Alternatively, the first operation panel activation unit may further be used to:

[0123] After sensing an event of dragging a target element in a local folder of the terminal device to the information page, if the dragged element is related to visual data, an operation panel corresponding to the visual search function is activated.

[0124] In addition, the device may further include:

[0125] The visual effect providing unit is used to generate a thumbnail of the dragged element after sensing the element drag event, and provide a visual effect that the thumbnail follows the movement of the cursor.

[0126] Specifically, the search request submitting unit may be used to:

[0127] After a first event of the target element being dragged to a target area in the operation panel is sensed, submission of the search request is triggered by the first event.

[0128] Alternatively, the search request submitting unit may be specifically configured to:

[0129] After sensing a second event in which the target element is dragged to a target area in the operation panel and a target operation option in the operation panel is operated, the second event triggers submission of the search request.

[0130] Specifically, there may be multiple target elements dragged to the target area in the operation panel;

[0131] At this time, the search request submitting unit may be specifically configured to:

[0132] After the second event is perceived, the search request is generated and submitted according to the visual data contents corresponding to the multiple target elements and the pre-configured logical relationship between the multiple target elements.

[0133] Corresponding to the second embodiment, this embodiment of the present application further provides a device for initiating a visual search request, which may include:

[0134] A second event perception initiation unit is configured to initiate perception of element drag events after the visual search function for searching components on the information page is loaded;

[0135] a second operation panel activation unit, configured to activate an operation panel corresponding to the visual search function after sensing an element drag event and if the dragged target element is related to the visual data;

[0136] The second search request submission unit is used to trigger submission of a search request for information search based on the visual data content corresponding to the target element after the target element is dragged to the target area in the operation panel.

[0137] Corresponding to the third embodiment, the embodiment of the present application further provides a visual search device, which may include:

[0138] An event-aware startup unit, which is used to start the awareness of element drag events after the visual search function for searching components in the information page is loaded;

[0139] An operation panel activation unit, configured to activate an operation panel corresponding to the visual search function after sensing an element drag event and if the dragged target element is related to the visual data;

[0140] a search request submitting unit, configured to submit, after a target element is dragged to a target area within the operation panel, a search request for information search based on the visual data content corresponding to the target element to a server, so that the server processes the search request and obtains search results;

[0141] The search result display unit is used to display the search results returned by the server.

[0142] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.

[0143] And an electronic device comprising:

[0144] one or more processors; and

[0145] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the aforementioned method embodiments.

[0146] A computer program product includes a computer program / computer executable instructions, which implement the steps of the method described in the above method embodiment when executed by a processor in an electronic device.

[0147] in, Figure 6 The electronic device architecture is shown as an example, and may include a processor 610, a video display adapter 611, a disk drive 612, an input / output interface 613, a network interface 614, and a memory 620. The processor 610, the video display adapter 611, the disk drive 612, the input / output interface 613, the network interface 614, and the memory 620 may be communicatively connected via a communication bus 630.

[0148] Among them, the processor 610 can be implemented by a general-purpose CPU (Central Processing Unit, processor), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., to execute relevant programs to implement the technical solutions provided in this application.

[0149] The memory 620 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 620 can store an operating system 621 for controlling the operation of the electronic device 600, and a basic input and output system (BIOS) for controlling the low-level operations of the electronic device 600. In addition, a web browser 623, a data storage management system 624, and a visual search processing system 625, etc. can also be stored. The above-mentioned visual search processing system 625 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided by the present application is implemented by software or firmware, the relevant program code is stored in the memory 620 and is called and executed by the processor 610.

[0150] The input / output interface 613 is used to connect to input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0151] The network interface 614 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0152] The bus 630 comprises a pathway for transmitting information between the various components of the device (eg, the processor 610 , the video display adapter 611 , the disk drive 612 , the input / output interface 613 , the network interface 614 , and the memory 620 ).

[0153] It should be noted that although the above device only shows a processor 610, a video display adapter 611, a disk drive 612, an input / output interface 613, a network interface 614, a memory 620, a bus 630, etc., in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may also include only the components necessary to implement the solution of the present application, and does not necessarily include all the components shown in the figure.

[0154] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application or certain parts of the embodiments.

[0155] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0156] The above is a detailed introduction to the visual search request, visual search method, and electronic device provided by this application. Specific examples are used in this article to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting this application.

Claims

1. A method for initiating a visual search request, characterized in that: include: After the visual search function of the search component on the product information page is loaded, the perception of element drag events is enabled; After sensing an element drag event, if the dragged target element is related to the visual data, activating an operation panel corresponding to the visual search function; After the target element is dragged to the target area in the operation panel, a search request for searching for products based on the visual data content corresponding to the target element is submitted.

2. The method according to claim 1, characterized in that After sensing the element drag event, if the dragged element is related to the visual data, activating the operation panel corresponding to the visual search function of the search component includes: After sensing an event in which a target page element in the product information page is dragged, it is determined whether the target page element is related to visual data. If so, an operation panel corresponding to the visual search function of the search component is activated.

3. The method according to claim 2, characterized in that Determining whether the target page element is related to the visual data includes: Determine whether the target page element is an image element. If so, determine the image content of the image element as the visual data content corresponding to the target element.

4. The method according to claim 2, characterized in that Determining whether the target page element is related to the visual data includes: Determine whether the target page element is a link-type element containing a background image. If so, extract the background image of the link-type element by parsing the composite style information of the link-type element, so as to determine the image content of the extracted background image as the visual data content corresponding to the target element.

5. The method according to claim 4, characterized in that The content area of ​​the product information page includes multiple resource locations; The link element including a background image includes: a product details page link in the form of an information card displayed in the resource position, and the information card includes the background image of the link.

6. The method according to claim 2, characterized in that Determining whether the target page element is related to the visual data includes: It is determined whether the target page element is a canvas element and a valid image exists in the canvas element. If so, the image content of the valid image in the canvas element is used as the visual data content corresponding to the target element.

7. The method according to claim 1, characterized in that After sensing the element drag event, if the dragged element is related to the visual data, activating the operation panel corresponding to the visual search function of the search component includes: Upon sensing an event of dragging a page element from another information page into the product information page, if the dragged page element is related to visual data, activating an operation panel corresponding to a visual search function of the search component in the product information page; The other information pages include other information pages in the same website to which the product information page belongs, or information pages on other websites.

8. The method according to claim 1, characterized in that After sensing the element drag event, if the dragged element is related to the visual data, activating the operation panel corresponding to the visual search function includes: After sensing an event of dragging a target element in a local folder of the terminal device to the information page, if the dragged element is related to visual data, an operation panel corresponding to the visual search function is activated.

9. The method according to any one of claims 1 to 8, characterized in that Also includes: After the element drag event is sensed, a thumbnail of the dragged element is generated, and a visual effect of the thumbnail following the movement of the cursor is provided.

10. The method according to any one of claims 1 to 8, characterized in that After the target element is dragged to the target area in the operation panel, submitting a search request for searching for products based on the visual data content corresponding to the target element includes: After a first event of the target element being dragged to a target area in the operation panel is sensed, submission of the search request is triggered by the first event.

11. The method according to any one of claims 1 to 8, characterized in that After the target element is dragged to the target area in the operation panel, submitting a search request for searching for products based on the visual data content corresponding to the target element includes: After sensing a second event in which the target element is dragged to a target area in the operation panel and a target operation option in the operation panel is operated, the second event triggers submission of the search request.

12. The method according to claim 11, characterized in that There are multiple target elements dragged to the target area in the operation panel; The submitting of the search request triggered by the second event includes: After the second event is perceived, the search request is generated and submitted according to the visual data contents corresponding to the multiple target elements and the pre-configured logical relationship between the multiple target elements.

13. A method for initiating a visual search request, characterized in that: include: After the visual search function of the search component in the information page is loaded, the perception of element drag events is enabled; After sensing an element drag event, if the dragged target element is related to the visual data, activating an operation panel corresponding to the visual search function; After the target element is dragged to the target area in the operation panel, a search request for information search based on the visual data content corresponding to the target element is submitted.

14. A visual search method, characterized in that: include: After the visual search function of the search component in the information page is loaded, the perception of element drag events is enabled; After sensing an element drag event, if the dragged target element is related to the visual data, activating an operation panel corresponding to the visual search function; After the target element is dragged to the target area in the operation panel, a search request for information search based on the visual data content corresponding to the target element is submitted to the server, so that the server processes the search request and obtains search results; The search results returned by the server are displayed.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 14 are implemented.

16. An electronic device, characterized in that: include: one or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 14.

17. A computer program product comprising a computer program / computer executable instructions, characterized in that When the computer program / computer executable instructions are executed by a processor in an electronic device, the steps of the method according to any one of claims 1 to 14 are implemented.

Images