Method for initiating visual search request, and visual search method and electronic device

By sensing element drag events after the visual search function is loaded, automatically activating the operation panel and submitting a search request, the problems of complexity and failure in visual search operations are solved, and an efficient and simplified visual search process is achieved.

WO2026232994A1PCT designated stage Publication Date: 2026-11-12HANGZHOU ALIBABA INT INTERNET IND CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2026/084644
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-07
Filing Date
2026-03-19
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

When using visual search, users face complex operation paths and are prone to interruptions due to operation failures. In particular, text search is ineffective in cross-border scenarios, resulting in a complex user process and a high failure rate.

Method used

By enabling drag-and-drop event awareness after the visual search function is loaded, the visual search operation panel is automatically activated, and a search request is submitted after the target element is dragged to the target area, simplifying the operation to be completed in one drag-and-drop operation.

Benefits of technology

The traditional multi-step click operation of visual search is reduced to a single drag, improving user experience, reducing operation complexity and the probability of interruption of intent, and reducing search time from minutes to seconds or even milliseconds.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2026084644_12112026_PF_FP_ABST
    Figure CN2026084644_12112026_PF_FP_ABST
Patent Text Reader

Abstract

Disclosed in the embodiments of the present disclosure are a method for initiating a visual search request, and a visual search method and an electronic device. The method comprises: after a visual search function of a search component on a commodity information page has been loaded, starting the sensing of an element drag event; after an element drag event is sensed, if a dragged target element is related to visual data, activating an operation panel corresponding to the visual search function; and after the target element is dragged to a target area within the operation panel, submitting a search request for performing commodity search on the basis of visual data content corresponding to the target element. By means of the embodiments of the present disclosure, the operational complexity of initiating a visual search request can be reduced, and the probability of intent interruption caused by operation failures can be lowered.
Need to check novelty before this filing date? Find Prior Art

Description

Initiating a visual search request, visual search methods, and electronic devices

[0001] This disclosure claims priority to Chinese Patent Application No. 202510589328.X, filed on May 7, 2025 with the China National Intellectual Property Administration, entitled “Initiating a Visual Search Request, Visual Search Method and Electronic Device”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of information search technology, and in particular to initiating visual search requests, visual search methods, and electronic devices. Background Technology

[0003] In information service systems, besides searching for information by entering keywords (which can be called "text search"), searching for information by entering visual data (including images, videos, and other types of visual data) (which can be called "visual search" for ease of description) is also a common method used by users. For example, in product information service systems, it is very common to search for products by entering images, especially in cross-border scenarios. Global B-type users (Business Users, referring to individuals who make decisions, purchase, or use products / services on behalf of enterprises or organizations in business scenarios, who usually do not participate in transactions as individual consumers but act based on the interests and needs of enterprises) may find that text search results are not as good as expected due to differences in regional culture and multiple languages ​​during the procurement and sourcing process. Therefore, B-type users will make more use of visual search functions during procurement and sourcing, such as using existing product images to initiate visual searches to find the same or similar products.

[0004] However, users may encounter issues such as lengthy operation paths and a certain probability of failure when using the visual search function. For example, before initiating a search using images, users first need to obtain specific visual data, such as images. This visual data itself may also come from the internet. For instance, when browsing a product on an e-commerce platform, a user might need to search for similar or identical products based on the product image to compare across multiple sellers. Therefore, using the visual search function might involve the following steps: downloading product images from the internet to the local device and saving them - opening the visual search panel on the system page - selecting the product image from the local folder - uploading the product image. This process can be quite lengthy. For users unfamiliar with the operation process, it is too complex, and failure at any step can interrupt their search intent. For example, the downloaded image might not be found, or the download might even fail. For instance, some images may have links, and when users try to download an image by right-clicking, they might find that options like "Save image" or "Save image as" are not available in the right-click menu, preventing the image from being downloaded and thus hindering the search based on that image. Summary of the Invention

[0005] This disclosure provides a method for initiating a visual search request, a visual search method, and an electronic device, which can reduce the operational complexity of initiating a visual search request and reduce the probability of intent interruption due to operational failure.

[0006] This disclosure provides the following solutions:

[0007] A method for initiating a visual search request includes:

[0008] Once the visual search function of the search component on the product information page has finished loading, the system will start detecting element drag events.

[0009] After detecting an element drag event, if the dragged target element is related to the visual data, the operation panel corresponding to the visual search function is activated.

[0010] After the target element is dragged to the target area within the operation panel, a search request is submitted to search for products based on the visual data content corresponding to the target element.

[0011] Specifically, upon detecting an element drag event, if the dragged element is related to visual data, the operation panel corresponding to the visual search function of the search component is activated, including:

[0012] After detecting an event that a target page element in the product information page is dragged, it is determined whether the target page element is related to visual data. If so, the operation panel corresponding to the visual search function of the search component is activated.

[0013] The step of determining whether the target page element is related to visual data includes:

[0014] Determine whether the target page element is an image element. If it is, determine the image content of the image element as the visual data content corresponding to the target element.

[0015] The step of determining whether the target page element is related to visual data includes:

[0016] Determine whether the target page element is a link element containing a background image. If so, extract the background image of the link element by parsing its composite style information, so that the image content of the extracted background image can be determined as the visual data content corresponding to the target element.

[0017] The product information page includes multiple resource slots within its content area;

[0018] The link elements containing background images include: product detail page links displayed in the resource slot in the style of information cards, wherein the information card includes the background image of the link.

[0019] The step of determining whether the target page element is related to visual data includes:

[0020] Determine whether the target page element is a canvas element and whether there is a valid image within the canvas element. If so, use the image content of the valid image within the canvas element as the visual data content corresponding to the target element.

[0021] Specifically, upon detecting an element drag event, if the dragged element is related to visual data, the operation panel corresponding to the visual search function of the search component is activated, including:

[0022] After sensing an event that drags a page element from another information page into the product information page, if the dragged page element is related to visual data, the operation panel corresponding to the visual search function of the search component in the product information page is activated.

[0023] The other information pages include other information pages within the same website as the product information page, or information pages from other websites.

[0024] Specifically, upon detecting an element drag event, if the dragged element is related to visual data, the operation panel corresponding to the visual search function is activated, including:

[0025] Upon detecting an event where a target element from a local folder on a terminal device is dragged to the information page, if the dragged element is related to visual data, the operation panel corresponding to the visual search function is activated.

[0026] This also includes:

[0027] After detecting the element drag event, a thumbnail of the dragged element is generated, and the thumbnail is provided with a visual effect that follows the cursor movement.

[0028] The step of submitting a search request for product search based on the visual data content corresponding to the target element after the target element is dragged to the target area within the operation panel includes:

[0029] Upon sensing a first event that the target element has been dragged to the target area within the operation panel, the first event triggers the submission of the search request.

[0030] The step of submitting a search request for product search based on the visual data content corresponding to the target element after the target element is dragged to the target area within the operation panel includes:

[0031] Upon sensing a second event that the target element has been dragged to the target area within the operation panel and the target operation option within the operation panel has been operated, the second event triggers the submission of the search request.

[0032] Among them, there are multiple target elements that are dragged to the target area within the operation panel;

[0033] The submission of the search request triggered by the second event includes:

[0034] After sensing the second event, the search request is generated and submitted based on the visual data content corresponding to the multiple target elements and the pre-configured logical relationship between the multiple target elements.

[0035] A method for initiating a visual search request includes:

[0036] After the visual search function of the search component within the information page has finished loading, the awareness of element drag events will be activated.

[0037] After detecting an element drag event, if the dragged target element is related to the visual data, the operation panel corresponding to the visual search function is activated.

[0038] After the target element is dragged to the target area within the operation panel, a search request is submitted to search for information based on the visual data content corresponding to the target element.

[0039] A visual search method, comprising:

[0040] After the visual search function of the search component within the information page has finished loading, the awareness of element drag events will be activated.

[0041] After detecting an element drag event, if the dragged target element is related to the visual data, the operation panel corresponding to the visual search function is activated.

[0042] After the target element is dragged to the target area within the operation panel, a search request is submitted to the server to search for information based on the visual data content corresponding to the target element, so that the server can process the search request and obtain the search results.

[0043] The search results returned by the server are displayed.

[0044] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of any of the preceding methods.

[0045] An electronic device, comprising:

[0046] One or more processors; and

[0047] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the preceding descriptions.

[0048] A computer program product includes a computer program / computer executable instructions that, when executed by a processor in an electronic device, implement the steps of any of the preceding methods.

[0049] Based on the specific embodiments provided in this disclosure, the following technical effects are disclosed:

[0050] In this embodiment, after the visual search function of the search component on the product information page is loaded, the detection of element drag events can be initiated. Upon detecting an element drag event, if the dragged target element is related to visual data, the operation panel corresponding to the visual search function of the search component can be automatically activated. Then, after the target element is dragged to the target area within the operation panel, a search request for products based on the visual data content corresponding to the target element can be submitted. In this way, by introducing a drag-and-drop interaction mechanism into the visual search scenario, the five or more click operations required in traditional visual search can be reduced to a single drag operation. The time for a user to complete a visual search action is reduced from minutes to seconds. Therefore, user experience is improved, operational complexity is reduced, and the probability of intention interruption due to operation failure is decreased.

[0051] Of course, implementing any product of this disclosure does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 is a schematic diagram of system interaction provided in an embodiment of this disclosure;

[0054] Figure 2 is a flowchart of the first method provided in an embodiment of this disclosure;

[0055] Figure 3 is a schematic diagram of the interface provided in an embodiment of this disclosure;

[0056] Figure 4 is a flowchart of the second method provided in an embodiment of this disclosure;

[0057] Figure 5 is a flowchart of the third method provided in an embodiment of this disclosure;

[0058] Figure 6 is a schematic diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0059] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure are within the scope of protection of this disclosure.

[0060] In this embodiment, a solution is provided to enable users to initiate visual search requests more efficiently. This solution extends the functionality of the search component on the information page. Specifically, if the search component has a visual search function, it can initiate the detection of element drag events after the visual search function is loaded. Users do not need to download images or other visual data in advance, nor do they need to manually open the operation panel corresponding to the visual search function of the search component (hereinafter referred to as the visual search panel). Instead, they can trigger the visual search process by directly dragging the target element. Correspondingly, after detecting the element drag event, the search component can determine whether the dragged target element is related to the visual data. If it is, it can automatically activate the visual search panel of the search component. The user can then drag the target element to the target area within the visual search panel. At this point, the search component can detect the completion of the drag event and trigger the submission of a search request based on the visual data content corresponding to the target element. In this way, the five or more clicks required in the traditional image search process can be reduced to a single drag operation, thus improving efficiency and reducing the likelihood of interrupted search intent.

[0061] Specifically, the source of the draggable target element can be varied. For example, it can originate directly from the information page containing the search component. Pages with search components are often information aggregation pages, such as the client homepage, store page, or event page in a product information service system. These pages typically contain various types of elements, including those related to visual data. Therefore, users can initiate a visual search process by dragging and dropping this element. For instance, a client homepage usually provides a recommended product information stream, including multiple resource slots. Each resource slot can display product links in the form of information cards, which often include product images. Although the information card is essentially a link, and the product image may serve as the link's background image, users can still roughly determine their interest in the product based on the displayed image and may have a need to search for similar products based on that image. Therefore, users can initiate a visual search process by dragging and dropping this product image. Additionally, the page may include image elements, which users can drag and drop to initiate a visual search. Furthermore, the page may contain canvas elements, which allow images to be generated directly on the webpage. Therefore, if such canvas elements exist and contain valid image content, this image content can also be dragged and dropped to initiate a visual search.

[0062] In the scenario described above, where the user directly drags and drops the target element within the information page where the search component is located, the visual search panel of the search component will be automatically activated as long as the component detects that the user has begun dragging the target element and that the target element is related to the visual data. For example, if the user clicks on an element with the mouse and moves the mouse while holding down the left mouse button, the search component can detect the dragging operation on the target element and then activate the visual search panel.

[0063] Another source of the target element can be other information pages. For example, the information page with the search component provided in this embodiment is page A. In the aforementioned way, the user can directly select a target element from page A and drag it. In another way, the user can also select a page element from page B and drag it to page A. In this case, the search component in page A can also detect the element dragging operation, and if the dragged target element is related to visual data, it can also activate the visual search panel. The user can submit the visual search request by placing the dragged element in the target area of ​​the visual search panel.

[0064] For example, suppose page A is the homepage of a product information service system. While browsing page A through a web browser, a user is also browsing page B. At this time, page A and page B will be two different tabs in the web browser, and the browser's tab bar will display tab options for both pages. While browsing page B, if the user finds an image or other element related to a product and wants to search for the same or similar product in the product information service system, according to existing technology, the user would first need to download the image from page B to their local computer, then open the visual search panel on page A, select the image from their local folder, and then upload the image, among other steps. However, in this embodiment, the user only needs to drag and drop the image element from page B to page A. For example, while dragging an image element, the user can first move the mouse pointer to the tab option for page A in the tab bar, then move it to the browser's page display area, which will then give page A focus. At this point, the search component on page A will be able to detect the element dragging event and determine whether the dragged element is related to visual data. For example, if it determines that the dragged element is an image, it can automatically activate the visual search panel. Users can drag elements to the target area of ​​this visual search panel, thereby triggering the submission of a visual search request. Page B and page A can be different pages within the same website, or they can belong to different websites; this is not a limitation.

[0065] Another source of target elements can be content from a local folder on the user's terminal device. For example, assuming the information page with the search component provided in this embodiment is page A, when page A is open, the user can drag and drop an element such as an image from a local folder on the terminal device onto page A. At this time, the search component on page A can detect the drag event and determine whether the dragged element is related to visual data. If so, it can automatically activate the visual search panel of the search component on page A. The user can then submit a visual search request by placing the dragged element into the target area of ​​the visual search panel.

[0066] In summary, the embodiments of this disclosure provide a drag-and-drop interaction mechanism for visual search scenarios. By listening to the drag-and-drop event stream, the user operation path can be simplified, reducing the multi-step process of traditional image search (saving image - opening panel - selecting file - uploading) to a single drag-and-drop operation. This improves user experience, reduces operational complexity, minimizes interruptions caused by operation failures, and reduces visual search time from minutes to seconds or even milliseconds.

[0067] From a system architecture perspective, the embodiments of this disclosure mainly improve the visual search function of the search component in the information page. The improved search component can be applied to various specific information pages. In other words, any information page using the search component provided in the embodiments of this disclosure can realize the function of quickly initiating a visual search provided in the embodiments of this disclosure. Specifically, after a user opens an information page (e.g., the client homepage of a product information service system), the components on the page are automatically loaded, including the search component. If the search component supports the visual search function, it will also load that function. Specifically, taking the method of performing element drag-and-drop interaction within the information page where the search component is located as an example, referring to Figure 1, the specific interaction process can include: after the information page is opened and the visual search function is loaded, the perception of element drag-and-drop events can be initiated. Subsequently, if the user needs to initiate a visual search based on a browsed element during the browsing process, they can perform a specific element drag-and-drop operation. The search component can perceive this operation and automatically activate the visual search panel. After the user drags the specific element to the target area of ​​the panel, the search component can perceive the drag completion event, and therefore, can automatically trigger the submission of the visual search request. The specific search request is submitted to the server, which performs the visual search, including retrieving search results from databases such as the product information database that match the input visual data content, and then returning them to the client for display.

[0068] The specific implementation schemes provided by the embodiments of this disclosure will be described in detail below.

[0069] Example 1

[0070] First, this embodiment provides a method for initiating a visual search request for a product search application in a product information service system. The executing entity of this method can be a search component in a product information page (including the client homepage, store page, event venue page, etc. of the product information service system). Referring to Figure 2, the method may specifically include:

[0071] S201: After the visual search function of the search component on the product information page is loaded, start the detection of element drag events.

[0072] In this embodiment of the disclosure, since the search process is initiated by dragging and dropping, the perception of element dragging events can be started after the visual search function of the search component on the page is loaded. This allows users to initiate the search process by dragging and dropping elements without having to manually open the visual search panel in advance.

[0073] Since many common web design languages ​​support drag-and-drop events, we can detect these events to determine if a user has performed a drag-and-drop operation. For example, JavaScript defines drag and drop events, which represent the process of a user selecting a draggable element with the mouse, dragging it to a droppable element, and releasing the mouse button to place the element. Specific events include dragstart (mouse button pressed, triggered when an element begins to be dragged), draggenter (mouse enters an element, triggered when the dragged element moves over a droppable element), dragover (mouse over an element, triggered when the dragged element moves over a droppable element), drop (mouse released, triggered when the dragged element is on a droppable element and the mouse is released), dragend (event triggered when the drag is complete), and so on. In specific implementation, in this embodiment of the disclosure, some or all of the above-mentioned events can be sensed to sense the event of a specific element being dragged.

[0074] S202: After sensing an element drag event, if the dragged target element is related to visual data, the operation panel corresponding to the visual search function is activated.

[0075] After detecting a specific element drag event, the system can first determine whether the target element is related to visual data. For example, is it an image element, a link element with a background image, etc.? If so, the corresponding operation panel of the visual search function of the search component can be automatically activated.

[0076] As mentioned earlier, the dragged element can come from various sources. In one scenario, it can be a target element within the content area of ​​the page where the search component is located. In this case, after detecting the dragging event of a target page element within the content area of ​​the product information page, it can be determined whether the target page element is related to visual data. If so, the operation panel corresponding to the visual search function of the search component is activated.

[0077] For example, as shown in Figure 3, which is a product information page, it includes the improved search component provided in this embodiment. Point 33 shows the visual search operation panel of the search component, which is closed by default. If a user performs a drag operation on the element shown at point 31, the operation panel will be activated, as shown at point 33 in Figure 3. During the dragging process, a thumbnail of the dragged target element can be displayed along the dragging trajectory; for example, point 32 in Figure 3 shows a thumbnail at a point on the dragging trajectory.

[0078] To determine whether a draggable target page element is relevant to visual data, an intelligent content recognition engine can be built to identify the type of the draggable target element through multimodal feature extraction. Multimodal features refer to the characteristics of information from multiple different modalities, which may include DOM (Document Object Model) structure analysis, CSSOM (CSS Object Model, CSS is an abbreviation for Cascading Style Sheets) style parsing, Canvas pixel detection, SVG (Scalable Vector Graphics) parsing, and so on.

[0079] In practice, it can be determined whether the target page element is an image element. If it is, the image content of the image element can be directly determined as the visual data content corresponding to the target page element.

[0080] Alternatively, it can be determined whether the target page element is a link element containing a background image. If so, the background image of the link element can be extracted by parsing its composite style information (CSSOM styles, etc.). Then, the image content of the extracted background image can be determined as the visual data content corresponding to the target element. In practical applications, it is quite common for the dragged target page element to be a link element. In this embodiment, even if what is dragged is a link (i.e., not directly an image element such as a picture), as long as the specific link has a background image (e.g., a product image displayed in a product information card-style detail page link), this background image can be extracted, and the dragged element can be determined as an element related to visual data. Consequently, the visual search panel can be automatically activated. For example, in a specific implementation, since the specific search component is located within the product information page, and the content area of ​​the product information page may include multiple resource slots, each of which can display a product detail page link in the form of an information card, the product image displayed in the specific information card is actually the background image of the specific link. In the traditional approach, if a user needs to search for a product based on its image, they might need to download the image. However, since it's actually a link, there might be issues like the image failing to download, making it difficult to continue the product search based on the image. In this embodiment, however, the user only needs to drag and drop the product image. The search component can detect this drag event and extract the background image from it, then submit a visual search request based on that background image.

[0081] Furthermore, some product information pages may include canvas elements to support image editing within the webpage. For example, the "Crop Image" area shown on the left side of Figure 3 includes a canvas element, allowing users to crop product images according to their specific needs. Correspondingly, users may also have the need to search for valid images based on the canvas element's values. Therefore, it's also possible to determine whether the target page element is a canvas element and whether a valid image exists within it. If so, the image content of the valid image within the canvas element is used as the visual data content corresponding to the target element.

[0082] Besides using page elements within the information page where the search component is located as drag-and-drop objects, page elements from other information pages can also be used. In this case, after the search component on the current product information page detects the event of dragging a page element from another information page into the current product information page, it can also determine whether the dragged page element is related to visual data. If so, it can activate the operation panel corresponding to the visual search function of the search component on the product information page. The determination of the type of page element dragged from other pages can also be achieved through the aforementioned intelligent content recognition engine. Specifically, for elements dragged from other pages, only image elements can be dragged, or link elements containing background images can also be dragged. For the latter, since it is a page element from another information page, if it is difficult to extract the background image data of the link directly through composite style information analysis, the relevant image data content can be obtained by extracting information contained in the page the link points to.

[0083] Furthermore, target elements such as images from the device's local folder can also be dragged and dropped. In this case, the search component on the current product information page, upon detecting the dragging of a target element from the device's local folder onto the product information page, can also determine whether the dragged element is related to visual data. If so, it can activate the operation panel corresponding to the visual search function. The target elements located on the device's local folder can primarily be image files, etc.

[0084] Of course, in practice, the target element to be dragged can come from other sources, which will not be introduced one by one here.

[0085] Furthermore, upon detecting an element drag event, a thumbnail of the dragged element can be generated, and the thumbnail can be visually displayed to follow the cursor's movement. In other words, as mentioned earlier, during the dragging process, a thumbnail of the dragged element can be displayed along the drag path to provide real-time visual feedback on the dragging effect. Additionally, the reason for displaying a thumbnail rather than the original image size along the drag path is to avoid excessive obstruction of other content on the current product information page.

[0086] S203: After the target element is dragged to the target area within the operation panel, a search request is submitted to search for products based on the visual data content corresponding to the target element.

[0087] After a target element is dragged to the target area of ​​the visual search operation panel, a search request for products based on the visual data content corresponding to the target element can be submitted. This search request can be triggered in several ways. For example, in one way, the submission of the search request based on the visual data content corresponding to the target element can be automatically triggered based on the event of the drag-and-drop completion (referred to as the first event). Alternatively, in another way, an operation option such as "Submit" can be provided in the operation panel, allowing the user to trigger the submission of the search request by clicking this option after completing the drag-and-drop operation. In other words, for the latter case, the search request can be triggered by the second event after the target element is detected being dragged to the target area within the operation panel and the target operation option within the operation panel is activated.

[0088] The visual data content corresponding to the target element can be obtained through drag-and-drop APIs (Application Programming Interfaces) provided by JavaScript. For example, for image-type target elements, the information obtained can include the image filename, file type, and image content. The image content can be read as Base64 encoding (one of the encoding methods used to transmit 8-bit byte code, a method of representing binary data based on 64 printable characters). For link-type elements containing background images, the background image can be extracted from the link by parsing composite styles using interfaces such as CSSOM, and then the image content of the background image can be obtained as the visual data content, and so on.

[0089] It should be noted that the visual search operation panel provided in this embodiment includes not only a first area for placing images, but also a second area for placing links. Users can drag and drop image links from the current page or other information pages to this second area. In this case, the URL (Uniform Resource Locator) information of the link can be displayed in the second area, and a search request can be initiated based on the URL. Furthermore, this embodiment can implement a cross-protocol data processing pipeline to ensure unified parsing of various resource formats such as Base64, Blob (Binary Large Object), and URL, thereby improving recognition coverage, supporting more diverse image sources, and ensuring that users can successfully initiate visual searches in different scenarios.

[0090] Additionally, it should be noted that in the specific implementation, a user can drag a target element to the target area of ​​the operation panel and initiate a search request based on the visual data content of that target element. Alternatively, in the case of triggering the submission of the search request through the second event mentioned above, the user can also drag multiple target elements to the target area of ​​the operation panel and then trigger the submission of the search request by clicking the operation option. The logical relationship between multiple target elements can be pre-configured; for example, it can be assumed that multiple different target elements are related by a "logical OR" relationship or a "logical AND" relationship, etc. In other words, after sensing the aforementioned second event, the search request can be generated and submitted based on the visual data content corresponding to each of the multiple target elements and the pre-configured logical relationship between the multiple target elements.

[0091] In summary, through the embodiments of this disclosure, after the visual search function of the search component on the product information page is loaded, the perception of element drag events can be initiated. Upon sensing an element drag event, if the dragged target element is related to visual data, the operation panel corresponding to the visual search function of the search component can be automatically activated. Then, after the target element is dragged to the target area within the operation panel, a search request for products based on the visual data content corresponding to the target element can be submitted. In this way, by introducing a drag-and-drop interaction mechanism into the visual search scenario, the five or more click operations required in traditional visual search can be reduced to a single drag operation. The time for a user to complete a visual search action is reduced from minutes to seconds. Therefore, user experience can be improved, operational complexity reduced, and the probability of intention interruption due to operation failure decreased.

[0092] Example 2

[0093] In the aforementioned Embodiment 1, a specific implementation scheme for initiating a visual search based on drag-and-drop interaction was introduced, primarily targeting the scenario of product search. In practical applications, this scheme for initiating a visual search can also be used in other scenarios. For example, for websites or applications that specifically provide information search services, or in any information system, as long as the relevant webpage requires a visual search function, the scheme provided by this disclosure can be used to achieve a more efficient visual search initiation process. Therefore, Embodiment 2 also provides a method for initiating a visual search request. Referring to Figure 4, this method may include:

[0094] S401: After the visual search function of the search component in the information page has been loaded, start the awareness of element drag events;

[0095] S402: After sensing an element drag event, if the dragged target element is related to visual data, then activate the operation panel corresponding to the visual search function.

[0096] S403: After the target element is dragged to the target area within the operation panel, a search request is triggered to perform an information search based on the visual data content corresponding to the target element.

[0097] Example 3

[0098] The foregoing embodiments mainly introduced the solutions provided by the embodiments of this disclosure from the perspective of submitting search requests. In this third embodiment, a visual search method is also provided. Referring to Figure 5, the method may specifically include:

[0099] S501: After the visual search function of the search component in the information page is loaded, start the awareness of element drag events;

[0100] S502: After sensing the element drag event, if the dragged target element is related to the visual data, then activate the operation panel corresponding to the visual search function.

[0101] S503: After the target element is dragged to the target area in the operation panel, a search request is submitted to the server to search for information based on the visual data content corresponding to the target element, so that the server can process the search request and obtain the search results.

[0102] S504: Display the search results returned by the server.

[0103] The specific details of how the server obtains search results based on the visual data content in the search request are not the focus of this disclosure and will not be described in detail here.

[0104] For the parts of Embodiments 2 and 3 that are not described in detail, please refer to Embodiment 1 and other parts of this specification. They will not be repeated here.

[0105] It should be noted that the embodiments disclosed herein may involve the use of user data. In practical applications, user-specific personal data may be used in the scheme described herein within the scope permitted by applicable laws and regulations, provided that it complies with the applicable laws and regulations of the country in which it is located (e.g., with the user's explicit consent, with the user being properly notified, etc.).

[0106] Corresponding to the aforementioned Embodiment 1, this disclosure also provides an apparatus for initiating a visual search request, which may include:

[0107] The first event-aware activation unit is used to activate the awareness of element drag events after the visual search function of the search component in the product information page has been loaded.

[0108] The first operation panel activation unit is used to activate the operation panel corresponding to the visual search function if the dragged target element is related to the visual data after sensing the element dragging event.

[0109] The first search request submission unit is used to submit a search request for product search based on the visual data content corresponding to the target element after the target element is dragged to the target area in the operation panel.

[0110] Specifically, the first operation panel activation unit can be used for:

[0111] After detecting an event that a target page element in the product information page is dragged, it is determined whether the target page element is related to visual data. If so, the operation panel corresponding to the visual search function of the search component is activated.

[0112] Specifically, the following methods can be used to determine whether the target page element is related to the visual data:

[0113] Determine whether the target page element is an image element. If it is, determine the image content of the image element as the visual data content corresponding to the target element.

[0114] Alternatively, the following methods can be used to determine whether the target page element is related to the visual data:

[0115] Determine whether the target page element is a link element containing a background image. If so, extract the background image of the link element by parsing its composite style information, so that the image content of the extracted background image can be determined as the visual data content corresponding to the target element.

[0116] Specifically, the content area of ​​the product information page may include multiple resource slots;

[0117] The link elements containing background images include: product detail page links displayed in the resource slot in the style of information cards, wherein the information card includes the background image of the link.

[0118] Alternatively, the relevance of the target page element to the visual data can be determined using the following methods:

[0119] Determine whether the target page element is a canvas element and whether there is a valid image within the canvas element. If so, use the image content of the valid image within the canvas element as the visual data content corresponding to the target element.

[0120] In addition, the first operation panel activation unit can specifically be used for:

[0121] After sensing an event that drags a page element from another information page into the product information page, if the dragged page element is related to visual data, the operation panel corresponding to the visual search function of the search component in the product information page is activated.

[0122] The other information pages include other information pages within the same website as the product information page, or information pages from other websites.

[0123] Alternatively, the first operation panel activation unit can also be used for:

[0124] Upon detecting an event where a target element from a local folder on a terminal device is dragged to the information page, if the dragged element is related to visual data, the operation panel corresponding to the visual search function is activated.

[0125] Additionally, the device may also include:

[0126] A visual effects providing unit is used to generate a thumbnail of the dragged element after sensing the element drag event, and to provide a visual effect in which the thumbnail follows the cursor movement.

[0127] Specifically, the search request submission unit can be used for:

[0128] Upon sensing a first event that the target element has been dragged to the target area within the operation panel, the first event triggers the submission of the search request.

[0129] Alternatively, the search request submission unit can be specifically used for:

[0130] Upon sensing a second event that the target element has been dragged to the target area within the operation panel and the target operation option within the operation panel has been operated, the second event triggers the submission of the search request.

[0131] Specifically, there can be multiple target elements that are dragged to the target area within the operation panel;

[0132] At this point, the search request submission unit can specifically be used for:

[0133] After sensing the second event, the search request is generated and submitted based on the visual data content corresponding to the multiple target elements and the pre-configured logical relationship between the multiple target elements.

[0134] Corresponding to Embodiment 2, this disclosure also provides an apparatus for initiating a visual search request, which may include:

[0135] The second event-aware activation unit is used to activate the awareness of element drag events after the visual search function of the search component in the information page has been loaded.

[0136] The second operation panel activation unit is used to activate the operation panel corresponding to the visual search function if the dragged target element is related to the visual data after sensing the element dragging event.

[0137] The second search request submission unit is used to trigger the submission of a search request based on the visual data content corresponding to the target element after the target element is dragged to the target area in the operation panel.

[0138] Corresponding to Embodiment 3, this disclosure also provides a visual search device, which may include:

[0139] The event-aware startup unit is used to initiate the awareness of element drag events after the visual search function of the search component in the information page has been loaded.

[0140] The operation panel activation unit is used to activate the operation panel corresponding to the visual search function if the dragged target element is related to the visual data after sensing the element dragging event.

[0141] The search request submission unit is used to submit a search request to the server based on the visual data content corresponding to the target element after the target element is dragged to the target area in the operation panel, so that the server can process the search request and obtain the search results.

[0142] The search results display unit is used to display the search results returned by the server.

[0143] In addition, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0144] And an electronic device, comprising:

[0145] One or more processors; and

[0146] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0147] A computer program product includes a computer program / computer executable instructions that, when executed by a processor in an electronic device, implement the steps of the method described in the foregoing method embodiments.

[0148] Figure 6 illustrates the architecture of an electronic device, which may include a processor 610, a video display adapter 611, a disk drive 612, an input / output interface 613, a network interface 614, and a memory 620. The processor 610, video display adapter 611, disk drive 612, input / output interface 613, network interface 614, and memory 620 can communicate with each other via a communication bus 630.

[0149] The processor 610 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in this disclosure.

[0150] The memory 620 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 620 can store the operating system 621 for controlling the operation of the electronic device 600, and the basic input / output system (BIOS) for controlling the low-level operations of the electronic device 600. Additionally, it can store a web browser 623, a data storage management system 624, and a visual search processing system 625, etc. The aforementioned visual search processing system 625 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when implementing the technical solution provided by this disclosure through software or firmware, the relevant program code is stored in the memory 620 and is called and executed by the processor 610.

[0151] Input / output interface 613 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0152] Network interface 614 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0153] Bus 630 includes a pathway for transmitting information between various components of the device, such as processor 610, video display adapter 611, disk drive 612, input / output interface 613, network interface 614, and memory 620.

[0154] It should be noted that although the above-described device only shows the processor 610, video display adapter 611, disk drive 612, input / output interface 613, network interface 614, memory 620, bus 630, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the present disclosure, and need not include all the components shown in the figures.

[0155] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this disclosure can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this disclosure.

[0156] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0157] The foregoing has provided a detailed description of the visual search request initiation method, visual search method, and electronic device provided by this disclosure. Specific examples have been used to illustrate the principles and implementation methods of this disclosure. The descriptions of the embodiments above are merely for the purpose of helping to understand the methods and core ideas of this disclosure. Furthermore, those skilled in the art will recognize that, based on the ideas of this disclosure, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this disclosure.

Claims

1. A method for initiating a visual search request, wherein, include: Once the visual search function of the search component on the product information page has finished loading, the system will start detecting element drag events. After detecting an element drag event, if the dragged target element is related to the visual data, the operation panel corresponding to the visual search function is activated. After the target element is dragged to the target area within the operation panel, a search request is submitted to search for products based on the visual data content corresponding to the target element.

2. The method according to claim 1, wherein, Upon detecting an element drag event, if the dragged element is related to visual data, the operation panel corresponding to the visual search function of the search component is activated, including: After detecting an event that a target page element in the product information page is dragged, it is determined whether the target page element is related to visual data. If so, the operation panel corresponding to the visual search function of the search component is activated.

3. The method according to claim 2, wherein, The step of determining whether the target page element is related to visual data includes: Determine whether the target page element is an image element. If it is, determine the image content of the image element as the visual data content corresponding to the target element.

4. The method according to claim 2, wherein, The step of determining whether the target page element is related to visual data includes: Determine whether the target page element is a link element containing a background image. If so, extract the background image of the link element by parsing its composite style information, so that the image content of the extracted background image can be determined as the visual data content corresponding to the target element.

5. The method according to claim 4, wherein, The product information page includes multiple resource slots within its content area; The link elements containing background images include: product detail page links displayed in the resource slot in the style of information cards, wherein the information card includes the background image of the link.

6. The method according to claim 4 or 5, characterized in that, The step of extracting the background image of the link element by parsing the composite style information of the link element includes: The composite styles of the link elements are parsed using the CSSOM interface to obtain the attribute information of the background image. The image content of the background image is read using the drag-and-drop API provided by JS, and the image content is converted into Base64 encoding format or Blob format as the visual data content corresponding to the target element.

7. The method according to claim 2, wherein, The step of determining whether the target page element is related to visual data includes: Determine whether the target page element is a canvas element and whether there is a valid image within the canvas element. If so, use the image content of the valid image within the canvas element as the visual data content corresponding to the target element.

8. The method according to claim 2, characterized in that, The determination of whether the target page element is related to visual data specifically includes: Construct an intelligent content recognition engine to extract multimodal features from the target page elements; The multimodal feature extraction includes at least one of the following: DOM structure analysis, CSSOM style parsing, Canvas pixel detection, or SVG parsing of target page elements; The type of the target page element is determined based on the extracted multimodal features to determine whether it contains visual data.

9. The method according to any one of claims 1-8, wherein, Upon detecting an element drag event, if the dragged element is related to visual data, the operation panel corresponding to the visual search function of the search component is activated, including: After sensing an event that drags a page element from another information page into the product information page, if the dragged page element is related to visual data, the operation panel corresponding to the visual search function of the search component in the product information page is activated. The other information pages include other information pages within the same website as the product information page, or information pages from other websites.

10. The method according to any one of claims 1-8, wherein, Upon detecting an element drag event, if the dragged element is related to visual data, the operation panel corresponding to the visual search function is activated, including: Upon detecting an event where a target element from a local folder on a terminal device is dragged to the information page, if the dragged element is related to visual data, the operation panel corresponding to the visual search function is activated.

11. The method according to any one of claims 1 to 10, wherein, Also includes: After detecting the element drag event, a thumbnail of the dragged element is generated, and the thumbnail is provided with a visual effect that follows the cursor movement.

12. The method according to any one of claims 1 to 10, wherein, After the target element is dragged to the target area within the operation panel, a search request is submitted to perform a product search based on the visual data content corresponding to the target element, including: Upon sensing a first event that the target element has been dragged to the target area within the operation panel, the first event triggers the submission of the search request.

13. The method according to any one of claims 1 to 10, wherein, After the target element is dragged to the target area within the operation panel, a search request is submitted to perform a product search based on the visual data content corresponding to the target element, including: Upon sensing a second event that the target element has been dragged to the target area within the operation panel and the target operation option within the operation panel has been operated, the second event triggers the submission of the search request.

14. The method according to claim 13, wherein, The target element that is dragged to the target area within the operation panel can be multiple; The submission of the search request triggered by the second event includes: After sensing the second event, the search request is generated and submitted based on the visual data content corresponding to the multiple target elements and the pre-configured logical relationship between the multiple target elements.

15. The method according to claim 1, characterized in that, The submission of a search request based on the visual data content corresponding to the target element includes: The visual data content corresponding to the target element is uniformly parsed through a cross-protocol data processing pipeline; The cross-protocol data processing pipeline is compatible with at least one of Base64 encoding format, Blob format, and URL format; After successful parsing, the search request is generated and submitted based on the parsed visual data content to support image resources from different sources dragged and dropped from the current page, other information pages, or local folders on the terminal device.

16. A method for initiating a visual search request, wherein, include: After the visual search function of the search component within the information page has finished loading, the awareness of element drag events will be activated. After detecting an element drag event, if the dragged target element is related to the visual data, the operation panel corresponding to the visual search function is activated. After the target element is dragged to the target area within the operation panel, a search request is submitted to search for information based on the visual data content corresponding to the target element.

17. A visual search method, wherein, include: Once the visual search function of the search component within the information page has finished loading, it will start detecting element drag events. After detecting an element drag event, if the dragged target element is related to the visual data, the operation panel corresponding to the visual search function is activated. After the target element is dragged to the target area within the operation panel, a search request is submitted to the server to search for information based on the visual data content corresponding to the target element, so that the server can process the search request and obtain the search results. The search results returned by the server are displayed.

18. A computer-readable storage medium having a computer program stored thereon, wherein, When executed by a processor, the program performs the steps of the method described in any one of claims 1 to 17.

19. An electronic device, wherein, include: One or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 17.

20. A computer program product comprising a computer program / computer-executable instructions, wherein, When the computer program / computer-executable instructions are executed by a processor in an electronic device, they implement the steps of the method according to any one of claims 1 to 17.